Verification Methods

Hypothesis, being an assumption that requires proof, seeks to reveal and explain why something happened or what might happen under certain conditions. It is a valuable tool that allows to verify if the company’s thoughts and expectations are correct before putting them into action. Accordingly, every hypothesis needs to be verified. How to do that? Keep on reading and there won’t be any questions left. Pay attention that there are only some of verification methods provided in the text, if you want to know more, please watch the video.

Hypothesis Verification Methods

Verifying a hypothesis is an important step, which involves proving an assumption right or wrong, with an understanding that it will have a significant impact on the development or growth of your company. After you have come up with a hypothesis and assume that something will happen if something else will be done, you need to test it using one or more (depending on the situation) hypothesis verification methods.

Research. As a matter of fact, sometimes it won’t take too long to verify or decline a hypothesis. All you need is to do some research, find and analyse existing data. Maybe someone has already done what you are planning to do and got some significant results that you can use and save yourself a lot of time. For example, earlier we generated a website in 3 seconds and decided to change it to 1 second. After some time we analysed the results and noticed that no metrics increased — the number of registrations and retention stayed the same. The following allowed to understand that there’s no use in changing the time to 0.5 seconds.
Analyse users’ behaviour. If the analysis shows that users don’t experience any problems in specific areas, then there is no need in running a hypothesis and spending time on something that won’t bring any positive changes.
Degradation testing. When you decide to do something that is technically difficult to implement, stop right there. What if you waste a lot of time and resources on something that is not worth it? Try doing the other way. For example, earlier we generated a website in 3 seconds and decided to change it to 0.5 seconds. It will take a lot of efforts to make our wish a reality, but, as a matter of fact, it won’t take too long to double or triple generation time. We change time to 10 seconds and notice that after making it worse nothing changes. Therefore, if making something worse doesn’t change metrics, we don’t need to run the hypothesis, because it means that providing better results will not increase our metrics as well.

Customer development. Indeed, if you want to get some information about your product or services you need to talk to your (potential) clients, carry out problem- or solution-based interviews. Therefore, the first and main thing you must do is understand whether there’s a problem that needs to be solved. If there’s no problem — no need to run a hypothesis. If the problem exists, but you have a solution and that solution perfectly solves the problem — no need to run a hypothesis. If there’s no solutions, then, of course, you need to find one and test it.
Surveys, ratings and feedback also provide information and insights whether there’s a need to change or improve something. If, for example, you have a hotel industry and all of your guests are incredibly satisfied with the services you are providing and rate you 5 stars already after one stay, then there’s no need to run a hypothesis as you are doing everything right.

UX-testing or in other words user experience testing is also about talking to the clients, but this method of verification is not about collecting opinions, but rather results and facts. When conducting UX-testing, you ask users to complete a specific task, without expressing their opinions. As a matter of fact, this method of verification requires to thoroughly think over the questions you are going to ask, because asking wrong questions will provide wrong results and your product might not be as successful as it should have been.
Hallway or corridor tests, when you, for example, print some pages or show something on the screen of your phone to random people going through the hallway. There’s nothing better to ask people who are not as interested in your product as you are and can provide real objective facts. By simply asking 5 people, you can actually statistically find 85% of problems.
Ghost button — when you want to test something without putting too much effort, you can always check using it. It’s also called a fake button or a placebo button, but the point is the same — it’s a button that does nothing. For example, you’re pondering over whether to add emails to your current services. You create a fake button that says “Pay €1 for emails”. After some time, you check your balance (afterwards all money is refunded) and can make a decision if there are enough clients ready to pay for that. As a matter of fact, this type of testing saves a lot of time and allows to understand 2 things: 1) sometimes the customer does need that service and 2) sometimes you put a button that nobody uses.

A/B/C testing is a way of testing when you compare three or two versions (A/B testing) of something to understand which of those versions is, looks or performs better. It is a great and the least complex method of verification, because you run three or two tests at the same time and get more results.
Multivariate testing is similar to split testing, but a higher number of variables are compared and, as a result, more information about the interaction of those variables is revealed.
False-positive readings (A/A test). Sometimes, testing can be difficult and provide bad data. Thus, sometimes it is necessary to test two identical versions of the same product to see if you get the same results. Experts typically run A/A tests for double checking, i.e. to understand if the experiment is fair.

Finally, you cannot doubt the necessity and importance of carrying out experiments. Some experiments are completely free of charge and can be implemented without putting any effort. For example, you’re thinking about introducing a new feature, but still have some doubts or money issues. Why not trick the audience into believing you’ve already developed it. You can simply advertise that your product already has that new feature and then track how people react to that message: if people click hoping to get that feature, develop it. If people remain indifferent, then you know the answer.

Final Words

Right decisions are the basis for success, while data and information are the basis for right decisions. In order for your business to grow and flourish for many years to come, you need to introduce changes and/or improvements, and accordingly, generate hypotheses, carry out tests, and conduct monthly/weekly/daily experiments, but most importantly you need to make the right and evidence-based decisions for every action you are about to take.