ICE Scoring

Managers, product owners, developers, designers, and businesspeople in general, usually have a lot of ideas about how to improve or change their product so that it could provide an even better experience to the customer. Hypotheses might flow like water and sometimes there could be so many of them that it’ll be really hard to understand where to start. Despite that they all might seem to be important, and on the basis of the fact that it’s hard to understand what to work on and which hypothesis to test first you might decide to test them all at the same time, immediately stop and take a deep breath. The choice must be made.

Which Hypothesis to Choose? ICE Scoring to the Rescue!

The answer is — prioritization. There are various systems of scoring that have been created in order to make prioritization of different tasks or hypotheses easier, and ICE is one of them.
ICE Scoring is a method of determining priorities, where I stands for ‘Impact’ — potential impact on the affected and target metrics; C stands for ‘Confidence’ — confidence in the success of the experiment; and E for ‘Ease’— ease of experiment organisation that can be used to check the hypothesis.

ICE'y Assessment

The title of ICE scoring consists of three factors that are taken into account when prioritizing: Impact, Confidence, and Ease. Each factor is evaluated on a 10-point scale.
Let’s say, you’ve come up with 50 hypotheses. All of them seem plausible and will apparently bring positive changes. But will they? Every hypothesis needs to be evaluated. Will the results of the hypothesis have any impact on the main metric? No? Yes? How big will the impact be? On a scale from 1 to 10, where 10 means that it will have a profound impact and 1 — no impact at all, you need to give the aspect of impact a grade. As a matter of fact, the following needs to be done not only by you, but by other members of the team as well. Everybody involved should express their opinion and give a grade in order to get a more subjective opinion.
Secondly, let’s be true, the level of your trust and belief in the presented hypotheses is different. Therefore, now you need to understand how sure you are that exactly this hypothesis will work. Moreover, are you the only one thinking that this hypothesis will bring your company millions of millions? Or maybe your team is confident about that, too? Maybe your competitors also do it this way? Maybe your customers humbly ask for that? The more people believe in it and the more customers ask for it, the higher the level of confidence in the hypothesis and its assessment grade are.
Finally, the last factor is ease. Nobody likes to spend precious time on something that is extremely difficult to implement and get no results in the end. Accordingly, this part deals with the fact how easy it will be to run a hypothesis. On a scale from 1 to 10, where 10 means it'll be very easy to run a hypothesis, assign a grade that you believe will best describe the situation.

Cards on the Table!

Let's say that we have evaluated how big the impact is, how high the confidence is, and how easy it’s to implement the hypothesis. What next? Next you need to sum the grades that you yourself and other members of the team have given every hypothesis. Every person in your growth team should write their opinion about the hypothesis! Then you calculate the score: sum up the numbers provided by the team to specific factors and compare the numbers provided to all hypotheses. This way you'll be able to set priorities. If there are hypotheses with similar scores — select the one you mostly believe in.

One More Thing

You have “calculated” the right hypothesis that will be tested. Now it’s very important that you clarify exactly what you will be doing with it: how you will test it, why you should test it, why do you believe that it will work, and what metric you will increase. Indicate which verification methods you will use, if you are planning to ask customers, or you will do A/B testing or corridor testing. Write down the following knowledge for other members of your company (existing or new ones) to get acquainted and be aware.
Finally, keep in mind that each hypothesis needs to have an owner — a person who is going to take care of it from the start to the very end. Of course, depending on the scale and importance of the hypothesis, there might be more than one owner, but at all events there should be a person(s) responsible for generating a hypothesis, collecting data, conducting an experiment, and drawing results.

Final Words

Usually, decisions about the actions that have to be implemented first and those that can be delegated or ignored are made on the basis of some factors, like urgency, due time, ability to do that task, etc. The same is with hypotheses: don’t rush to run all hypotheses at the same time, because you’ll fail. There’re no two ways about that. Choose wisely, take into account inner (how you feel) and outer factors (what influence it will have), and consider all aspects, even those that seem minor to you. This way you are sure to reach your goals and achieve success.