Experiments and Hypotheses: How to Systematically Grow Your Business

The core goal of literally any business is constant, long-term, and predictable growth, but achieving that is no easy task.

Regardless of the scale and size of a company, one particular approach is often successfully applied: growing by experimenting.

What does experimenting mean in business? Why and how does it work? Let’s dive in, step by step.

The Price of Uncertainty

Whether you're starting a new business from scratch or growing your existing one, there's always a scope of uncertainty.

You may not know how to promote your new product, how to price it, how different regions or countries may perceive your USP, or how the UX will perform. You may be in the process of finding new ways to improve the profits (increase ROI and retention, decrease CAC, etc.) of your existing business.

In every case, the set of unknown parameters will be unique, but it's always significant.

Every time you step into a new field with a new product, service, or marketing challenge, uncertainty comes on stage.

The only thing you can rely on is the actual practice-backed data:

• Which price points converted better
• Which variant of a landing page or paywall worked better
• Conversion rates
• How the LTV or CAC changed
• Feedback, and so on

The more experiments you run, the more actual data you receive, the more control you gain over your business, and the more you may rely on your success.

Conversely, the more you rely on your conjectures, the less accuracy you get, the more money you lose, and the higher your chances of failure.

Now let's see how it works.

How to Formulate Your Hypotheses

Every time you make a suggestion, it is a hypothesis.

"I think this is their core issue we may solve."

"I think these pricing packages may work well."

"I think this UX will be easy to understand."

Basically, a hypothesis is any suggestion not backed by data that is about to be tried and tested.

But it's important to formulate your thoughts in the right way so that a suggestion will turn into a real hypothesis to test.

How to formulate a hypothesis

Well-formed business hypotheses are:

• precise,
• discrete,
• and testable.

This means that in every single case you should be aiming to test one specific KPI, rather than an abstract mass of related metrics. For example:

In the formulation on the right, you can find a target audience and a specified KPI, which makes two out of three. Of course, you should understand exactly how you are going to test that.

What Is an Experiment?

An experiment is the actual process of testing your hypothesis and getting a real-life market response.

Before you start, it is important to understand a few factors:

• Test only one thing at a time. Of course, you may run a series of simultaneous experiments, but they should not interfere with each other (e.g., don't test a CTA and a title on the same page.)
• Not every experiment is supposed to be successful. It's not a win versus a fail; it's a process of checking whether every particular idea works, on a smaller scale. Most of your experiments may be unsuccessful, and that’s absolutely okay. But some of them will bring you results, and these will form the basis of your future marketing or improvements.
• Tests need time. Don't make any conclusions until you get enough impressions, because on a smaller scale, the data may be irrelevant, and preliminary conclusions may change several times.

Now let's see how it actually works.

Types of experiments

There are two main ways to run an experiment:

• A/B testing (split testing). You compare two different versions of what you test (e.g., one CTA versus another, or one color scheme versus another). These may be two new untested things or an improvement versus an old option.
• Multivariate testing. This type of test compares multiple versions within the same principle.

A/B tests are applicable to any kind of business, while multivariate tests are usually used by large companies with huge volumes of organic traffic in order to get trustworthy results.

How to run an experiment

At the very first step, you will formulate the experimentation process.

There are different approaches to that, but the overall principle is always the same:

• Write down the hypothesis you are going to test.
• Define the KPIs you are going to measure. There may be one core KPI and several additional ones.

For each KPI, write down:

• Current level (if applicable)
• Target level of improvement
• Define exactly how the test will be run.
• Specify the duration of the experiment, required resources, and further actions.

For example:

Experimenting should be a continuous cycle. This means you should always be generating new hypotheses and testing not only new ideas but also what has already been tested before.

There is no limit to perfection, which means every result can be improved.

How to Prioritize Your Hypotheses

In the beginning, you might struggle to ideate new things to test. But once you start, more and more ideas will come to your mind, as well as to the minds of your teammates. So, you may come up with a long list of ideas very quickly.

But how do you choose the right one to run first?

There's a universal formula used by many companies to prioritize hypotheses called ICE, which consists of:

Impact. What is the potential upside of a successful experiment? Assign scores according to the expected value of the impact, for example:

• 1 - 2 — very low (You likely won’t get much additional revenue.)
• 3 - 4 — low
• 5 - 6 — average
• 7 - 8 — high
• 9 - 10 — very high (Such hypotheses may bring you a lot of profit and impact your whole business or its valuable parts.)

Confidence. How likely is this experiment to succeed? Score it following the same principle. To help you in that, a very useful framework is the Gilad Confidence Meter. It can be easily adapted to your scale.  

Ease. How easy is it to run? If it requires many people or a lot of time and other resources, your "Ease" will be low, and vice versa. So, here it will be:

• 1 - 2 — very difficult (You will probably need to spend a lot of time and attract many people to run this as an experiment.)
• 3 - 4 — difficult
• 5 - 6 — average
• 7 - 8 — easy
• 9 - 10 — very easy (You can run this experiment at almost no cost with little effort.)  

Each parameter may be scored from 1 to 5, from 1 to 10, from 1 to 100, or by any other scale that is convenient for you and your team.

Want to read full article? Subscribe to my blog (it's free) 👇