How to get started with CRO experiments | Articles

If you are working for a company which is generating online conversions, you probably already looked for ways to generate more conversions on your website. This can be done thanks to Conversion Rate Optimization. CRO is a process that will help you improve the conversion rate of your website, by increasing the percentage of website visitors who convert. A conversion can be any action taken on your website, might it be a micro conversion (for example a registration) or a macro conversion (for example a purchase).

In this article we would like to focus on the experiments, which allow you to test changes on your website and see how they perform. We’ll present 4 useful guidelines to follow in order to make the most out of your experiments. 

1) Every experiment needs a hypothesis

The starting point should always be a hypothesis, which consists of identifying a problem on the website and suggesting a solution that could help solving that problem. In order to define a good hypothesis, you need to make this clear:

  • What do you want to test?
  • What is the expected outcome?
  • Why do you test that?

Example: by adapting the message on the homepage for users who come on the website for the first time, we would like to increase the conversion rate of account creation. Knowing that the membership rate is rather stable, if we manage to increase the number of account creations, we should be able to generate more memberships.

In order to formulate your hypothesis, don’t trust your feelings, trust your data. Once you collected enough data, make decisions based on actual numbers.

Some tools and techniques can help you collect data and gather insights such as: marketing analytics tools, heatmap analysis, feedback polls, asking people (who have never used your site before) to perform a task on your website to get their feedback, etc.

2) Test only one variable at a time

Don’t run too many tests at the same time on the same page. Testing one element at a time is the only way to get actionable feedback, as it will allow you to know what type of changes made what type of effect. If you test different elements at the same time, you won’t know at the end which element caused the potential change.

If you still want to test different elements at the same time, you should go for multivariate testing, not A/B testing. A MVT (multivariate test) tests variants of two or more elements simultaneously to see which combination creates the best outcome.

3) The goal must be well-defined

Goals measure the success or failure of an experiment. That’s why it’s important to clearly identify a key metric that is directly linked to the experiment and that will define whether it is a success or not. For example, if you are optimizing the call-to-action to encourage website visitors to create an account, the goal will likely be to increase the number of account creations, not the number of memberships. 

If there are different goals that you would like to monitor, just make sure that each experiment has only one primary goal. This primary goal will determine whether or not your experiment is successful, all others will be secondary goals. 

4) Let the experiment run for as long as it takes

Once your experiment starts, you might tend to constantly check how the test is running, but know that experiments can deliver quite volatile results in the beginning. Give it some time and wait to generate enough data before validating your results and taking conclusions. It is recommended to let your test run until it has reached a statistical significance of 95% or more.

Even if no winner has been found, the data might show that there was a clear loser. In this case it means that this change shouldn’t be implemented on the website. 


There is no secret recipe that will help get your number of conversions off the ground. This is because of context and because all websites are different, all website visitors have different needs and will interact in different ways. 

CRO is a never ending process of defining hypothesis, testing, measuring, learning and optimizing. At Semetis we believe that the best way to succeed is to never stop testing! Yes, a lot of tests will fail. But don’t get discouraged if your tests have a negative outcome, it’s actually an opportunity to learn and the insights you get from your tests can become future experiments. There are endless opportunities for things to test and the more you test, the more chances you have to find ways to improve the conversion rate on your website.

Happy testing!

publication author justine heeren
Justine Heeren

| LinkedinThis email address is being protected from spambots. You need JavaScript enabled to view it.


Get in touch

Semetis | Boulevard Saint-Lazare 4, 1210 Brussels - Belgium

Connect with us

Cookie Policy

This website uses cookies that are necessary to its functioning and required to achieve the purposes illustrated in the privacy policy. By accepting this OR scrolling this page OR continuing to browse, you agree to our privacy policy.