Common A/B testing mistakes and how to avoid them

Posted on Nov 15th 2021 by Jason Lempiäinen

It can be frustrating to run an A/B testing campaign and not see the results you expect. Don't worry! Most of the time, simple adjustments can be applied to improve the outcomes of your test. I'll go through some of the most common A/B testing mistakes and how to prevent making them in the future.


Targeting, or the lack of it, is one of the most common mistakes made by game developers, and it can lead to less-than-desirable outcomes. Because the user base is not targeted, many consumers will click the ad link with the intention of never installing the game or app. This can be easily avoided by narrowing the target audience even a bit. Finally, if you're running an ad campaign on Facebook, remember to follow our step-by-step instructions because skipping a step can affect the outcome of your test. Step-by-step guide : 

Note: Use pixel in your ad campaign to improve your outcomes even more.

Look-a-like page

It's important to give users an authentic sense of the look-alike page. There are a few mistakes that could ruin the user's experience. The most crucial of these is a lack of screenshots on the look-alike page. To create an authentic first impression of the website, you should have at least 3 portrait screenshots on iOS and 4 on Android. It is recommended, however, that you use more screenshots in your test. It's also important to use only one orientation when selecting screenshots, e.g., when using portrait screenshots, don't include any landscape screenshots.

We put a lot of effort into making the look-a-like pages as authentic as possible at Geeklab. Customers can now build review templates that they can use on the look-alike page. We have seen that the users perform better on pages with reviews than on pages without reviews. You can create review templates and use them for any type of test, whether you're testing a real app or a game that's still in development. 


I could go on and on about the hypothesis, but to keep things brief, I'll list some of the most common errors and how to avoid them. Many tests show that the hypotheses are based on assumptions and randomly selected key selling points. As a result, even after determining the winner, you are still unsure as to why users downloaded the game and are unable to capitalize on the findings further. To get the most out of testing, it's critical to understand the fundamental concepts behind testing.

To avoid this : 

When testing key selling points, it is critical to base the hypotheses on motivations. Assume you have three different variations, so you'll use three different motivations to build the key selling points. This way, once you've determined the winner, you'll understand why users download the game and will be positioned to further exploit on the motivation in your store page. 

Once you've identified the right motivations and have the fundamentals in place, you can optimize the store page. This includes, for example, experimenting with different colors and facial expressions on icons. For example, you can test if a smiling face or a character with a fierce expression results in more downloads.

These are some of the most common A/B testing mistakes we see. I hope you found this article helpful, and if you have any further questions, please contact us via live chat or send an email to:

Jason Lempiäinen
Jason Lempiäinen

Customer Success Manager

Latest Post

Product update 10/22

May 11th

Update: 27.04

Apr 27th

NEW: Geeklab Taxonomy

Apr 12th

You may also like:

Product update 10/22

Product Update 10/2022In this weeks product update, Campaign Details and Dashboa...

Jason Lempiäinen May 11th

ASO Guide: Keep your ASO strategy alive

There are numerous stages to consider before, during, and after the launch of an...

Miira Vehviläinen Jul 14th

Update: Campaign Builder

We updated our Campaign builder with help of your awesome suggestions. On this p...

Jason Lempiäinen Dec 9th

Let's chat.