In this concept validation set of tests, the aim was to find the right visual concept that would appeal the most to its users. The concept validation began before the launch in order to find the most appealing theme before launching the app. By doing this, Funcraft was able to base their decisions on data instead of assumptions. The result was a success and here’s how they achieved it.
Research
Research began by taking a closer look at the target audience. What their demographics were and their motivations. By also studying competitors, it was possible to create three different variations to test against the current one. These variations differed from each other quite noticeably.
First test – Finding the visual theme
The first test showed a major increase in IPM (install per mille) and a decrease in CPI (cost per install) between the worst performing and the best performing variants. The winning variant, the simplistic theme, outperformed the worst performing variant by far, against all expectations. The IPM increased by 50% and the CPI dropped by 84.4%.
As a part of the concept validation, we gathered responses from the audience with our survey tool. The set of questions helped gain insight about user behavior and preferences. Surprisingly, survey results showed a different favorite icon, when the audience got to choose between all of the variations. This is a good example of why it is a good idea to run both qualitative and quantitative testing and to base performance on quantitative results because numbers don’t lie
Second test – Refining the right visuals
Previous findings led to running a second test, where the object was to refine the simplistic theme and optimize it further by finding the best background and game style combination. We also wanted to confirm that the numbers are right since the result took us by surprise. This time the results were more even, yet a winning variant could be defined.
Findings
Out of the tests that were run, we were able to, first of all, help decide upon a creative angle. The most surprising element was, that even after analyzing the audience, the winning variant was not the one that initially was expected to be the winner. This goes to show that it is not easy to hit the jackpot with
assumptions and to put those assumptions to test.
Furthermore, we were able to narrow down on the simplistic creatives and find the best-performing visuals within that concept, bumping up the IPM rate and lowering the CPI. FunCraft was able to save efforts by taking these into account already in the development phase to continue developing in the right direction.
We highly recommend taking a look and downloading the game, it’s so much fun and you can play with your friends! How cool is that?
Download the app here for iOS and Android 😎
Interested in hearing how you could achieve awesome results like this?
Book a meeting with us!
