A/B icon test

Your game's icon and cover play an important part in generating traffic from the catalog — well-picked and eye-catching creatives are more likely to draw user attention. To gain an understanding of which icon and cover will produce the best results for your project, run an A/B test for each creative. Catalog users will automatically be split into two groups:

• 50% of users will see materials from the control group.
• 50% of users will see materials from the test group.

Wait a few days and go to the A/B icon test tab in the Yandex Games Console. There you'll find the preliminary data that show how the test icon and cover impact catalog conversion rates and in-game metrics.

Launching an A/B test

Step 1. Prepare promo materials

Prepare a new icon and cover, ensuring they meet the requirements specified in clause 8.3.

Step 2. Create a request

1. Open the Yandex Games Console.

2. Select the game.

3. Go to the A/B icon test tab.

4. Select the audience for which you want to create a request.

   Note

   A test for the entire audience is the recommended flexible method for testing creatives.

5. Click Go to request.

Step 3. Upload files

Click Upload to attach a test icon and cover.

If you are testing creatives for multiple languages and plan to use the same images, you can speed up file uploads using the option Apply icon and cover for other languages (it is enabled by default):

1. Upload the icon and cover for the top variant. They will automatically apply to all languages.
2. If necessary, change the icons and covers for specific languages separately.

Step 4. Submit the game for moderation

Click Submit for moderation.

Assessing the results

To view the results of an A/B test:

1. Open the Yandex Games Console.

2. Select the game.

3. Go to the A/B icon test tab.

The top of the page displays the experiment's start date and the date when it will automatically conclude.

The preliminary results of your testing will become available in a few days. But to gain more reliable data, we recommend running the experiment for at least a week.

The creatives are compared based on the following metrics:

• Icon CTR in recommendations
• Cover CTR in recommendations
• Icon CTR in "My games"
• Conversion To Play
• Players
• Play time per player
• Play time

If the metrics are colored in:

• Green, the results are positive and statistically significant.
• Red, the results are negative and statistically significant.

If metrics are not colored in any way, it is impossible to definitively determine whether the experiment is affecting the user.

Making a decision

1. Open the Yandex Games Console.

2. Select the game.

3. Go to the A/B icon test tab.

4. Review the testing metrics and make a decision based on the metrics' colors:

   • Green: Click Accept the experiment to use the test icon and cover as the default.

     Note

     You can select which audience to apply the experiment to, depending on its results. For instance, if metrics are declining among Russian-speaking players but not among international ones, use experimental creatives for the international audience only. In this case, there's no need to update the draft additionally.

   • Red: Click Stop to end the experiment. The test will automatically conclude after 14 days.