A/B experiments

The Yandex Advertising Network interface has a dedicated split testing technology called Varioqub.

You can use it to run A/B experiments with your ad units to:

Test out new overlay units. Add Fullscreen, Floor Ad, and Top Ad units and see how they work on your site.
Compare ad units with different parameters to determine which are the most effective. For example, you can compare banners with different CPM floors and blocked topics.

Note

To test out different different ad unit designs, use the design auction.

Creating an experiment

Add a Yandex Metrica tag to your site. It collects the data necessary for running experiments.
Connect Varioqub.
In the Yandex Advertising Network interface, go to Ads on websites → Sites and open the settings of the desired site.
Go to A/B experiments → To create.
Add the experiment conditions:
- Title: Experiment name.
- Counter: Yandex Metrica tag ID.
- Experiment Application Pages: Site domain and URL without the protocol (HTTP/HTTPS) and hash (the # character and the value that follows it in the URL).
  
  For example, if you want your experiment to include only the site's home page, write it as example.com. For information on adding more flexible conditions, see the Varioqub Help.
- Audience share: Percentage of the site's audience (from 1% to 100%) that will participate in the experiment. The share is evenly divided between the samples. By default, experiments cover 100% of the site's audience.
- Period: Experiment duration, from 14 to 90 days.
Set up ad unit options:
1. If you have a reference ad unit you want to compare other options to, select it from the drop-down list in the The control option field. During the experiment, the control unit is replaced with ad units from your experimental options.
  
  If you don't need a control option, leave the Without a block value to measure effectiveness with and without different ad unit options.
2. To add an experimental ad unit, enter the unit ID in the Option 1 field or select it from the list.
  
  To add multiple experimental options, click Add an option and select an ad unit. You can choose up to five options: they will rotate over the course of the experiment.
To open an ad unit's settings, click View settings to the right of the ad unit selection field. You can only change the settings when the experiment isn't running.
Click Save and Run.
If you used a control ad unit, replace the ad tag. Copy the new ad tag that will appear after you launch the experiment and add it to your site.

Note

If you used the clientFeatures argument or modified its parameters when connecting Varioqub, do the same for the ad tag of the control unit.

You can view the ad tag by clicking The code for the site on the experiment page.
Click Complete.

Viewing the experiment results

To view the experiment report, go to the A/B experiments tab, select the desired experiment, and click Extended Report. For more information on the report objectives, structure, and settings, see the Varioqub Help.

Brief statistics are also available under Statistics on the experiment page:

Option: Ad unit variant.
Block: Ad unit name and ID.
Time on the site: How much time the user spent on the site.
Bounce rate: Percentage of visitors who bounced from the site.
Remuneration on the website: Revenue from ad impressions.

Stopping an experiment

The experiment stops automatically when it reaches the end of the specified period. You can also stop it manually at any time:

Select the experiment in the A/B experiments tab.
At the top right, click Stop → Complete.

Warning

Once an experiment is stopped, you can't resume it.

Extending an experiment

You can extend an experiment after it reaches the end of the specified period.

Select the experiment in the A/B experiments tab.
Change the end date and click Save → Extend.

Applying the experiment results

In the A/B experiments tab, click How to apply the result next to the desired experiment.
Copy the code and add it to your site between the <body> and </body> tags at the location where the ad unit should be displayed. The ad unit will be displayed to all site visitors.

If you don't want to apply the result, you don't need to do anything.