Conducting experiments

Yandex Advertising Network provides partners with convenient statistical analysis tools to improve the effectiveness of their ad blocks. You can also experiment with the blocks yourself to improve the efficiency of your advertising and increase the revenue from your website.

Experimentation is a controllable method for researching some kind of phenomenon. Experimental results serve as confirmation or refutation of a hypothesis.

Note the following:

  • When conducting one experiment, it is best to test no more than one hypothesis at a time (for example, only use a new block configuration or only one change in placement location).

  • Select a time period in which no significant changes are occurring to the site (for example, in the site's design or layout).

  • Try to select equivalent time periods when comparing any kind of indicators. For example, it would be better to compare impression dynamics for January of this year with the impression dynamics for January of the previous year rather than for February of this year (for more information, see Seasonal statistical fluctuations).

  • Statistical data is only worthwhile if a large volume of data has been accumulated.

Here is an experiment you can conduct right now.

Hypothesis: using icons for advertisers' sites in ad blocks could increase CTR.

Test method: Select a section of the site for running the experiment. Create new ad blocks in the visual code designer and add the advertisers' icons to the ad blocks. Reinsert the code on the site and create an event in the Event log.

Results analysis: After two weeks, compare the CTR of the ad blocks in the tested cross-section with the CTR of the ad blocks before the ad code was modified, and compare the tested CTR with the CTR of ad blocks in other sections of the site (if statistical cross-sections were created).

Conclusion: The ad block option that showed the highest CTR should be selected as the permanent ad block.

Other experiment ideas:

  • Change the number of ads in an ad block.
  • Test how enabling and disabling behavioral targeting affects your site revenue.
  • Find out how family filtering and blocking competitors affects ad effectiveness and site traffic.
  • Change the location of blocks on a page and see if this changes the number of bounces and the time spent on your website (according to Yandex.Metrica).