Ad experiments

With Varioqub, you can run experiments on ad units created in the Yandex Advertising Network and Adfox. This is helpful when you need to compare different ad unit formats and settings, test their size and placement, and measure the effect of new ad units on your revenue and user metrics.

After an experiment's completion, you can check the indicator report, which helps determine the variant that generates the most statistically significant increase in revenue without negatively impacting user metrics.

Connecting the experiment

Step 1. Connect Varioqub

To learn more, see Getting started and Connecting. We recommend using JavaScript to connect Varioqub.

Step 2. Add a Yandex Metrica tag to the advertising dashboard

  1. In the Yandex Advertising Network, go to the list of sites: Ads on sites → Sites and ad units → Sites.

  2. Next to the site you need, click and select Edit.

  3. On the General tab, enable the option Show YAN reports in Yandex Metrica.

  4. In Metrica tags, enter the tag ID and save changes.

  1. In Adfox, go to the list of sites: Settings → System → Yandex Metrica tags.

  2. In Connected tags, enter the tag ID and save the changes.

Launching an experiment

  1. In Yandex Metrica, go to Varioqub → Experiments.

  2. Click Create experiment.

  3. Add experiment conditions.

  4. Choose the key metrics by which you'll determine the experiment's success. For advanced analysis, you can also select up to 10 secondary metrics.

    To measure the success of your ad experiments, we recommend choosing “Average revenue per visit” (or per user) as the primary metric. For additional metrics, you can choose custom ones, such as “Page depth” and “Visual bounce rate”.

  5. Customize the experiment design. To do this, use:

    • Ad units: To add a new In-image, Floor Ad, Top Ad, or Interstitial ad unit.

    • Visual editor: To compare changes in ad display, size, and placement of an ad unit.

    • Flags in code: To compare ad units, to experiment with methods to call ad units, etc.

Ad units

On this tab, you can create an experiment with the placement and format of ad units on your site. If you connect Varioqub (exp.js), you can do this without adding the ad unit code (context.js).

Note

If your site has a CSP, in addition to connecting Varioqub, you also need to add rules for ads to conduct an experiment.

To create an experiment:

  1. Select the ad unit type:

    • Interstitial: Displayed in full screen. Create a separate unit for each site version.

    • Top Ad: Stays fixed at the top of the screen, overlapping the site content. Only works in the mobile site version.

    • Floor Ad: Stays fixed at the bottom of the screen, overlapping the site content. Create a separate unit for each site version.

    • In-image: Displayed over images on the site. The same ad unit works across all site versions.

      In-image unit limitations

      In the current version of Varioqub, the following settings aren't used for In-image units:

      • Page limit

      • Limitation for images

      • Type of integration

      For more information, see the In-Image section in the Yandex Advertising Network Help.

      By default, these ad units use Automatic integration. Ads appear on all images with a size of at least 320 × 190 pixels.

    Following the rules of the Yandex Advertising Network:

    • You can't display both Top Ad and Floor Ad units on the same page at the same time.

    • Top Ad units can only be used on mobile devices.

    See Unit types for more details.

  2. Specify the blockId for units that you're setting up your experiment for:

    • Desktop block ID: For the desktop site version.

    • Touch block ID: For the mobile site version.

    Warning

    Make sure that your In-image unit has the same blockId specified for all site versions.

Working in the visual editor

The editor is embedded into your site and enables you to change an ad unit's size and appearance. All changes are applied in real time. Once you save the changes and run the experiment, the edits you made in the editor become visible to users selected for the experiment.

Experiment changing an ad unit's appearance
  1. On the site, modify the ad unit code. Example of a modified YAN code:

    <!-- Yandex.RTB R-A-111111-1 -->
    <div id='yandex_rtb_R-A-111111-1'></div>
    <script>
        ymab('metrika.XXXX', 'init', () => {
            window.yaContextCb.push(()=>{
                Ya.Context.AdvManager.render({
                    'blockId': 'R-A-111111-1',
                    'renderTo': 'yandex_rtb_R-A-111111-1'})
            })
        })
    </script>
    

    where

    • R-A-111111-1 is the ad unit ID.
    • XXXX is the ID of the Yandex Metrica tag installed on your site.
  2. In Yandex Metrica, go to Varioqub → Experiments.

  3. Choose the Visual editor experiment type and specify the URL of the page you want to edit.

  4. Click Edit next to Variant 1. This opens the page you specified in the visual editor field. You don't need to make any changes to the control variant. That version of the page is used in the experiment in its current form.

  5. Click the unit you want to apply experimental changes to. The visual editor window opens.

  6. To change the size or appearance of the container, go to the unit's parent element. To do this, click .

  7. On the CSS tab, specify the property that you want to apply in the experimental variant and click Save.

    To make sure that your changes are applied, under Experiment testing, click Test next to each variant. The site version associated with the selected variant opens at a link generated by Varioqub. You can share this link within 24 hours of its generation. Clicking Test again generates a new link.

  8. Click Save and run.

Example of changing an ad unit's height

To run an experiment with different unit heights, on the CSS tab, specify the style property for the unit's parent element.

For example, for an experiment that switches from a fixed height to an adaptive one, change the following CSS properties:

  • Remove the height restriction with height: unset.

  • Set the maximum unit height with max-height: 70vh.

Example of making a side banner sticky when scrolling articles

To run an experiment that makes a side banner sticky when scrolling articles, on the CSS tab, specify the required properties for the unit's parent element.

For example:

  • Change the way the unit is displayed with position: sticky.

  • Set the unit's position relative to the parent element with top: 20px.

Experiment adding a new ad unit

You can also use the visual editor to add an ad unit embed code. This can be useful for testing new ad placements on your site. We recommend choosing Floor Ad, Top Ad, or Interstitial ad units. For example, you can do this in the Yandex Advertising Network.

  1. Configure the Varioqub library by adding the following code to the library:

    <script>
    ymab('metrika.XXXX', 'setConfig', {
        enableJS: true,
    });
    </script>
    
  2. In Yandex Metrica, go to Varioqub → Experiments.

  3. Choose the Visual editor experiment type and specify the URL of the page you want to edit.

  4. Click Edit next to Variant 1. This opens the page you specified in the visual editor field. You don't need to make any changes to the control variant. That version of the page is used in the experiment in its current form.

  5. On the page, select an element that appears on all pages of your site, for example, a footer.

  6. Select the JS tab and add the function from the ad unit code.

    An ad container will appear on all pages where your chosen element is present, and the code from the JS tab will be called.

  7. Click Save.

  8. Check if your code is processed correctly on different site pages: for example, on the home page, in an article, or in other sections of your site.

Setting up an experiment using flags

  1. In Yandex Metrica, go to Varioqub → Experiments.

  2. Choose the Flags in code experiment type.

Note

Make sure the flag name and values in Yandex Metrica and in the modified embed code are specified in the language and case that is used in the code.

Flags in code experiment
  1. In Varioqub, set the name and values of the flag. For example, enter block for the flag name, A for the control value, and B for Variant 1.

  2. In the HTML code of the page, specify the ad containers to run the experiment on:

    ...
    <div id="rsya_varioqub_A"></div>
    <div id="rsya_varioqub_B"></div>
    ...
    

    Next, change the ad unit code on the site as described below.

  3. Create a function for calling the control variant. In this example, the R-A-111111-1 unit is called:

    <script>
    function vq_call_block_a() {
    window.yaContextCb.push(()=>{
      Ya.Context.AdvManager.render({
        renderTo: 'rsya_varioqub_A',
        blockId: 'R-A-111111-1'
      })
    })
    }
    </script>
    
  4. Create a function for calling Variant 1. In this example, the R-A-111111-2 unit is called:

    <script>
    function vq_call_block_b() {
    window.yaContextCb.push(()=>{
      Ya.Context.AdvManager.render({
        renderTo: 'rsya_varioqub_B',
        blockId: 'R-A-111111-2'
      })
    })
    }
    </srcipt>
    
  5. Create a function for handling what happens when the experiment ends or in other scenarios (for example, when a page view doesn't match the filters set in the experiment settings):

    <script>
    function vq_call_block_default() {
    console.log('The experiment is complete or not launched on this page');
    }
    </script>
    
  6. Create a function for applying experiment flags as follows:

    <srcipt>
    function render_ab_answer(flags) {
      const flagVal = flags && flags.block && flags.block[0];
    switch
    (flagVal)
    {
       case 'A':
            vq_call_block_a();
            break;
       case 'B':
            vq_call_block_b();
            break;
       default:
            vq_call_block_default();
            break;
    }}
    </script>
    
  7. Configure the passing of flags to the function from step 4 using the ymab function:

    <srcipt>
    ymab('metrika.XXXX', 'init', function (answer) {
      render_ab_answer(answer.flags);
    })
    </script>
    

    Note

    The ymab function with the init method is called only once to pass flags.

  8. Check if your code is processed correctly on different site pages: for example, on the home page, in an article, or in other sections of your site.

Example of code for comparing two ad units
<script>
//Function to call the control variant
function vq_call_block_a() {
window.yaContextCb.push(()=>{
 Ya.Context.AdvManager.render({
  renderTo: 'rsya_varioqub_A',
  blockId: 'R-A-11111-1'
 })
})
};
//Function to call Variant 1
function vq_call_block_b() {
window.yaContextCb.push(()=>{
 Ya.Context.AdvManager.render({
  renderTo: 'rsya_varioqub_B',
  blockId: 'R-A-11111-2'
 })
})
};
//Function to call the default variant
function vq_call_block_default() {
console.log('The experiment is complete or not launched on this page');
};
//Function to apply flags
function render_ab_answer(flags) {
  const flagVal = flags && flags.block && flags.block[0];
switch
(flagVal)
{
   case 'A':
        vq_call_block_a();
        break;
   case 'B':
        vq_call_block_b();
        break;
   default:
        vq_call_block_default();
        break;
}};
//Function to get values from the settings of an experiment launched in Yandex Metrica and then pass the values to the function above
ymab('metrika.XXXX', 'init', function (answer) {
  render_ab_answer(answer.flags);
})
</script>

Look up examples of connection and creating experiments on GitHub.

FAQ and troubleshooting

I'm trying to create an experiment, but all the fields are locked.

Can factors that are not available in the experiment creation interface be used as conditions?

Can I use a custom JS script in an A/B test?

How do I disable a specific experiment type for an entire site or page?

An element loads after the page. How do I include it in the experiment?

My site has dynamic element IDs. Can I still use the visual editor?

My monetization statistics in Yandex Metrica and the partner interface are different.

Contact support

Yandex Advertising Network

Ad unit ID in the Yandex Advertising Network
formatted as follows: A-B-123456-78.