--- metadata: - name: generator content: Diplodoc Platform v5.48.2 alternate: - https://yandex.com/support/varioqub/en/stat.md - https://yandex.com/support/varioqub/ru/stat.md - https://yandex.com/support/varioqub/tr/stat.md --- > **Documentation Index:** Fetch the complete configuration index at https://yandex.com/support/varioqub/en/llms.txt # Experiment results report To view a report: 1. In Yandex Metrica, go to **Varioqub → Experiments**. 1. Under the name of the experiment, click **View result**. ## Ways to use this report {#case} With this report, you can see if the experimental variant has any statistically significant changes in the primary metrics compared to the control variant. The experiment tests hypotheses: - **H₀**: The metric value for this variant doesn't differ from the control one. - **H₁**: The metric value for this variant differs from the control one. The reports use the `Mann-Whitney U test` with the application of bucketing methods as the test statistic. If there is too little data, discreteness corrections are used. When the test statistic of a hypothesis takes on the value of **H₀**, the corresponding row isn't highlighted in the Yandex Metrica report. Keep in mind that if your hypothesis has a value of **H₀**, that doesn't mean that the metric doesn't change. The only conclusion that can be made with sufficient certainty is that the effect is smaller than the **MDE** (minimum detectable effect). You can detect smaller changes in metrics by increasing the audience size and duration of the experiment. When the test statistic of a hypothesis takes on the value of **H₁** with **P-value** ≤ 0.05, the corresponding row in the Yandex Metrica report is color-coded. The color intensity has three levels based on the **P-value**: - **Light**: P-value ≤ 0.05, moderate statistical significance. - **Medium**: P-value ≤ 0.01, high statistical significance. - **Deep**: P-value ≤ 0.001, very high statistical significance. Red indicates a negative metric change, while green indicates a positive one. ## Report structure and settings {#data} The report data is grouped by the variants included in the experiment. The control variant is placed in the first row. When creating a new experiment, you can select its primary and secondary metrics. During the course of the experiment, these metrics become available in the report: #### **Primary metrics** - **Time on site**: Information about how much time users spent on the site, taking into account the number of sessions. - **Page depth**: Number of site page views per session. - **Visual bounce rate**. Only the sessions of users selected for the experiment count toward the metrics. For example, if a user clicks a button and immediately leaves the page, the experiment doesn't trigger: such a session is considered a [bounce](https://yandex.com/support/metrica/en/general/glossary.html#glossary__failure), but not a visual bounce. Therefore, there tends to be fewer visual bounces than Yandex Metrica bounces. - **Audience (count of unique users)**: Analysis of visitor characteristics. A unique visitor is a user who's assigned a unique site visitor ID ([i](*i)) for the first time. - **Income of all goals per visit**: - Revenue for a specific goal. - Revenue for all goals. - Revenue for a specific step of a multi-step goal. - **Conversion to goal**: Percentage of sessions that complete the selected goal out of the total number of sessions recorded for the variant. - **Conversion to multiple goals**: Percentage of sessions that complete the specified number of selected goals out of the total number of sessions recorded for the variant. - **Conversion per user to goal (Bayes)**: Ratio of unique users who achieved the goal to the total number of unique users. This calculation uses the [Bayesian method](https://yandex.com/support/varioqub/en/stat/bayes.md). - **Conversion to step of multi-step goal**: Set a sequence of steps that the user takes toward a conversion. These steps can be pageviews or JavaScript events. - **Goal income per visit**: When a session goal is completed, Yandex Metrica can record revenue for that goal. This helps you measure cost-efficiency for the goal. - **Step income of multi-step goal per visit**: When a step of a multi-step goal per session is completed, Yandex Metrica can count revenue for that goal. This helps you measure cost-efficiency for the goal. #### **E-commerce metrics** To see these metrics in the report, [enable e-commerce](https://yandex.com/support/metrica/en/ecommerce/about.html). - **Share of buying sessions**. - **Average GMV per buying session**. - **Average GMV per session**. #### **Advertising metrics** {#advertising-metrics} To see advertising metrics in the report, connect a Yandex Metrica tag to your ad. To learn more, see [Ad experiments](https://yandex.com/support/varioqub/en/ads.md). The revenue data in Yandex Metrica may differ from the data shown in the advertising dashboard and the actual amounts, as Yandex Metrica and other services have different revenue calculation algorithms. - **Average revenue per visit (YAN)**: Revenue generated by ad impressions per session. - **Average revenue per user (YAN)**: Revenue generated by ad impressions per user. In addition, the report includes: - Auxiliary metrics. For example, for Conversion for goal, the report shows the numerator and the denominator: the number of sessions that complete the goal and the total number of sessions. - The value of the metric itself. For example, Conversion for goal is the percentage of converted sessions from the total number of sessions. - **Delta**: Difference between the metric values in the experimental and control version. - **Delta (%)**: Delta expressed as a percentage from the control version's metric value. - **Confidence interval (±2σ)**: Visualization with the confidence intervals of the experimental and control variants on a number line. - **P-value**: Main quantifier of the test statistic result. It represents the probability of getting as extreme or more extreme results under the assumption that the metric value doesn't change (hypothesis **H₀** in the example). To learn more, see [this article](https://en.wikipedia.org/wiki/P-value). To accept a hypothesis, the **P-value** is compared with the significance level: **P-value** <= alpha. The default threshold is `alpha = 0.05`. Keep in mind that alpha sets the probability of type I errors (false positives). It's impractical to set a very low alpha value, as it increases the probability of type II errors (false negatives) and raises the **MDE**. - **MDE (%)**: Minimum detectable effect. This is the smallest change in the metric that can be detected with the existing amount of data and with the probability of type I errors `alpha = 0.05` and type II errors `beta = 0.2`. The MDE is expressed as a percentage of a metric value in the control variant. You can lower the **MDE** by increasing the audience size and duration of the experiment. - **Filters** (Add filter): {% list tabs %} - Basic plan #| || **Filter** | **Description** || || **Without robots** | Excludes traffic from [web crawler](https://yandex.com/support/varioqub/en/create/conditions.md#robots) sessions from the report. || || **Without bounce visits** | Excludes traffic from sessions identified as [bounces](https://yandex.ru/support/metrica/en/general/glossary.html#glossary__failure) from the report. || || **Only full days** | Exclude days when the experiment didn't run for a full day, such as the start or end day of the experiment. || |# - Standard plan #| || **Filter** | **Description** || || **Without robots** | Excludes traffic from [web crawler](https://yandex.com/support/varioqub/en/create/conditions.md#robots) sessions from the report. || || **Without bounce visits** | Excludes traffic from sessions identified as [bounces](https://yandex.ru/support/metrica/en/general/glossary.html#glossary__failure) from the report. || || **Only full days** | Exclude days when the experiment didn't run for a full day, such as the start or end day of the experiment. || || **Platforms** | Segments data by device. Platforms available for filtering: desktops, mobile phones, tablets, TVs. || || **Traffic sources** | Use this filter to track the origin of traffic to your site. To do this, select the source type, for example: **Direct traffics**, **Search engine traffic**, **Click-throughs by site link**. In the report, you can select the desired attribution model: **Last click**, **Last non-direct click**, or **Last click from Yandex Direct**. The default setting is automatic attribution. For more information, see [Yandex Metrica Help](https://yandex.com/support/metrica/en/reports/attribution-model.html#attribution-model__auto). || || **Operating systems** | Segments data by operating system. Operating systems available for filtering: iOS, Windows, GNU/Linux, macOS, Android. || || **Gender** | Filtering by gender: male or female. || || **Age** | Segments the audience by age groups: under 18, 18–24, 25–34, 35–44, 45–54, 55 and older. || || **Interests** | Segments the audience by interests: entertainment and recreation; finance; construction; improvement and repair; health and beauty; real estate; food and drinks; telecom; family and children; business. || |# - Extended plan #| || **Filter** | **Description** || || **Without robots** | Excludes traffic from [web crawler](https://yandex.com/support/varioqub/en/create/conditions.md#robots) sessions from the report. || || **Without bounce visits** | Excludes traffic from sessions identified as [bounces](https://yandex.com/support/metrica/en/general/glossary.html#glossary__failure) from the report. || || **Only full days** | Exclude days when the experiment didn't run for a full day, such as the start or end day of the experiment. || || **Conversion** | Filters report data by one of the completed goals: **button click**, **click to cart**, or others || || **Platforms** | Segments data by device. Platforms available for filtering: desktops, mobile phones, tablets, TVs. || || **Traffic sources** | Use this filter to track the origin of traffic to your site. To do this, select the source type, for example: **Direct traffics**, **Search engine traffic**, **Click-throughs by site link**. In the report, you can select the desired attribution model: **Last click**, **Last non-direct click**, or **Last click from Yandex Direct**. The default setting is automatic attribution. For more information, see [Yandex Metrica Help](https://yandex.com/support/metrica/en/reports/attribution-model.html#attribution-model__auto). || || **Operating systems** | Segments data by operating system. Operating systems available for filtering: iOS, Windows, GNU/Linux, macOS, Android. || || **Gender** | Filtering by gender: male or female. || || **Age** | Segments the audience by age groups: under 18, 18–24, 25–34, 35–44, 45–54, 55 and older. || || **Interests** | Segments the audience by interests: entertainment and recreation; finance; construction; improvement and repair; health and beauty; real estate; food and drinks; telecom; family and children; business. || |# {% endlist %} This report doesn't support Yandex Metrica [default settings](https://yandex.com/support/metrica/en/reports/report-general.html). ## About reports on dynamic traffic {#dynamic-traffic} {% note info %} Only available in the [Varioqub extended version](https://yandex.com/support/varioqub/en/paid-functionality.md). {% endnote %} For experiments with dynamic traffic distribution, the report has the following features: 1. Two key metrics are available in the report: * **Conversion**: The selected conversion metric that determines how traffic is distributed. * **Audience**: The number of sessions in each variant, helping you track distribution across variants over time. 2. Charts and tables look the same as in regular experiments, but take into account dynamic traffic distribution. 3. The **Detailed analysis** button is disabled in this report. ## Other metrics {#additional-metric} {% note info %} Only available in the [Varioqub extended version](https://yandex.com/support/varioqub/en/paid-functionality.md). {% endnote %} You can expand reports on ongoing or stopped experiments with additional metrics. This is useful if you selected an irrelevant metric when creating the experiment or if you need to analyze the experiment's impact on other metrics. 1. In Yandex Metrica, go to **Varioqub** → **Experiments**. 1. Under the name of the experiment, click **View result**. 1. In the menu that appears, click the selected metric and choose **Other metric**. A list of available metrics opens, excluding those you've already selected. Select the desired metric to see how it was affected by the experiment. ## Detailed analysis {#detailed-analysis} On the main report page, you can see a **Detailed analysis** button that opens a table with statistically significant metric changes grouped by various filters. This table displays only the results where **P-value ≤ 0.05**. For more information, see [Detailed analysis](https://yandex.com/support/varioqub/en/stat/detailed-analysis.md). ## Complete experiment report {#pdf-report} {% note info %} Only available in the [Varioqub extended version](https://yandex.com/support/varioqub/en/paid-functionality.md). {% endnote %} You can view and download the complete report on a stopped experiment. 1. In Yandex Metrica, go to **Varioqub** → **Experiments**. 1. Click → **View full report** next to the name of the experiment for which you want to get the complete report. A page will open with a report that contains complete information about all experiment settings, detailed data for the main metric with a chart, and summarized data for all auxiliary metrics. - Click **Update report** to update the report data. Reports can't be updated more than once per hour. - Click next to the **Update report** button to print the report. ## Publishing a variant of the experiment via personalization {#publish-via-personalization} To apply a successful experiment variant, create a [personalization](https://yandex.com/support/varioqub/en/personality.md) from it. 1. In Yandex Metrica, go to **Varioqub** → **Experiments**. 1. Under the name of the experiment, click **View result**. 1. Under the variant with changes, click **Create personalization**. The personalization draft with the conditions of this variant will open. 1. Click **Save**. The changes from the selected variant will be applied to the entire website audience, taking into account the conditions of the experiment. For example, the personalization will include platform or region-specific conditions and URL filters. To cancel the changes, stop the personalization: 1. In Yandex Metrica, go to **Varioqub** → **Personalizations**. 1. Next to the personalization, click → **Stop**. ## FAQ and troubleshooting [The experiment data doesn't match the Yandex Metrica report data](https://yandex.com/support/varioqub/en/faq.md#differs-form-metrica). [In my redirect experiment, the test and control values differ significantly](https://yandex.com/support/varioqub/en/faq.md#redirect). [After deleting cookies from my device, I was in multiple experiments, but the report shows only one session](https://yandex.com/support/varioqub/en/faq.md#cookie-deletion). [My monetization statistics in Yandex Metrica and the partner interface are different.](https://yandex.com/support/varioqub/en/faq.md#monetization-statistics)

Contact support

[*i]: Site user's ID stored in first-party cookies. Make sure to set it in the `_ymab_param` cookie upon receiving the response. It may be empty during the user's first visit to the site. We recommend passing it in each API request. The field may be empty in the first request.