--- metadata: - name: generator content: Diplodoc Platform v5.55.3 alternate: - https://yandex.com/support/varioqub-app/en/web/create.md - https://yandex.com/support/varioqub-app/ru/web/create.md - href: https://yandex.com/support/varioqub-app/en/web/create.md type: text/markdown title: Markdown version - href: https://yandex.com/support/varioqub-app/en/llms.txt type: text/markdown title: llms.txt --- > **Documentation Index:** Fetch the complete configuration index at https://yandex.com/support/varioqub-app/en/llms.txt # Creating an experiment To create an experiment: 1. Go to AppMetrica. 2. Go to the **A/B experiments** page and click **Create experiment**. ## Adding conditions {#conditions} By default, experiments are only limited in time. You can further restrict your experiment's conditions by adding rules. Added conditions are joined using the AND operator. - **Audience share**: Percentage of the total audience across all experimental variants (set to 100% be default). You can change this value to run the experiment only on a portion of your app's audience. - **Mobile OS**: Operating system your app is built for. - **OS versions**: Version of the user's operating system. This value must only contain digits separated by dots. To specify a single version, set the same value for the start and end of the range. For Android, specify the [API level](https://source.android.com/docs/setup/about/build-numbers?hl=en#platform-code-names-versions-api-levels-and-ndk-releases). - **Languages**: Device language of the users who will see your configuration. - **Regions**: Region set in the device settings of the users. - **App versions**: Version or range of versions of your app that will show the configuration to users. - **Custom parameters**: You can define your own parameters as key-value pairs. For example, `param=value`. To apply multiple conditions in Varioqub, separate them with commas: `aparam=avalue,bparam=bvalue` (the conditions are joined by a logical AND). Condition blocks are combined by the OR operator. {% cut "Here are the characters you can use with parameters and their values" %} - `=` — Placed between the parameter name and its value. If absent, only the parameter's presence is taken into account. Example: `param=value` - `?` — Means any character that occurs zero or one time. Example: `pa?am=va?ue` - `*` — Means any characters that occur zero or several times. Example: `p*m=v*e` - `+` — Means any characters that occur one or several times. Example: `p+m=v+e` - `!` — Means negation of a condition or exclusion of a parameter from influencing the experiment. Example: `!param=val` {% note warning %} Negation `!` must be placed **as the first** character in a key. - Correct example: `!param=value`. - Incorrect examples: `param!=value` or `param=!value`. {% endnote %} {% endcut %} {% cut "Usage examples" %} #| || **Usage example** | **What is included in the experiment** || || `param` | Only the presence of the parameter, regardless of its value. || || `param=value` | Presence of the exact parameter name and its exact value. || || `!param` | Absence of a given parameter with any value. || || `*param*=value` | All parameters containing the `param` fragment with the given `value`. || || `!*param*=value` | The value of any parameter with the `param` fragment mustn't equal `value`. || || `!*param*=*value*` | The value of any parameter with the `param` fragment shouldn't contain `value`. || || `!*param*` | Absence of a parameter containing the `param` fragment. || |# {% endcut %} {% note info %} The option is available with [paid AppMetrica pricing plans](https://appmetrica.yandex.com/docs/en/common/pricing/ru-currency). {% endnote %} ## Intersections with experiments {#intersection} Intersection is when two or more experiments run concurrently and may influence each other. For example, if you're conducting an experiment where you change the color of the **Buy** button and simultaneously testing a new product description, you may see changes that come from both experiments, which can impact the results. If you want to conduct the experiment independently, select the experiments you want to exclude in the list. This way, the experiment you're creating won't overlap with the excluded ones. The new experiment will also not affect the excluded experiments' results. The sum of the percentages (shares) of the audiences of the new and excluded experiments can't exceed 100%. For example, if the audience of the new experiment is 30%, then the audience of the excluded experiments can't exceed 70% so that the shares add up to 100%. {% note info %} The option is available with [paid AppMetrica pricing plans](https://appmetrica.yandex.com/docs/en/common/pricing/ru-currency). {% endnote %} ## Selected metrics {#metrics} Choose the key metrics by which you'll determine the experiment's success. You can also select up to 10 secondary metrics for advanced analysis (this option is available for [paid AppMetrica pricing plans](https://appmetrica.yandex.com/docs/en/common/pricing/ru-currency)). Some metrics support advanced settings. You can specify additional parameters in the parameter input field. To do this, type a parameter name and press **Enter**. The total length of each parameter in the event mustn't exceed 50 characters. #| || **Metric name** | **Advanced settings** || || **Conversion from event to event...** | You can specify up to five parameter levels for each event. {.cell-align-center} || || **Step-by-step conversion from event to event...** | ^ || || **Event conversion...** | - You can specify up to five parameter levels for each event. - You can create up to five separate conditions. Each condition can contain up to five parameters. Conditions can be combined using the logical AND and OR. {% note info %} Users with the Pro plan can combine up to five conditions using the AND or OR operators. The Basic plan offers only one condition and doesn't support logical operators. {% endnote %} || |# The metrics you select will be included in the experiment results [report](https://yandex.com/support/varioqub-app/en/web/stat.md#data). ## Setting up an experimental variant {#variant} You can set up multiple variants to use in the experiment. You can either use the version of your app without the changes as the control variant or customize changes for the control variant. Experimental changes applied to the control and experimental variants can be configured with flags. You don't need to make changes to the app itself. The flags accept _String_ values. 1. Select the variant for which you want to apply the experimental changes. 2. Set the flags with your modified parameters. ## Sample size calculator {#calculator} {% note info %} The option is available with [paid AppMetrica pricing plans](https://appmetrica.yandex.com/docs/en/common/pricing/ru-currency). {% endnote %} Using the sample size calculator, you can assess whether the experiment's result will be statistically significant with the given audience size. This tool calculates the [MDE](*mde) to show the smallest significant change in a metric. You can use it to determine the likelihood of capturing small changes in the experiment with the given data and error levels. The smaller the MDE, the more subtle changes your experiment can detect. To reduce the MDE, increase the sample size or the experiment duration. If the MDE is high, you'll only be able to spot significant changes. You can conduct such an experiment on a small audience. Fill in the calculator fields so that they match the information about your experiment. - **Total users per month**: Consider your experiment conditions but not the sample size here. Specify the latter in **Audience share**. To calculate the value for this field, use data from [AppMetrica](https://appmetrica.yandex.com/docs/en/mobile-reports/audience-report). - **Duration**: Duration of the experiment in days. Corresponds to the duration in the **Date range** field. - **Audience share**: Sample size, corresponds to the **Audience share** field. - **Variants**: Number of variants in the experiment, can take values from **2** to **26**. - **Metric conversion**: Percentage of users converted in the control variant. What action counts as a conversion depends on the exact metric. If you want to get statistically significant results for all metrics, specify the minimum conversion here. Borders of the detected effect show which indicators will be statistically significant. The values outside these borders will be significant: lower than the red border and greater than the green border. Indicator values in-between can be random deviations from the conversion of the control variant. If this range is too broad and you're interested in less pronounced changes in metrics, try adjusting the conditions of your experiment. ## Launching an experiment {#start} {% note warning %} You won't be able to change the conditions, metrics, and variants for your experiment after you launch it. {% endnote %} To launch your experiment, click **Save and run**. Familiarize yourself with the brief information about the experiment and click **Run** if everything is okay. Once the experiment is launched, you'll be able to view a preliminary report on the **A/B experiments** tab. ## Checking changes {#check} You can check how your changes were applied on different devices. To do this: 1. Make sure that the [Varioqub SDK is installed](https://yandex.com/support/varioqub-app/en/sdk/platforms.yaml) on the device. 2. Use the SDK to get the device ID: - For Android, use the [Varioqub.getId()](https://yandex.com/support/varioqub-app/en/sdk/android/integration.md#get-id) method. - For iOS, use [VarioqubFacade.shared.varioqubId](https://yandex.com/support/varioqub-app/en/sdk/ios/integration.md#get-id). 3. Specify the retrieved ID under **Experiment testing** and click **Activate**. The changes will become available after some time. They will be deactivated automatically one day after activation. You can deactivate the ID manually.
Contact support
[*mde]: Minimum Detectable Effect.