Troubleshooting

Use the Yandex.Passport recommendations. If this doesn't help, write to the support service.

If you see this warning, make sure you entered the phone number correctly.

Please note that you can only have one requester account per phone number (see the User Agreement).

If you have a wrong number linked in Yandex.Passport, change it. For more information about linking a phone number, see the Help for Yandex.Passport.

If you don't remember your credentials in Toloka, use Restoring access.

If you deleted your account in Toloka, create a new Yandex account and register in Toloka.

To change your phone number, go to Yandex.Passport and change your main number.

If you registered as a requester but want to become a performer, create a new Yandex account and register again as a performer . You can't change the type of the account you created.

Yes, each version of Toloka has its own account. To create a task in the sandbox, register there as a requester. To complete your task, register another account as a performer.

Make sure that:

Possible reasons:

• The user isn't registered in the sandbox. The account that you registered in Toloka's production version won't do. Make sure you have specified the correct performer's account as a trusted one.
• The performer's account is authorized through social networks. Register a new user in Yandex.

Toloka lets you know that something is wrong with the project. The blank screen often appears when there are errors in the task interface, including the JavaScript code. The Next button may be disabled if the output specification lacks some field or contains invalid values, or if, for example, you configured validation for a nonexistent field in JavaScript.

If images, audio or video from Yandex.Disk don't appear in the instructions or on the task page, make sure you connected Yandex.Disk correctly and uploaded the files:

• How do I connect to Yandex.Disk?
• How do I upload files for instructions?
• How do I upload files for a task?

Frequent mistakes when connecting to Yandex.Disk and uploading files

• The Input data field in the project settings has the link type. You should choose the string type.
• The Task interface block doesn't contain the proxy <input data field> structure. For example, to add a photo for a task, use the {{img src=(proxy image)}} component or the <img src={{proxy image}}> tag.
• The TSV file contains absolute references to the task files. You need to insert a link <unique name>/<path and file name>. For example: yadisk/image1.jpg or yadisk/photos/image1.png.
• Photos from Yandex.Disk are used in the task instructions in the mobile app. To display the photos in the instructions, use only direct links.
• Files are deleted or aren't located in the Yandex.Disk folder that the link leads to.
• The OAuth token isn't active. Update the token on the External services integration page.

If these recommendations didn't help, contact support:

• Fill in the form:

Overlap defines how many performers complete the same pool task.

The best overlap is an overlap that provides satisfying quality of results. For most tasks that are not reviewed, overlap from “3” to “5” is enough. If the tasks are simple, overlap of “3” is likely to be enough. For tasks that are reviewed, set overlap to “1”.

Yes. Open edit mode for the pool and set a new overlap value. You don't need to restart the pool. Updating the settings is usually fast, but if there are many tasks, it may take several minutes.

Try completing the tasks yourself. Ask your colleagues and friends to complete them. Find out average completion time and add 50% to it.

The number of tasks on a page depends on how difficult and time-consuming the tasks are. Don't make task sets too large. They are unpopular, partly because they are inconvenient for performers (for example, if the internet connection is unstable).

How do I set quality control in a pool correctly?

The settings for quality control rules depend on the type of tasks. General recommendations:

• Always use one or more ways to control quality of answers.

• Counting quick responses makes sense for most tasks.

• If the user has to choose between options (for example, by selecting checkboxes), check the answers using majority vote or control tasks.

• If the user has to provide a response as a text or link or upload a photo, the best way to control quality is by reviewing assignments. You can outsource task acceptance to performers. Create tasks with a question (for example, "Is the phrase translated correctly?") and response options (for example, "Yes"/"No"). Set up overlap and majority vote checking.

• If a task is more like an opinion poll (for example, choosing nice pictures from a set), majority vote is not a good way to control quality. Make control tasks with artificial examples where the choice is evident.

How many control tasks do I need to add?

We recommend adding at least 1% of control tasks to the main pool. To filter out performers, use the Golden Set quality control rule. To rank performers by the quality of responses in control tasks, use a skill.

How are the correct responses to control questions counted?

The Golden Set rule starts working after the performer completes the number of control tasks you specified. If your pool contains both training and control tasks, you can take into account the answers in both of them (the Number of responses parameter) or only in control tasks (the Number of control responses parameter).

As soon as the needed number of responses is collected, Toloka calculates the percentage of correct and incorrect responses and performs an action (assigns a skill, or blocks the user in the pool or in the project). Then this percentage is updated as the tasks are completed by the performer. The number of the performer's last responses used for the calculation is set in the History size field. If you leave it empty, all the responses from the performer in the pool are counted.

Should I create a skill for every pool?

It is better to use one skill in a project. You can choose the way to calculate the skill:

• Calculate the skill for each pool separately. The current skill value is the value of the skill in the pool the user completed last. This option is convenient if:

  • The pools are intended for different groups of performers (for example, there are filters by city or country).

  • Pools are started one by one and you don't want to take into account the answers in the previous pools to calculate the skill in the current pool.

  This calculation method is used by default when adding a quality control rule to a pool. For the control tasks section, leave the History size field empty.

• Calculate skill based on all tasks in a project This option is good if the pools are small and you don't need to have skill calculated for each pool.

  This option is available only for Golden Set skills. To use it, fill in the History size field in quality control rules in pools.

Can I use the same skill in different projects?

This isn't a good idea. If the performer completes a task well, this doesn't mean that they will successfully complete other ones. In addition, by using filters with skills set long ago, you limit the number of available performers.

You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes too many mistakes in control tasks. Tasks completed by such performers can be given to other performers.

No, you can't fix anything in the task itself. However, you can do this manually in the results file.

No. After sending a task, the performer can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.

The pool shows the total number of performers that completed at least one task page. A training skill can be lost over time if you set repeated training in the pool settings. This setting allows a performer to pass the training again after a certain period if the performer didn't complete any main tasks or if there was a large time gap between completing tasks (for example, because of a ban). The training skill displays the performers who either recently completed training, or regularly complete your tasks so that the skill doesn't expire.

1. To motivate performers, assign a public skill and use dynamic pricing.
2. Try to increase the project rating, so that your task is higher in the list of tasks for performers.
3. Adjust the quality-speed ratio.
4. Set a higher priority for the pool among other project pools.