Troubleshooting

If you aren't using Toloka yet and need consulting
If you have a problem working with Toloka

In the section contents, select the stage where you encountered the problem and find the appropriate solution in the list. If not found, select the last item Other questions, fill out the form and wait for a response from the support team.

Registration

I can't get an SMS with the confirmation code

Use the Yandex.Passport recommendations. If this doesn't help, write to the support service.

The phone number belongs to another user

If you see this warning, make sure you entered the phone number correctly.

Please note that you can only have one requester account per phone number (see the User Agreement).

If you have a wrong number linked in Yandex.Passport, change it. For more information about linking a phone number, see the Help for Yandex.Passport.

If you don't remember your credentials in Toloka, use Restoring access.

If you deleted your account in Toloka, create a new Yandex account and register in Toloka.

How do I change the phone number in my account?

To change your phone number, go to Yandex.Passport and change your main number.

Note. If you don't have access to the old number, it takes a month to replace it. For more information about changing your phone number, see the Help for Yandex.Passport.
I want to complete tasks in Yandex.Toloka
If you registered as a requester but want to become a performer, create a new Yandex account and register again as a performer . You can't change the type of the account you created.
Other questions

Working in the sandbox

There is already an account in the production version of Toloka. Do I need to re-register in the sandbox?
Yes, each version of Toloka has its own account. To create a task in the sandbox, register there as a requester. To complete your task, register another account as a performer.
Why don't I see my task in the sandbox?

Make sure that:

  1. The pool is started.
  2. The user is added as a trusted user.
  3. The trusted user is registered in the sandbox as a performer.
  4. The trusted user didn't use social networks when registering.
  5. Your performer fits the filters you set.
I can't add a trusted user.
Possible reasons:
  • The user isn't registered in the sandbox. The account that you registered in Toloka's production version won't do. Make sure you have specified the correct performer's account as a trusted one.
  • The performer's account is authorized through social networks. Register a new user in Yandex.
Other questions

Starting and tuning a project

I created a project and a pool, but for some reason the Next button doesn't work or the preview shows a blank screen.
Toloka lets you know that something is wrong with the project. The blank screen often appears when there are errors in the task interface, including the JavaScript code. The Next button may be disabled if the output specification lacks some field or contains invalid values, or if, for example, you configured validation for a nonexistent field in JavaScript.
I can't upload files from Yandex.Disk

If images, audio or video from Yandex.Disk don't appear in the instructions or on the task page, make sure you connected Yandex.Disk correctly and uploaded the files:

Frequent mistakes when connecting to Yandex.Disk and uploading files
  • The Input data field in the project settings has the link type. You should choose the string type.
  • The Task interface block doesn't contain the proxy <input data field> structure. For example, to add a photo for a task, use the {{img src=(proxy image)}} component or the <img src={{proxy image}}> tag.
  • The TSV file contains absolute references to the task files. You need to insert a link <unique name>/<path and file name>. For example: yadisk/image1.jpg or yadisk/photos/image1.png.
  • Photos from Yandex.Disk are used in the task instructions in the mobile app. To display the photos in the instructions, use only direct links.
  • Files are deleted or aren't located in the Yandex.Disk folder that the link leads to.
  • The OAuth token isn't active. Update the token on the External services integration page.
If these recommendations didn't help, contact support:
What overlap should I set?

Overlap defines how many performers complete the same pool task.

The best overlap is an overlap that provides satisfying quality of results. For most tasks that are not reviewed, overlap from “3” to “5” is enough. If the tasks are simple, overlap of “3” is likely to be enough. For tasks that are reviewed, set overlap to “1”.

Can I change overlap after the pool is started?

Yes. Open edit mode for the pool and set a new overlap value. You don't need to restart the pool. Updating the settings is usually fast, but if there are many tasks, it may take several minutes.

What is the right time limit for the task completion?
Try completing the tasks yourself. Ask your colleagues and friends to complete them. Find out average completion time and add 50% to it.
How many tasks should I include in a task set?

The number of tasks on a page depends on how difficult and time-consuming the tasks are. Don't make task sets too large. They are unpopular, partly because they are inconvenient for performers (for example, if the internet connection is unstable).

Question about setting up quality control
How do I set quality control in a pool correctly?

The settings for quality control rules depend on the type of tasks. General recommendations:

  • Always use one or more ways to control quality of answers.

  • Counting quick responses makes sense for most tasks.

  • If the user has to choose between options (for example, by selecting checkboxes), check the answers using majority vote or control tasks.

  • If the user has to provide a response as a text or link or upload a photo, the best way to control quality is by reviewing assignments. You can outsource task acceptance to performers. Create tasks with a question (for example, "Is the phrase translated correctly?") and response options (for example, "Yes"/"No"). Set up overlap and majority vote checking.

  • If a task is more like an opinion poll (for example, choosing nice pictures from a set), majority vote is not a good way to control quality. Make control tasks with artificial examples where the choice is evident.

How many control tasks do I need to add?

We recommend adding at least 1% of control tasks to the main pool. To filter out performers, use the Golden Set quality control rule. To rank performers by the quality of responses in control tasks, use a skill.

How are the correct responses to control questions counted?

The Golden Set rule starts working after the performer completes the number of control tasks you specified. If your pool contains both training and control tasks, you can take into account the answers in both of them (the Number of responses parameter) or only in control tasks (the Number of control responses parameter).

As soon as the needed number of responses is collected, Toloka calculates the percentage of correct and incorrect responses and performs an action (assigns a skill, or blocks the user in the pool or in the project). Then this percentage is updated as the tasks are completed by the performer. The number of the performer's last responses used for the calculation is set in the History size field. If you leave it empty, all the responses from the performer in the pool are counted.

Should I create a skill for every pool?

It is better to use one skill in a project. You can choose the way to calculate the skill:

  • Calculate the skill for each pool separately. The current skill value is the value of the skill in the pool the user completed last. This option is convenient if:

    • The pools are intended for different groups of performers (for example, there are filters by city or country).

    • Pools are started one by one and you don't want to take into account the answers in the previous pools to calculate the skill in the current pool.

    This calculation method is used by default when adding a quality control rule to a pool. For the control tasks section, leave the History size field empty.

  • Calculate skill based on all tasks in a project This option is good if the pools are small and you don't need to have skill calculated for each pool.

    This option is available only for Golden Set skills. To use it, fill in the History size field in quality control rules in pools.

Can I use the same skill in different projects?

This isn't a good idea. If the performer completes a task well, this doesn't mean that they will successfully complete other ones. In addition, by using filters with skills set long ago, you limit the number of available performers.

Other questions

Working with results

Can I disable tasks for performers who complete the tasks poorly?

You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes too many mistakes in control tasks. Tasks completed by such performers can be given to other performers.

Can I fix something in a completed task myself?

No, you can't fix anything in the task itself. However, you can do this manually in the results file.

Can I ask a performer to redo the task if they made mistakes in it?

No. After sending a task, the performer can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.

How can I notify the performer of changes in the instructions?
Add the notification to the project description (for example: “Attention! The instructions changed”) and send a message to all the people who completed your tasks. To do this:
  1. Assign them a hidden skill, or use an existing skill linked to the pool.
  2. Go to Messages and click WriteGroupAdd filterChoose your skill<skill name>.
  3. If you created a new skill, specify the value you assigned to the chosen group, (for example, 1). If you use an existing skill, specify the minimum value.
Why has the speed of pool completion dropped?
Possible reasons:
  • You've stopped the training pool linked to the main one. This could limit the number of performers with access to the pool. Start the training pool again. There will be more performers who can access the tasks.

  • The filters you set are too strict. For example, a strong restriction on a certain skill that most users don't have.
  • Too many users are banned. Ease the quality control rules.
More than 500 performers passed the training, but the training skill shows only 30.

The pool shows the total number of performers that completed at least one task page. A training skill can be lost over time if you set repeated training in the pool settings. This setting allows a performer to pass the training again after a certain period if the performer didn't complete any main tasks or if there was a large time gap between completing tasks (for example, because of a ban). The training skill displays the performers who either recently completed training, or regularly complete your tasks so that the skill doesn't expire.

How can I speed up the pool completion?
  1. To motivate performers, assign a public skill and use dynamic pricing.
  2. Try to increase the project rating, so that your task is higher in the list of tasks for performers.
  3. Set a higher priority for the pool among other project pools.
How can I increase the project's rating?
The project rating is the average rating across all categories. Pay particular attention to the categories for which you got the least points.
Low rating for “Will you take similar tasks in the future”

Perhaps the performers found your tasks too difficult. Try to simplify them.

Low rating for “Clarity of instructions”

Shorten instructions and rewrite using simpler language. Add pictures and examples.

Low rating for “Task interface usability”

Make the interface more user-friendly and don't make the performer complete unnecessary actions. Use keyboard shortcuts.

Low rating for “Communication with the requester”

Reply to messages from performers regularly. Try to provide feedback as fast as possible. Correct errors promptly and use mailing lists to notify performers of changes.

If you don't understand what the problem is, run a mini-survey and ask the performers who completed your tasks what they like and what they don't like.
Other questions

Managing the requester's personal account

Return the funds transferred to the Toloka account
Get closing documents and invoices
Other questions