Toloka

Starting tasks

A project can include the following types of tasks:

  • Main tasks — the tasks created to get responses from users.

  • Training tasks — the tasks to train users. Training tasks contain a hint and correct response. The user's response is compared to the correct one. If a user responds incorrectly, a hint is displayed on the screen. The user must enter the correct response before continuing to the next task. It is possible to allow completing the main tasks only for the users who passed training successfully.

  • Control tasks — tasks for checking. Control tasks contain correct answer. They are used to fiter the users who give incorrect responses too often.

Pools of tasks

To release tasks for completion, you need to create a pool. A pool consists of tasks sent out for completion at the same time. A project can contain several pools. For training tasks, it makes sense to create a separate pool. The users who complete it can then be allowed to pass to the main tasks. Control tasks are usually added to the pool with the main tasks. It is necessary to set the quality control rule.

Task suites

A task suite is a set of tasks placed on the same webpage. The users get a reward for completing a task suite. You can define the number of tasks in a suite, depending on the complexity of tasks, their interface and time needed to complete them.

You can use one of the following methods to split the pool into task suites (the method is chosen when loading tasks into the pool):

  • By youself.

  • With the help of Toloka (“smart mixing”). You have to upload tasks and define how many main, training and control tasks should be included into a task suite. Toloka generates task suites so that the user doesn't get repeated control and training tasks.

Data for the tasks

The data for tasks (such as images) can be stored on Yandex.Disk or received from other HTTP services.