Toloka

Overview of Toloka

After registering in Toloka, you can create tasks for users. When designing tasks, follow the requirements and recommendations. You need to define how tasks look, customize their distribution, and set up quality control for the responses. When the tasks are ready, they are sent to users. Payment for completed tasks is deducted from your Toloka account. You can get users' responses in a TSV results file.

Tasks in Toloka

A task is part of a web page that can include various objects, such as images, text, and input fields. The user reads the instructions on the page, performs the task, and enters responses. You can use Toloka for any tasks that meet the requirements.

When creating tasks, you need to make settings such as the fields included, the data types, the logic for validating responses, and so on. To launch tasks, you need to define:

Input data is uploaded to Toloka in a TSV file. The tasks that are formed from this TSV file are sent out for completion simultaneously. This group of tasks is called a pool. You can get users' responses in a TSV results file.

Task suite

An HTML page can display just one task, or multiple tasks. Putting multiple tasks on a page allows you to:

  • Reduce the page loading time.

  • Set a lower price per task. The user is paid for completing the entire set (this means that the task price can be lower than the minimum rate of $0.01).

Distributing tasks

You can restrict the list of users for distributing tasks to. In the pool settings, set up filters by region, qualifications, device used, or other user characteristics.

For more reliable results, tasks should be overlapped so that each task is completed by several users.

Options for task distribution:

  • Issue tasks in random order (by default).

  • Issue tasks in the order in which they are listed in the TSV file (see Pool parameters).

  • (In the mobile version.) Let the user choose a task. For example, in projects related to geodata, users choose tasks depending on their location (see Project parameters).

Controlling the quality of responses

To improve the quality of responses, you can use the following tools:

  • Quality control rules.

    Allow you to set quality criteria and to assign tasks only to those who meet these criteria.

    Examples of quality control rules: “If the percent of correct responses in the pool is less than 60%, block project access”, or “If the user skipped 5 tasks in a row, suspend access to the pool”.

  • Offline acceptance of tasks.

    Allows you to manually check tasks and reject responses with poor quality. Rejected tasks are not paid.

Payment

For each accepted task, an amount of money is deducted from your account: payment for the task, plus the commission for using Toloka. The amount to pay for the task is set when creating the pool.

You can also award a bonus to one or more users.