Toloka

Training tasks

Training tasks contain correct responses and hints. The user response is compared to the correct one. If a user responds incorrectly, a hint is displayed on the screen. The user must enter the correct response before continuing to the next task.

The results of the training can be used as a criteria for admission of the user to the main tasks. Create a training pool and define the minimum proportion of correct responses (only the first response of the user is used for the calculation). You can choose if the users are allowed to repeat the training if they failed to pass it from the first time.

You can decide if the trainig tasks are paid.

Creating a training pool

To run a pool with training tasks:

  1. Click Add PoolAdd Training on the project page. Fill in the fields:

    Field

    Description

    Guidelines

    You can use the project guidelines or change them for the training purposes.

    Training title

    Name of the training pool (only visible to the requester).

    Price per task

    Payment per task suite in U.S. dollars. For cents, use the dot (".") as the separator.

    free training

    Select the field to make training tasks unpaid.

    Adult content

    Whether the trainig tasks have porn content.

    Time on task

    The time allowed for completing a task suite, in seconds.

    Retry after

    You can allow to repeat the training if a user failed to complete it successfully and was not admitted to the main tasks. Set the period after which the user is allowed to repeat the training (enter the number of days).

  2. Click Create training. You will be redirected to the training pool page.

  3. Create a TSV file with trainig tasks. Include the fields with hints and correct responses into it:

    • GOLDEN:<field ID in the input data description>. Correct responses (they must correspond to the values in allowedValues in the output specification).

    • HINT:text. Hint text.

    Note. 

    The “GOLDEN” and “HINT” prefixes must be upper-case.

  4. Click Upload on the training pool page. Specify the number of training tasks on page and select the TSV file.

  5. Link the training pool to the main pool. To do it, fill in the fields on the main pool page:

    Field

    Description

    Training

    The name of the training pool.

    Level required

    Proportion of correct answers in training tasks (from 0 to 100) required to be admitted to the main tasks. Only first answer of the user in each task is taken into account.

The list of training pools is available on the project page.