Toloka for requesters

Getting responses

You can download users' responses as tasks are completed in TSV file. If tasks are assigned with overlap (for example, each task is completed by 5 users), you can get aggregated responses. Toloka will analyse the users' responses in the task and select one of them. Aggregation cannot be completed for the fields containing a file, coordinates, JSON object or a text string.

Files sent by the users in responses can be downloaded separately (for example, if a task requires attaching a photo).

Getting the TSV file with responses

To get a TSV file with responses, click Download results on the pool page.

Fields in the TSV file with responses:

  • INPUT:<object ID in the input data description> — Input data for tasks.

  • OUTPUT:<field ID in the output data description> — Responses from users. For training tasks, the field includes first attempted answers.

  • GOLDEN:<field ID in the output data description> — Responses for control tasks.

  • HINT:text — Hints for training taks.

  • Information about task completion (is the same for all tasks in a suite):

    • ASSIGNMENT:link — Link for viewing the task suite.

    • ASSIGNMENT:assignment_id — ID of the assigned task suite.

    • ASSIGNMENT:worker_id — ID of the user who completed the task.

    • ASSIGNMENT:started — Date and time the task suite was assigned.

    • ASSIGNMENT:status — Task status (“SUBMITTED” — completed, “APPROVED” — accepted, “REJECTED” — declined).

Sample TSV file:

Aggregated responses

To download TSV file with aggregated responses:

  1. Launch data analisys: click Download resultsAnswer aggregation on the pool page.

    Aggregation takes some time (from 10 minutes to several hours). You can track it on the Operations page.

  2. Download the TSV file when it is ready: click the Download link on the Operations page.

TSV file with aggregated responses contains fieds CONFIDENCE: <field ID in the output data description>. The fields contain statistical significance of the response in per cents (see Statistical significance).

Getting files

If the users were asked to upload files, TSV file with responses will contain IDs of the files received from users. To download files on your computer, click the Download resultsDownload attachments button on the pool page.