Receiving responses

You can download performers' responses in a TSV file as tasks are completed.

If tasks were distributed with an overlap of more than 2, run aggregation. Toloka will process all performers' responses for the task and form the resulting response.

  1. Getting the TSV file with responses
  2. Getting files
  3. Troubleshooting

Getting the TSV file with responses

To get a TSV file with performers' responses, click the Download results button on the pool page.

Fields in the TSV file with responses
  • INPUT:<name of the input data field> — Input data for tasks.

  • OUTPUT:<name of the output data field> — The performers' responses. For training tasks, the field includes first attempted answers.

  • GOLDEN:<name of the output data field> — Responses for control tasks.

  • HINT:text — Hints for training tasks.

  • Information about the task completion (the same for all tasks on the page):

    • ASSIGNMENT:link — Link for viewing the task suite.

    • ASSIGNMENT:assignment_id — ID of the assigned task suite.

    • ASSIGNMENT:worker_id — ID of the performer who completed the task.

    • ASSIGNMENT:started — Date and time the task suite was assigned.

    • ASSIGNMENT:status — Task status: SUBMITTED — completed, APPROVED — accepted, REJECTED — declined.

Sample TSV file

Getting files

If the performers were asked to upload files, the TSV file with responses will contain IDs of the files received from performers. To download files on your computer, click the Download results → Download attachments button on the pool page.

Troubleshooting

Can I ask a performer to redo the task if they made mistakes in it?

No. After sending a task, the performer can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.

Can I fix something in a completed task myself?

No, you can't fix anything in the task itself. However, you can do this manually in the results file.

How do I send an assignment back to the performer for revision?

You can only accept or reject an assignment.

Can I disable tasks for performers who complete the tasks poorly?

You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes many mistakes in control tasks. Tasks completed by such performers can be given to other performers.

What should I do if I want to accept a completed task but the pool is already archived or the performer wrote to me after the allowed time?

Simply give the performer a separate reward without changing the task status. You can't change the task status in the pool in this case.

What should I do if I rejected a task for a reason that isn't specified in the instructions?

Accept the task and update the instructions. Otherwise, you violate the Requester Agreement that requires to clearly state the task requirements and the results expected from the user.

Can I reject part of the responses on the page and accept part of them?

No. For example, there are 10 tasks in a suite that costs $0.10, and the performer did 2 of them incorrectly.

You can't accept the correct answers and pay for this part ($0.08). Response pages are accepted or rejected in their entirety.

How can I notify the performer of changes in the instructions?
Add the notification to the project description (for example: “Attention! The instructions changed”) and send a message to all the people who completed your tasks. To do this:
  • Assign them a hidden skill, or use an existing skill linked to the pool.
  • Go to Messages and click WriteGroupAdd filterChoose your skill<skill name>.
  • If you created a new skill, specify the value you assigned to the chosen group, (for example, 1). If you use an existing skill, specify the minimum value.
How can I increase the project's rating?
The project rating is the average rating across all categories. Pay particular attention to the categories for which you got the least points.
Low rating for “Will you take similar tasks in the future”

Perhaps the performers found your tasks too difficult. Try to simplify them.

Low rating for “Clarity of instructions”

Shorten instructions and rewrite using simpler language. Add pictures and examples.

Low rating for “Task interface usability”

Make the interface more user-friendly and don't make the performer complete unnecessary actions. Use keyboard shortcuts.

Low rating for “Communication with the requester”

Reply to messages from performers regularly. Try to provide feedback as fast as possible. Correct errors promptly and use mailing lists to notify performers of changes.

If you don't understand what the problem is, run a mini-survey and ask the performers who completed your tasks what they like and what they don't like.
Why do I get blank spaces inside my TSV file?

When you upload a file with rows, double quotes indicate an area where you can use special characters (tabs or line breaks). Toloka merges everything in between the quotes into one row to make up one task. To use double quotes inside such an area, you need to escape them with another quote. Read more here.

What is the result of processing an empty text field?
If a performer typed some text and then deleted it, the result is null, otherwise, it's undefined.