Task markup lets you add correct responses and hints to the tasks, as well as change task types. You can turn a main task into a control task by adding the correct answer, or into a training task by adding the correct answer and a hint.
Avoid wordings like: “You answered incorrectly, please provide the correct response”. The performer learns when the hint explains the essence of their mistake.
Make the hints clear. Explain which response should be chosen and why.
How do I mark up tasks?
- Make sure the pool isn't running.
- Click Edit in the Pool tasks block.
- Open the Main, Control tasks or Training tasks tab. Create a control or a training task from another type of task.
- Select the responses that should be checked. (The list of the output data fields is on the right.)
- Click Save and go to next.
In this example, the correct response is added for the control task. Only the choice in the
result field (Good/Bad/Loading error) is checked.
Improve the reliability of control tasks
After you have created the control tasks, make sure that different variations of correct responses occur with the same frequency. This will help avoid random guessing in responses.
- Go to the task markup page.
- Open thetab.
The distribution of responses is shown as a percentage.
- Example of response distribution
The number of tasks depends on how difficult and time-consuming the tasks are. Don't make task pages too large. They are unpopular, partly because they are inconvenient for performers (for example, if the internet connection is unstable).
The Control tasks rule starts working after the performer completes the number of control tasks you specified. If your pool contains both training and control tasks, you can take into account the responses in both of them (the Number of responses parameter) or only in control tasks (the Number of control responses parameter).
As soon as the needed number of responses is collected, Toloka calculates the percentage of correct and incorrect responses and performs an action (assigns a skill, or blocks the user in the pool or in the project). Then this percentage is updated as the tasks are completed by the performer. The number of the performer's last responses used for the calculation is set in the Recent values to use field. If you leave it empty, all the responses from the performer in the pool are counted.