All questions on one page
If you are looking for the answer to a specific question, use Ctrl+F to search the page (Cmd+F on MacOS).
- If you aren't using Yandex.Toloka yet and need advice
- If you have a problem working with Yandex.Toloka
-
In the section contents, select the stage where you encountered the problem and find the appropriate solution in the list. If not found, select the last item Other questions, fill out the form and wait for a response from the support team.
- If you need help posting tasks
- You can contact Yandex.Toloka Laboratory and our partners.
Registration and getting started
- Check the number you entered. You might have mistyped it.
- Check the phone number specified in Yandex.Passport. It must be set as your main number. If a different number is specified in Yandex.Passport, an SMS code will be sent to that number.
- Log out of your requester account.
- Go to the Yandex.Toloka home page.
- Click Join.
- Follow the system instructions.
You can also use the Yandex.Passport recommendations. If this doesn't help, contact support.
The account is created for the email address entered during registration. You can't link this account to another account. The requester can't connect the Yandex.Disk of another account to the active requester account either.
To manage your account balance and interact with external services, be sure to log in with the username and password specified during registration. If you can't share this data, the other user needs to create their own account.
You can't change your account type from "Requester" to "Performer".
If you want to complete tasks, you need to register in Yandex.Toloka once again, with a different username, but as a performer.
If you see this warning, make sure you entered the phone number correctly.
Please note that you can only have one requester account per phone number (see the User Agreement).
If Yandex.Passport linked to the wrong number, change it. For more information about linking a phone number, see Yandex.Passport help.
If you don't remember your credentials in Yandex.Toloka, use Restore access.
If you deleted your account in Yandex.Toloka, create a new Yandex account and register in Yandex.Toloka.
To change your phone number, go to Yandex.Passport and change your main number.
Sandbox
- Try exporting the project without pools.
- Select an exported project and don't select pools in the window that opens.
- If the error persists, refresh the token.
- Go to Yandex.Passport.
- The user isn't registered in the sandbox. The account you registered in the Yandex.Toloka production version doesn't suit. Make sure you have specified the correct performer's account as a trusted one.
- The performer's account is authorized through social networks. Register a new user in Yandex.
To test your task, add yourself as a trusted performer in the Sandbox. To do this, on the Users page, click Add trusted users. Enter the username for the performer's account (case-sensitive).
To do this, you will need a Sandbox version of the Toloka app. Write to support to get it.
You won't be able to update a previously exported project. If you clone a project from the sandbox again, its current copy is added. The old project doesn't change.
The user can't see the task if they don't match the filter or rating. You can remove the blocking filter from the pool. To test your task, add yourself as a trusted performer in the Sandbox.
Use your requester account to preview the pool and check whether it's possible to submit a task. If this fails, most likely there is an error in your project.
The tasks themselves are not exported, only the project configuration and the settings of the selected pool. You can download the completed tasks from the pool in the Sandbox and import them to the exported pool.
To download only the control tasks (if you completed them in the interface), go to Mark up, then click Control tasks and Download.
Possible reasons:
Setting up a project
Instructions
- Use a link, for example: /api/proxy/yadisk/image1.jpg .
- In the requester profile settings, go to.
- Set up integration with external services.
- Put your instructions inside the task, but make sure that it doesn't clutter the interface.
- Use a side window for your instructions so that the user can quickly expand or collapse them.
- Hide the instructions in an expandable section or add hints for the individual interface elements.
Configuring the task interface
- See how it's implemented in the template "Find information online".
- To solve the second problem, you can add another validation like this:
if (solution.output_values.url && solution.output_values.check) {return {task_id: this.getTask().id,errors: {'url': {code: 'Insert a link or select the checkbox if the site doesn't exist'}}}}
- Click the gear button in project editing mode.
- In the field that opens on the left, enter the links and press Enter.
- Click the gear button to open the settings.
- Delete
$TOLOKA_ASSETS/js/toloka-handlebars-templates.js
. - Select an arbitrary area in the image. For example, put a square in the upper-right corner.
Mention this in your instructions for reviewers.
- Ask the performer to skip the task and report it in a personal message. Messages are reviewed by the requester. If the selectable object is missing, the task is deleted from the pool (by resetting the overlap).
- Add the "No object" checkbox to the interface and make sure that your JS checks that either the object is selected or the checkbox is selected.
For control purposes, add information about the value of this checkbox to the task interface.
To the component that inserts the image, add the parameters: real-size=true
and screenshot=true
.
onKey: function(key) {
var el = this.getDOMElement().querySelector(".image-annotation-editor__shape-polygon");
if (key === 'D') {
el.click();
el.classList.add('image-annotation-editor__shape_active')
}
If you need to further modify the area-selection editor, use this library.
You can't check loading of audio files in the preview mode, but you can do it in the sandbox if you do your task. To do this, register in the sandbox as a performer and add the performer username to your trusted list on the Users page. For more information, see this post.
If none of the performers can submit the assignment, the most likely reason is JS validation. Run JS validation again.
Export your project to the sandbox and try to complete the task in the sandbox yourself.
To check the task's look-and-feel on the mobile phone screen, enable the emulation mode in Chrome or Safari and edit the CSS block.
You can also use the mobile version of the sandbox. Write to support to get access to it.
{{field type="checkbox" name=(concat "result." @index ) label=(concat "checkbox –
" @index) size="L"}}
Try to disable extensions in your browser. They might block iframe loading.
To pass a label
in the input data, enter the input field name into the label.
For example, if you have the asd
input field with the string type, the component would look like: {{field type="checkbox" name="like" label=asd hotkey="q"}}
.
If you want to pass different label values in different tasks or the number of checkboxes may differ, use concatenation.
You can hide text in an expandable section by using CSS styles, both in the task itself and in the instructions. You can see the sample code here.
{{field type="button-clicked" name="ads" label="Click me" href=name_escape action=true}}
The names of the output fields must differ: each checkbox must have its own unique name. For more information about this component, see here.
You can use JavaScript to add assignment validation depending on the checkbox. An example is provided in the "Find information online" template.
You can see an example in the comments for this project. The example includes the output data format and libraries to be added.
The issue is probably in the JS block. Try deleting its content, then test the Submit button in the preview mode.
You can't deselect a radio button. You can only select another radio button as a different response option.
In this case, you need to leave the field optional in the output data and set up validation depending on the performer response. You can see how it's implemented in the "Text classification with additional options" template.
Add sources="CAMERA"
to the attributes of the image loading component. This disables adding photos from the gallery.
setSolution: function(solution) {
var secondScale = this.getDOMElement().querySelector('.second-scale');
if(secondScale) {
secondScale.style.display = solution.output_values.grammar === 'no' ? 'block' : 'none';
}
TolokaHandlebarsTask.prototype.setSolution.call(this, solution);
},
You can create a selection and drop-down list with category selection. See how it is implemented on this page (Dropdown list tab).
Here you can read more about the template's JS extension.
To avoid conflict between the Vue markup and the Handlebars syntax, disable the "toloka-handlebars-templates" library and inherit from the Task/TaskSuite classes.
The "Side-by-side" template uses a component rather than HTML tag. This means that you should enclose your proxy in curly brackets as in the example: {{img src=(proxy image)}}
.
.task
or .task-suite
element. For example, to use a black background:.task-suite {
background-color: #000000;
}
.task {
background-color: #000000;
}
You can also assign a class to the interface block with the image and set the background for this block only.You can check the link format using regular expressions. To do this, add the link validation JavaScript code with regexp
to the task template.
For example: var regexp = /^(https://www.myurl.com/).{4,200}$/
.
input
field with the string type to the output data. Make the field mandatory. Then add the Text input field field in the task interface (in the HTML block) and specify the field name in the name
attribute:{{field type="textarea" name="input" width="270px" rows=5}}
You can use JavaScript to add assignment validation depending on a checkbox. An example is provided in the "Find information online" template.
Pass an array of strings as the input field. For example, as shown in the screenshot:
{{#each words}}
{{field type="radio" name="result" value=this label=this}}
{{/each}}
<input type=""range"" list=""rng"" class=""res"">
and use the following onRender function in your JS:
onRender: function() {
// DOM element of the task is loaded (available vi #getDOMElement())
//Add auxiliary variables
var $root = $(this.getDOMElement());
var _this = this;
var solution = TolokaHandlebarsTask.prototype.getSolution.apply(this, arguments);
$root.on('change', '.res', function(){
var range_result = $(this).val()
_this.setSolutionOutputValue('result', range_result);
return solution;
})
}
You can't expand the HTML window. To expand the JS and CSS fields, click any area within the field.
Enclose the input field in double curly brackets {{text}}
.
Enclose the input field in triple curly brackets {{{input_field}}}
.
For more information about using the component, see the Requester's guide.
To change the number of output fields dynamically, use the recommendations from this page.
We didn't intend this window for any sophisticated development. Usually, the content is prepared in a third-party prettified editor, and the resulting code is pasted to the window prior to the update.
However, in the context of TolokaHandlebars editability, there are no differences between our window and a third-party editor.
To display the text in the input field with HTML tags, use the <pre>
tag. For example:<pre>{{text}}</pre>
.
.task {
max-width: 800px;
}
pre {
white-space: pre-wrap;
}
If the tags or attributes disappear after you save the instructions (for example, checked="true"
), it means that they are not supported. For the full list of tags that can be used in the instructions, see the Guide.
Input and output data
You can find an example of the task template for selecting image groups at this link. The input and output data, as well as a fragment of the instructions, are in the comments to the project.
To validate the input data format, you can use the output field type, specifying the acceptable or minimum/maximum values. For example, create an output field for INN with the "string" type and enter its minimum and maximum length (like 10 and 12). To use a more sophisticated validation in the template, use RegExp.
To enter a date, you can add a calendar to the task interface. An example of a calendar is here.
You can't use a range as a fixed value.
When you download the results file, select the Start time and Submit time checkboxes. The downloaded file will include the exact UTC date and time when the performer accepted the task page and submitted it.
If you pass texts to the input data, you can upload 2 different tasks to the pool: pass Text 1 in the INPUT: <input field name>
field of Task 1. In Task 2, use this field to pass Text 2.
If the text is in the HTML block of the task template, then clone the project. To limit a performer to doing only one task in your project, use the Submitted responses rule. You can assign a skill or ban the performer after they submit one response.
Pool settings
Filters
- Has a skill value lower than the one specified in the Level required field.
- Didn't complete any tasks linked to training during this period.
- Upload the task file using Smart mixing.
- Specify
student
as the correct answer to the question. Don't take other questions into account (leave the fields empty or unselected). - Add the Control tasks rule to the pool:
if the percentage of correct control answers = 100, then set the skill value Student = 1
.
Yes, you can do that. In the filters, select . Please note that the profile data is entered by the user when they register in Yandex.Toloka. We recommended that you use the filters Region by phone number and Region by IP.
Yes, of course — you can use the same skill for different projects. But most often, a skill is intended for a specific project. If the performer completes a certain task well, this doesn't mean that they will complete other ones successfully. Another disadvantage is that if you filter by skills that were set long ago, you will artificially limit the number of available performers.
If you mean multiple different projects, you can't do that.
You can merge all the projects into one and use History size in the quality control rules. See examples in the Control tasks post.
You can use Aggregation by skill, but you'll need to list all the possible values, which is probably not the best choice. Perhaps you'll find another method of aggregation helpful.
Perhaps the output fields you want to aggregate don't have valid values in your project. For now, you have to specify the possible values for every type of output fields.
In the main Yandex.Toloka version, you can only assign a skill to users who have completed at least one of your tasks. There is no option to assign a skill to an arbitrary user. To limit the audience of users who will see your project, use filters. For example, specify the city, date of birth, gender, or some other parameters of your target performers.
Tasks from an open pool are available to every user that matches your pool filters. You can restrict access, like by using a skill.
Your users will need to be trained again.
To make your task available in the mobile app, set up the filter: client = mobile Toloka
in your pool.
If the user mismatches your preset filter or rating level, they can't see the task. You can only remove the restricting filter from the pool. You can test the task in the Sandbox by adding the desired user to your trusted list.
You can do that. To select performers for the pool, use filters.
Client = web version
or
= mobile Toloka
You can assign a skill to these people based on their performance in the previous pools. Use this skill as a filter in the new pool.
Specify this skill as a filter, but leave the value field empty (this is equivalent to absence of the skill).
If the user already has a given skill, you can't add the same skill to them from the task review interface. You can open the user's profile and edit the skill value.
If it's a public or training skill, they see it and they get a message about it.
There is no such option. If the skill is public, the performer sees it in their profile.
Requesters can't see the full details about specific performers. So you can't see information like the date of birth, gender, last name, or first name. However, you can use filters by date of birth and gender (in the pool settings). This way you can select a group of performers without accessing the personal information of individual performers. This decreases the risk of user de-anonymization.

Quality control
Always use one or more ways to control quality of answers.
Counting fast responses makes sense for most tasks.
If the user has to choose between options (for example, by selecting checkboxes), check the answers using majority vote or control tasks.
If the user has to provide a response as a text or link or upload a photo, the best way to control quality is by reviewing assignments. You can outsource task acceptance to performers. Create tasks with a question (for example, "Is the phrase translated correctly?") and response options (for example, "Yes"/"No"). Set up overlap and majority vote check.
If a task is more like an opinion poll (for example, choosing nice pictures from a set), majority vote is not a good way to control quality. Make control tasks with artificial examples where the choice is evident.
Calculate the skill for each pool separately. The current skill value is the value of the skill in the pool the user completed last. This option is convenient if:
The pools are intended for different groups of performers (for example, there are filters by city or country).
Pools are started one by one and you don't want to take into account the responses in the previous pools to calculate the skill in the current pool.
This calculation method is used by default when adding a quality control rule to a pool. For the control tasks block, leave the Recent values to use field empty.
Calculate skill based on all tasks in a project This option is good if the pools are small and you don't need to have skill calculated for each pool.
This option is available only for skills on control tasks. To use it, fill in the Recent values to use field in quality control rules in pools.
- No
- Don't show captchas.
- Low
- Show a captcha after every 20 task pages.
- Average/High
- Show a captcha after every 10 task pages.
- No
- Don't show captchas.
- Low
- Show a captcha after every 20 task pages.
- Average/High
- Show a captcha after every 10 task pages.
You've stopped the main pool. This could limit the number of performers with access to the pool. Start the training pool again. There will be more performers who can access the tasks.
- The filters you set are too strict. For example, a strong restriction on a certain skill that most users don't have.
- Too many users are banned. Ease the quality control rules.
- To motivate performers, assign a public skill and use dynamic pricing.
- Try to increase the project rating, so that your task is higher in the list of tasks for performers.
- Adjust the quality-speed ratio.
- Set a higher priority for the pool among other project pools.
The settings for quality control rules depend on the type of tasks. General recommendations:
We recommend adding at least 1% of control tasks in the pool. To filter out performers, use the Control tasks quality control rule. To rank performers by the quality of responses in control tasks, use a skill.
The Control tasks rule starts working after the performer completes the number of control tasks you specified. If your pool contains both training and control tasks, you can take into account the responses in both of them (the Number of responses parameter) or only in control tasks (the Number of control responses parameter).
As soon as the needed number of responses is collected, Toloka calculates the percentage of correct and incorrect responses and performs an action (assigns a skill, or blocks the user in the pool or in the project). Then this percentage is updated as the tasks are completed by the performer. The number of the performer's last responses used for the calculation is set in the Recent values to use field. If you leave it empty, all the responses from the performer in the pool are counted.
It is better to use one skill in a project. You can choose the way to calculate the skill:
Your training and control tasks have the same project specification. However, you can create a separate project with the tasks and assign a skill based on user responses. Then you can admit performers to the main project based on their skill.
An exam pool contains only control tasks. It's usually small and used for checking how well users learned to do your tasks after they read the instructions and completed the training. Unlike your main pool, you already know the correct responses for every task in this pool. You can set the price to zero.
Based on the results of responses to control tasks, you can assign a skill to the users and then specify it in the main pool as a filter. For example, MySkill = 80 or = Is missing
. You don't have to create an exam. For simple tasks, the training pool provides enough practice, but many requesters also use exams.
Yes, the fast response settings specify the time per task page.
Yes. When you copy the filter and quality control settings, the settings you previously added manually are overwritten. You should see a warning about this in the copy settings window.
Indeed, this rule is probably too strict. Even the most careful user can make a mistake, so you probably want to relax the rule. Besides the requester-specific bans, we have system processes that ban users who regularly fail captcha checks in Yandex.Toloka.
Yes, unfortunately, this can happen. This is why we recommend that you offer a training task or exam before the main task. In this case, only those people who showed good performance at the previous stage are selected for the main pool.
When you load tasks, use smart mixing. In this case, you'll have infinite overlap in your exam.
However, this poses the risk that you might spend a lot of money on the exam. You might want to open this pool only when the main pool opens, and close it when labeling of the main pool ends.
You can add a training pool to test your performers. Based on the test results, assign skills to the users for the tasks they do best.
Then open your pools only to the users that have a certain skill: use filters for this.
This won't lower performer ratings. Even if you ban users from your project based on the testing results, this won't affect their rating.
You can upload your main and control tasks separately using different files.
If the user already got paid for the tasks, the money can't be refunded to you.
Yes, if they can access both pools, they can do both of them. To restrict access to subsequent tasks for a performer, use the Completed tasks rule and select a ban at the project level.
No. The responses of these performers aren't automatically excluded from the final results file.
But you can do it yourself if you want. When downloading the results, select the option Exclude assignments by banned users. You can also forward all the assignments from banned users to other performers using a rule.
Yes, you can do that. In this case, create a practice pool based on your training pool and an exam pool based on your main pool. If a pool contains only control and/or training tasks, the price can be set to zero.
In the exam pool, you can create a skill reflecting the exam result and granting admission to the main pool. For example, if the number of responses is ≥ 10, set the skill value in the <exam skill> as % of correct responses
. In your exam pool user requirements, specify: <exam skill> < 80 or = Is missing>
. In the main pool, set up a filter: <exam skill> >= 80 and (<main skill> >= 70 or = Is missing)
. You can choose the skill values depending on how well the performers handle your task.
Captcha is usually used in simple projects with automatic acceptance, like classification, categorization, or information search. These are cases where there are few response options and users don't need to upload files or write texts. It helps you filter out bots and sloppy performers.
The percentage of correct responses is based on the total number of captchas processed by the performer within the “range” specified in the Recent values to use field. If the value is empty, the percentage is calculated using all the captchas that are shown for the tasks in the pool which uses the captcha rule.
All responses to the task are taken into account. If one response differs from the majority vote, the whole task is counted as mismatching the responses of other performers.
No, this is incorrect. With these settings, each time a rule condition is met, the performer gets skill = 1
. To change the skill value in the process of task review, you need a “multi-step” rule, which has multiple identical rules with different values of Total reviewed responses.
Technically, if you have only one task in your training pool, you don't have this option. The skill will be either 0
or 100
. We recommend that you add several tasks, or at least 2 so that the performer will practice on the first task and will be able to do the second task correctly. In this case, you can admit users to your main pool starting from the skill value of 50
.
You can also create a training pool based on the main pool. Assign a skill using the Control tasks rule: in this case, you can admit users with any skill level to your main pool, even if the value is zero. But we don't advise giving tasks to people who failed training.
No. However, based on your main pool you can create a training pool with practice tasks and enable non-automatic acceptance there.
No, they can't.
If you pass texts to the input data, you can load 2 different tasks in the pool. In one task, pass Text 1 in the INPUT: <input field name>
field, and in the other task, use this field to pass Text 2. But if the text is in the HTML block of the task template, you need to clone the project. To let a performer do only one task in your project, use the Submitted responses rule. You can assign a skill or ban the performer after they submit one response.
No, the users are unaware of the ban.
The tasks themselves are not exported, only the project configuration and the settings of the selected pool. However, you can download your marked up tasks from the Sandbox pool and import them to the pool you created. To download the control tasks only (if you marked them up in the interface), go to Mark up, then click Control tasks and Download.
The Recent values to use field is for the number of recent responses from the performer. If you use non-automatic acceptance for your task, then to set up your intended rule you need to specify 3
in Total reviewed responses.
To perform actions with users (assign a skill or ban them) based on the majority vote, add a relevant rule to the pool.
Don't forget to enable Keep task order in the pool parameters. Majority vote is used in the projects with preset options (radio buttons or checkboxes). This rule won't apply to the text entry or file upload fields.
For a control or training assignment to be counted as correct, it must exactly match the control assignment. To do this, you need to normalize the response text using JavaScript: remove spaces, punctuation marks, special characters, and capital letters, and write the result in a separate output field. Now you can match the processed assignment text against your control text.
Another option to select performers for this type of projects is non-automatic acceptance.
This is the total number of responses to the control questions.
You can create a task pool for all your performers and create performer skills in it. In this case, you can open your tasks only to the performers with the necessary skills. This won't affect their rating.
Even if you ban a performer from the project, this won't affect their rating either.
Overlap
Overlap defines how many performers complete the same pool task.
The best overlap is an overlap that provides satisfying quality of results. For most tasks that are not reviewed, overlap from “3” to “5” is enough. If the tasks are simple, overlap of “3” is likely to be enough. For tasks that are reviewed, set overlap to “1”.
Yes. Open edit mode for the pool and set a new overlap value. You don't need to restart the pool. Updating the settings is usually fast, but if there are many tasks, it may take several minutes.
Yes, this might happen. You must set an adequate pool closing interval.
In this case, if you don't have 3 identical responses for your task (response threshold), no user would be considered a good or poor performer, because the system can't see which of the users made an error.
But if you set response threshold = 2
with overlap = 3
, then two users with the same responses are considered good performers, but the third user, who gives a different response, is a poor performer.
Yes, you can do that. Set up incremental relabeling.
You can use overlap to let multiple performers do the same task. The overlap value is set up in the pool settings.
Adding tasks to a pool
TSV file
- The input field has the "link" type.
- The pool was created for an outdated project version. It means that the pool was created before you changed the input field type.
- Add requirements for the link format in your instructions and hints in your training pool.
- Use RegExp in your JS to trim the received links and write the result to the new output field, and then match the received value against the control value.
- Errors in column headers
-
If the column headings are incorrect, the whole file is rejected. Otherwise, Toloka specifies the number of tasks with processing errors.
- Processing errors table
-
Description How to fix "parsing_error_of": "https://tlk.s3.yandex.net/wsdm2020/photos/2d5f63a3184919ce7e3e7068cf93da4b.jpg\t\t", "exception_msg": "the nameMapping array and the sourceList should be the same size (nameMapping length = 1, sourceList size = 3)"
Extra tabs
If the TSV file contains more
\t
column separators after the data or the link than the number of columns set in the input data, you will get en error message.For example, if 1 column is defined in the input, and two more
\t\t
tabs are added in the TSV file after the link, you get 3 columns, 2 of which are extra.Remove extra column separators in the above example — both
\t\t
characters."exception_msg": "the nameMapping array and the sourceList should be the same size (nameMapping length = 4, sourceList size = 6)"
The number of fields in the header and in the row doesn't match.
Make sure that:
- The number of tabs in the file structure is correct.
- String values with tab characters are enclosed in quotation marks
" "
.
"code": "VALUE_REQUIRED", "message": "Value must be present and not equal to null"
The value for a required input field is not specified. Make sure that columns with required input data fields are filled.
"code": "INVALID_URL_SYNTAX", "message": "Value must be in valid url format"
Invalid data in the URL field. Make sure that:- Links start with the
http://
,https://
orwww
prefix. - When you upload the file from Yandex.Disk by relative link, the data type is set to string for the input data fields.
"exception_msg": "unexpected end of file while reading quoted column beginning on line 2 and ending on line 4"
The string includes unpaired quotation mark.
Check that all quotation marks are escaped.
Description How to fix "parsing_error_of": "https://tlk.s3.yandex.net/wsdm2020/photos/2d5f63a3184919ce7e3e7068cf93da4b.jpg\t\t", "exception_msg": "the nameMapping array and the sourceList should be the same size (nameMapping length = 1, sourceList size = 3)"
Extra tabs
If the TSV file contains more
\t
column separators after the data or the link than the number of columns set in the input data, you will get en error message.For example, if 1 column is defined in the input, and two more
\t\t
tabs are added in the TSV file after the link, you get 3 columns, 2 of which are extra.Remove extra column separators in the above example — both
\t\t
characters."exception_msg": "the nameMapping array and the sourceList should be the same size (nameMapping length = 4, sourceList size = 6)"
The number of fields in the header and in the row doesn't match.
Make sure that:
- The number of tabs in the file structure is correct.
- String values with tab characters are enclosed in quotation marks
" "
.
"code": "VALUE_REQUIRED", "message": "Value must be present and not equal to null"
The value for a required input field is not specified. Make sure that columns with required input data fields are filled.
"code": "INVALID_URL_SYNTAX", "message": "Value must be in valid url format"
Invalid data in the URL field. Make sure that:- Links start with the
http://
,https://
orwww
prefix. - When you upload the file from Yandex.Disk by relative link, the data type is set to string for the input data fields.
"exception_msg": "unexpected end of file while reading quoted column beginning on line 2 and ending on line 4"
The string includes unpaired quotation mark.
Check that all quotation marks are escaped.
The number of tasks depends on how difficult and time-consuming the tasks are. Don't make task pages too large. They are unpopular, partly because they are inconvenient for performers (for example, if the internet connection is unstable).
You can specify the number of tasks on the page when you upload your tasks to the pool. For more information about distributing tasks across pages, see here.
You can read about connecting Yandex.Disk here.
The project template must contain something like this:
<img src="{{proxy" img}}="" width="400">
, where img
is an input field in the string format.
Use the example.jpg
file for testing. You can find its URL under .
You must use a separate row for each task in your TSV file. For more information, see here.
When you create a pool, the pool will have settings for the number of tasks per page.
Escape commas with a backslash (\
).
Use the button Upload review results to upload your file. You can see the format here.
Assignments are reviewed in a TSV file.
The hint
column should be filled out for your training tasks. When creating a main task, you only need to fill out the input
fields. Omit the other fields or delete them along with their headers.
The file structure and how to fill it out is described here.
"Add your text here" is a hint for you. It means that you can replace the text in the field with your task data. The file structure and how to fill it out is described here.
If you have one word enclosed in quotes, format the uploaded assignment like this: "How many letters are there in the word ""Liechtenstein"""
. If you are escaping quotes inside your text, then the entire text must be enclosed in quotes. For more information, see the Guide.
Check the hint
field. For the main tasks, this field must be empty.
In the file with the main tasks, the columns with the INPUT
headers must be filled out. You can see those headers if you download a sample file from the pool.
If you are creating control tasks, fill out the GOLDEN
columns with the correct responses.
If you are creating a training task, you also need to fill in the HINT:text
column. For the main tasks you don't need any columns other than INPUT
, so feel free to delete them.
The file format must be TSV, and the encoding must be UTF-8.
If you are hosting your files on Yandex.Disk, watch the video on how to specify file links in a TSV file.
For more information about creating the file, see the Guide. If there are errors during the upload, look up the error description on this page.
The error might occur if the expected input type is URL, but a string is received.
It depends on the task. Technically, you can use as many tasks you want.
But users are reluctant to take lengthy tasks. They'd rather do 10 tasks that take one minute each than one task that takes 10 minutes.
In addition, if you use a large number of tasks on the page, there might be issues with uploading the files to be labeled. This problem might occur with images.
The third thing to consider is quality control and assignment review. If you use recompletion of assignments from banned users, you should split the task into smaller parts so that fewer assignments are recompleted. You are more likely to meet your budget this way.
Yandex.Toloka will split the links to images in the uploaded file into pages depending on the method you specified when uploading the TSV file. For more information about the three upload methods, see the Guide.
TSV files are insensitive to the order of fields. Use your preferred order of fields.
You can't use the interface to upload the tasks with multiple correct responses to the pool. You can only use the API for that.
If you have the Keep task order option enabled, labeling will start after the previously uploaded tasks are taken by users. If this option is disabled, we can't guarantee that the tasks are assigned in their sequence order.
The array of strings in the input data must be comma-separated. For example: INPUT:typestext1, text2, text3, text4
All the values are written to the same column. Make sure to escape quotes. For more information about escaping quotes in JSON format, see the Guide.
No. If you try to upload a file with missing headers to the pool, the system issues an upload error. All the INPUT fields required in the specification must be present in the TSV file with tasks. There must be no extra fields or columns.
If you don't want to show some data to performers, but you still need this data in the file, create the optional hidden input fields for such data in the project.
Text in the GOLDEN field must match the control text exactly.
Usually, if you copy site links from the browser, the copied links have the same format. But this is not the case when the link is trimmed or typed manually.
Smart mixing settings are specified for the file rather than for the pool.
The settings specified during the first file upload are applied to all the files that are uploaded to this pool later on.
All the values are written to the same column. Make sure to escape quotes.
For more information about escaping quotes in JSON format, see the Guide.
A task means a separate task. A task suite means a page with tasks. The performer gets paid for a task page.
result
match the line number of the uploaded file. Lines that were processed with an error have the status "success": false
.Using files from cloud storage
- How do I connect to Yandex.Disk?
- How do I upload files for instructions?
- How do I upload files for a task?
- The Input data field in the project settings has the link type. You should choose the string type.
- The TSV file contains absolute references to the task files. You need to insert a link
<unique name>/<path and file name>
. For example:yadisk/image1.jpg
oryadisk/photos/image1.png
. - Photos from Yandex.Disk are used in the task instructions in the mobile app. To display the photos in the instructions, use only direct links.
- Files are deleted or aren't located in the Yandex.Disk folder that the link leads to.
- The OAuth token isn't active. Update the token on the External services integration page.
- Link Yandex.Disk in your profile.
- Set the string type for the input data field.
- Insert a file link using the
proxy
component. - In the project, the input field where you pass the file link has the "string" type.
- The component in the task template uses the "proxy" expression.
- The format of relative links in the TSV file with tasks is correct: <unique name>/<file path and name>.
If images, audio or video from Yandex.Disk don't appear in the instructions or on the task page, make sure you connected Yandex.Disk correctly and uploaded the files.
Try the recommendations on this page or contact Yandex.Disk support.
To add images using links to Yandex.Disk, use the link format: /api/proxy/proxy name/path to image
.
In the requester profile settings, under page.
, set up integration with external services. For more information, see thisYou can base it on the video markup template.
To host your videos on Yandex.Disk, connect Yandex.Disk and set up the project. For a detailed video tutorial, visit our blog.
Training-exam-retry
- Has a skill value lower than in the Level required field.
- Didn't complete any tasks linked to the training during this period.
- Go to the user card.
- Click the Profile tab.
- Find the required skill in the list and download the history of its changes.
- Add requirements for the link format in your instructions and hints in your training pool.
- Use RegExp in your JS to trim the received links and write the result to the new output field, and then match the received value against the control value.
- Select the correct responses in the
GOLDEN:result
column. - Fill in the
HINT:text
column. It stores a hint to be shown if the user selects an incorrect response option.
The training and control questions must meet your project specification. However, you can create a separate project with your instructions, survey, and sample videos. Then you can assign a skill to users based on their responses. You can use this skill to admit performers to the main project.
The pool shows the total number of performers that completed at least one task page. A training skill can be lost over time if you set repeated training in the pool settings. This setting allows a performer to pass the training again after a certain period if the performer didn't complete any tasks in associated pools or if there was a large time gap between completing tasks (for example, because of the ban). The training skill displays the performers who either recently completed training, or regularly complete your tasks so that the skill doesn't expire.
An exam pool contains only control tasks. Usually it's small and intended to check how users learned to do your tasks after they read the instructions and completed the training.
Unlike your main pool, you already know the correct responses for every task in this pool. You can set the price to zero. Based on the results of responses to control tasks, you can assign a skill to the users and then specify it in the main pool as a filter. For example, = 80
or = Is missing>
. You don't have to create an exam, because the training pool provides enough practice for simple tasks. But many requesters also use exams.
The validity period of the training skills is controlled by the Retry after parameter.
In the task file, leave empty control values for the optional output data.
Tasks have infinite overlap in the training pool. As long as the training pool is open and the training is running, users can access the tasks. Learn more about training pools.
Text in the GOLDEN field must match the control text exactly.
Usually, if you copy site links from the browser, the copied links have the same format. But this is not the case when the link is trimmed or typed manually.
Smart mixing is set up when you upload tasks to the pool. After creating a pool, click Upload and select the method for generating task pages. You can upload them using separate files or one file, arranging them in any order.
You can close the pool manually at any time using the interface. However, you can't set the number of users that should complete the training pool for it to close automatically.
Training helps users learn how to complete your task and figure out the instructions.
Based on the training results, you can select performers who did well enough for the main pool.
However, the mere fact that a performer completes your training pool successfully doesn't guarantee that they will afterwards demonstrate high quality on your main tasks. Performers who show a high level of accuracy during the training could have obtained correct responses from others.
Besides the training, be sure to add quality control rules and control tasks to your main pools. This way you can ensure the quality throughout the task performance process.
If the task requires that the users send free-format responses or data files, use non-automatic acceptance to pay for tasks after they are reviewed.
This is a technical peculiarity of training pools. You can only upload tasks to your training pools this way. If you want to upload tasks to the training pool page-by-page, create the main pool, specify the pool type (Training) and set the price to zero.
Create a practice pool based on your training pool and an examination pool based on your main pool. If a pool contains only control and/or training tasks, the price can be set to zero.
In the exam pool, you can create a skill reflecting the exam result and granting admission to the main pool. For example, If the number of responses is ≥ 10, set the skill value in the <exam skill> as % of correct responses.
In your exam pool user requirements, specify: <exam skill> < 80 or = Is missing>
.
In the main pool, set up a filter: <exam skill> >= 80 and <main skill> >= 70 or = Is missing>
. You can choose the skill values depending on how well the performers handle your task.
Technically, if you have only one task in your training pool, you don't have this option. After the user completes the training task, their skill will be either 0 or 100.
We recommend that you add at least 2 steps: the performer will have enough practice with the first task to do the second task correctly. In this case, you can admit users to your main pool starting from the skill value = 50.
You can also create a training pool based on the main pool. To assign a skill, use the Control tasks rule. In this case, you can admit users with any skill level to your main pool, even if their skill is 0. But we don't advise giving tasks to people who failed training.
If you enabled incomplete training and specified the number of training pages required, users don't have to fully complete the training in order to pass. If such settings are disabled, the users have to complete all the tasks in the training pool to get a training skill.
You can create a retry pool similarly to an exam pool. In the pool settings, select the type Retry. In the retry pool filters, specify the upper and lower values of the <main skill> that the users must get in order to be admitted to the retry pool.
For example, if the main pool admits users with a skill of 70 or higher, then you can route the people with a skill between 40 and 69 to the retry pool.
To get a valid "range", enter the skill twice: with an upper and lower value. For example: <basic skill > <70 and <main skill> >=40
.
Yes, it is.
Training is designed to select performers for the main task. That's why training must be linked to the main pool and become inactive as soon as the main pool closes.
The user is trained to get access to your paid tasks. If the training is optional, there probably won't be very many people who choose to complete it. Technically, "optional" training like this can be based on the main pool that includes some training tasks.
To show the training separately from other pools, disable Use project description and use this field to specify that this is an optional set of training tasks. In the pool settings, select the Training type.
You can't use non-automatic acceptance in your training pool.
However, you can create a training pool with the Training type based on your main pool and enable non-automatic acceptance there.
You can't create a training like this, because for the response to be counted as correct it must exactly match the control text.
For projects using free text input or attached files, you can make a pre-selection task with non-automatic acceptance. You can admit good performers to your main pool based on their skill.
To do this, under Test result, go to Recent values to use and specify the number of recent responses from the performer.
Let's say you need to create an exam with three tasks, one task per page. If the performer succeeds in two out of three tasks, they get the skill.
If your task uses non-automatic acceptance, to set up such a rule you need to specify 3 as "Total reviewed responses". As you can see in the screenshot, in the first case, all the performers who completed 3 task pages and whose answers are reviewed will get the skill. In the second case, only those who have 2 or 3 tasks accepted will get the skill.
For a control or training assignment to be counted as correct, it must exactly match the control assignment. To do this, you need to normalize the response text using JavaScript: remove spaces, punctuation marks, special characters, and capital letters, and write the result in a separate output field. Now you can match the processed assignment text against your control text.
Another option for selecting performers for a project of this type is non-automatic acceptance.
Changing a running pool
If you change the time allocated for a task, the time value will apply to the tasks that have not yet been taken by the performers. The same applies to the case when you close the pool. A performer who has an assignment in the active status can complete the assignment.
If you uploaded tasks to the pool using smart mixing, you can stop the pool and mark up your tasks: edit answers, hints, or delete tasks.
If you uploaded them using a different method, clone your pool and upload the new file with the corrected list of data to be labeled.
Working with results
Aggregation
The pool page contains the List of Operations button.
You cannot aggregate by project fields that have no valid values. Specify the possible values for all the fields of all types.
You need to use smart mixing.
Processing the results file
When you upload a file with rows, double quotes indicate an area where you can use special characters (tabs or line breaks). Yandex.Toloka merges everything in between the quotes into one row to make up one task. To use double quotes inside such an area, you need to escape them with another quote. Read more here.
null
, otherwise, it's undefined
.Assignment review
- Assign them a hidden skill, or use an existing skill linked to the pool.
- Go to Messages and click Write → Group → Add filter → Choose your skill → <skill name>.
- If you created a new skill, specify the value you assigned to the chosen group, (for example, 1). If you use an existing skill, specify the minimum value.
You can only accept or reject an assignment.
You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes many mistakes in control tasks. Tasks completed by such performers can be given to other performers.
No, you can't fix anything in the task itself. However, you can do this manually in the results file.
Simply give the performer a separate reward without changing the task status. You can't change the task status in the pool in this case.
Accept the task and update the instructions. Otherwise, you violate the requirement to clearly state the task requirements and the results expected from the user.
No. For example, there are 10 tasks on the page that cost 0.1$, and the performer did only 2 of them incorrectly.
You can't accept the correct answers and pay him part of the cost - $0.08. Response pages are accepted or rejected only as a whole.
Perhaps the performers found your tasks too difficult. Try to simplify them.
Shorten instructions and rewrite using simpler language. Add pictures and examples.
Make the interface more user-friendly and don't make the performer complete unnecessary actions. Use keyboard shortcuts.
Reply to messages from performers regularly. Try to provide feedback as fast as possible. Correct errors promptly and use mailing lists to notify performers of changes.
Statistics
The percentage is calculated based on the total number of pages, including the overlap. If the pool uses smart mixing with control tasks, the maximum progress bar value and the currently completed percentage are calculated approximately.
“Quality: training tasks” includes only the training pools linked to the main pools, but the control and training tasks uploaded to the main pools are counted in “Quality: control tasks”.
Archiving pools and projects
You can't completely delete a pool, but you can archive it. The system won't delete my pools automatically.
If there is no activity in the pool for a month, it is archived. You can't run such a pool, but you can clone it or download the labeled data from it.
Inactive pools are archived after one month. To see them, go to the Pools tab, click Filters, and select the Archived status.

To find out the archiving date, get a list of operations with the POOL.ARCHIVE
type. The response will contain the pool number and the archiving date. You can't get the method used to archive the pool via the API.
If less than a month passed between the activity in the pool and its archiving, you can assume that the pool was archived manually, and if it's been one month or more, then it was archived automatically.
Simply give the performer a separate reward without changing the task status. You can't change the task status in the pool in this case.
Payments
Adding funds to your account
If the address is in Saint Petersburg, enter Saint Petersburg in the Region field, then specify the street, house number, and postal code. You don't need to fill out the other fields.
If the address is in Moscow, enter Moscow in the Region field, then specify the street, house number, and postal code. Learn more about connecting to billing.
If it still doesn't work, enter your full address with the postal code in the feedback form. We'll help you fill out the fields correctly.
That's normal. For example, if you entered $25 and you see "25,000", it's still $25 with a decimal delimiter. Your account will be topped up by $25 at the current exchange rate. You'll see the amount in rubles when you proceed to payment.
Yandex.Toloka is an international platform for users from different countries. The platform is provided by the Swiss company Yandex Services AG.
In accordance with the tax legislation of the Russian Federation, we have included Clause 3.8. in our Requester agreement: the VAT is charged in addition to the cost of services and included in the invoice. This is the same as paying VAT on purchases at any store. Your account in Yandex.Toloka will be topped up by the amount you entered.
If you pay using a bank card, the money is usually transferred to your Yandex.Toloka account within a few minutes. If you can't see the top-up amount on your account, write to us and we'll sort it out. Specify your requester username and account number and use Account top-up as your email subject.
You specify the top-up amount in USD. On the Balance, it's converted into rubles, including VAT. The conversion follows the Central Bank of Russia's exchange rate at the time of invoicing (UTC). Learn more about top-up.
You can top up your Yandex.Toloka account using a bank card or bank transfer. On the Profile page, click "Connect to billing", fill out the form, and you'll see the "Top up account" button. Learn more with step-by-step instructions.
Payment for tasks
You can set a price for your task page on the pool editing page. The minimum price is $0.01.
Here's the general rule of pricing: the more time is needed to complete the task, the higher the price is.
If the task is simple, like if the performer spends a few seconds to assess product relevance, then set the price to $0.01 - $0.02 for 10 tasks (products) on the page.
If you register in Yandex.Toloka as a performer, you can compare offers from other requesters.
Define the page price, multiply it by the overlap (it's usually 3-5 for a classification task) and add 20% VAT. Try topping up your account by $10 first, then continue topping it up based on the performance dynamics.
You can issue rewards after completion and describe the criteria for increased rewards in your task instructions. You can't change the task page price dynamically based on the completion results.
Track your money debited for rewards in Profile → Spent tab.
No. The price per task page is the same for all tasks in the pool. You can create multiple pools with different prices or change the price depending on the performer skill using Dynamic pricing. You can pay rewards to good performers.
Refund
Performers
Data about performers
You can view information about performers of your tasks on the Users page. To view information about a performer, select their ID. The requester can access the following data in the performer profile: country and city, age, education, language skills, rating. On the Metainformation tab, you can find the versions of the browser and operating system, the User-agent type, region detected by IP, and other performer parameters. To select the performers for your pool based on their profile, device, geo location, and other parameters, use filters.
It's the performer's responsibility to provide the information in the profile during registration. We don't request identity documents for registering in Yandex.Toloka. If a profile seems suspicious to us, we ask the owner to confirm the information.
No, only the requester can see the performer ID.
The requester can't see the full details about specific performers. For example, the requester can't see their date of birth, gender, last name, or first name. However, filters by date of birth and gender are available to the requester in the pool settings. You can use them to select a group of performers without accessing the details about specific performers. This decreases the risk of performer de-anonymization.
The pool shows the total number of performers that completed at least one task page. A training skill can be lost over time if you set repeated training in the pool settings. This setting allows a performer to pass the training again after a certain period if the performer didn't complete any tasks in associated pools or if there was a large time gap between completing tasks (for example, because of the ban). The training skill displays the performers who either recently completed training, or regularly complete your tasks so that the skill doesn't expire.
Rewards
You can increase the basic task price for performers that have a higher skill. To do this, set up Dynamic pricing. If you want to motivate high-quality performers regardless of their skill, use rewards. In your task instructions, specify the amount and conditions for the performer rewards.
To view your expenses that involve rewards, go to your profile and open the Expenses tab.
Cheaters
You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes many mistakes in control tasks. Tasks completed by such performers can be given to other performers.
No. After sending a task, the performer can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.
No, you can't fix anything in the task itself. However, you can do this manually in the results file.
Fraudulent performers aren't banned before the majority vote is known. That's why we recommend that you have new performers complete training or a test. Then you can select the users that successfully completed the training to do your tasks.
If the user was already paid for the tasks, you can't cancel the payment.
During the training, performers follow the task instructions and practice completing your tasks. Based on the training results, the requester can select performers who did well enough to get access to the main pool. However, the mere fact that the performer completes your training pool successfully does not guarantee that they will continue to demonstrate high-quality performance. Performers who did well on the training but had inadequate results in the main task might have obtained correct training responses from other people.
In addition to the training, be sure to set up quality control rules in your main pools. This lets you control the quality throughout the task completion process. If the task requires that users send free-format responses or data files, use non-automatic acceptance to pay for them only after reviewing the responses.
Questions about templates
Selecting a template
You can structure your text classification task using the source text and radio buttons. It can also be text and checkboxes, if you use multiple subjects.
We recommend that you take the "Text classification with additional options" template as a basis.
You can also create your own interface. All the available tools are described in the Requester's guide.
You can see how templates are implemented in the "Transcription" section: a string-type output field is used there. In the "Photo transcription" template, you can see how to define the text input fields. Please note that if you later want to use the Dawid-Skene aggregation method, you must use acceptable values.
You can create such an assignment based on the classification template. Show a product image and ask the question: "Does the product match the query?" Add two radio buttons for the responses: "Yes" and "No".
If your task contains many objects of different types, you should break it down. For example, you can ask users to select numbers in the first project, doorways and windows in the second project, walls in the third project, and plumbing in the fourth project.
The simpler the task, the cheaper it is and the better the quality of the final result. Set the cost of labeling a single class of objects in photos at about $0.01.
Use the Object selection in an image template. You can open this template in the editor and add a drop-down list for labeling the selected object. See how to do this in the editor description (Dropdown list tab).
You can use JavaScript to add assignment validation depending on a checkbox. An example is provided in the "Find information online" template.
Area selection
- Sorting images containing an object.
- Create a task using the "Image classification" template.
- Sort the images containing the object you are looking for.
- Show the image to the performer and ask if the object is in the image. Response options: Yes/No.
- Selecting objects in images.
- Select the object in the images you obtained after the previous project. You already have such a project. Run the task with non-automatic acceptance.
- Use the quality control rules: fast responses, non-automatic acceptance, and post-review re-assessment. Description of rules with examples.
- Reviewing object selection assignments.
- Create a task using the object selection template.
- Hide the editor and ask whether the object is selected correctly. Response options: Yes/No.
- In the input data, pass the images and coordinates of the labeled objects from the previous task.
To prevent the users who worked on the second project from doing the review, assign a skill to them. Use this skill as a filter in the pools of the third project.
- Select an arbitrary area in the image (for example, put a square in the upper-right corner). In this case, the project instructions for reviewers should also reflect this.
- Ask the performer to skip the task and report it in a personal message. Messages are reviewed by the requester. If the object is truly missing, the task is deleted from the pool by resetting the overlap.
- Add an additional "No object" checkbox to the interface. Make sure that your JS checks that either the object is selected or the checkbox is enabled. In this case, add information about the checkbox value in the review task interface.
- In the project, the input field where you pass the file link has the "string" type.
- The component in the task template uses the "proxy" expression.
- The format of relative links in the TSV file with tasks is correct: <unique name>/<file path and name>.
onKey: function(key) {
var el = this.getDOMElement().querySelector(".image-annotation-editor__shape-polygon");
if (key === 'D') {
el.click();
el.classList.add('image-annotation-editor__shape_active')
}
Use the C shortcut for closing areas.
You can also use the library to customize the keyboard shortcuts for your tasks.
You can use JavaScript to add assignment validation depending on a checkbox. An example is provided in the "Find information online" template.
In the case of crowdsourcing, it's better to break down this task. The simpler the task, the cheaper it is and the better the quality of the final result. The cost of labeling a single class of objects in photos might be about $0.01.
Use the "Object selection in an image" template as a basis. See the step-by-step guide for creating this type of project on this page.
The editor used in the template lets you add a drop-down list for labeling the selected object. See how to do this in the editor description (Dropdown list tab).
You can create a selection + drop-down list with category selection. See how to do this in the editor description (Dropdown list tab).
The coordinates are relative to the image.
In the standard template with an area selection editor, you can't use the control tasks, because in order for the assignment to be accepted by the system as correct, the object selected by the user must exactly match the control object. This is almost impossible. Therefore, you can leave the GOLDEN field empty in the task file or simply delete all the columns except INPUT.
In the standard template with an area selection editor, you can't use the control tasks, because in order for the assignment to be accepted by the system as correct, the object selected by the user must exactly match the control object. This is almost impossible. Therefore, you can leave the GOLDEN field empty in the task file or simply delete all the columns except INPUT.
Using training pools in an area selection project doesn't make sense, because for the response to be correct the user's selection must completely match the control object. This is almost impossible.
Such tasks are usually run with non-automatic acceptance: the performer submits an assignment, and then the assignment is rejected or accepted after the review.
For pre-selection of users you can use "examination tasks". Review the assignments and assign skills based on the percentage of accepted assignments. For this purpose, add the "Results of assignment review" rule to the pool. To make sure that only the good performers are admitted to the main pool, put a skill-based filter to the pool.
Field task
To open the camera instead of the gallery when the user taps the image upload button, in the Image upload button component specify camera=true
.
In the mobile apps, performers can add photos from the default gallery (iOS) or Google photo (Android). To limit the capacity of adding online images, specify in the component requiredCoordinates=true
. In this case, the system won't let the user add images without geotags.
Side-by-side comparison
- Use the "Side-by-side comparison" template.
- In the TSV file, specify the links to the compared images.
- In the file, create the tasks where all the images will be compared in pairs:
Image 1 and Image 2
Image 1 and Image 3
- Image 2 and Image 3
- Process the results.
You can also edit the project so that the user sees 3 images at once and selects one of them.
Survey
The survey includes an auxiliary input field. You can use it to pass any information, and it won't be visible to performers.
Upload one task and use the overlap parameter to enter the number of people you want to survey.