Tasks
In order to upload tasks to a pool, you need to create JSON objects that contain:
The input data for the tasks (such as text or image URLs).
Correct responses (for control tasks).
Hints (for training tasks).
Toloka automatically combines tasks into suites. You only need to specify the number of tasks to include in each suite (the mixer_config
key in the pool).
If you want to create the task suites yourself, follow the instructions in Task suites. For more information about creating task suites, see the Toloka. Requester's guide.
A sample task in JSON with key descriptions is shown below.
You can use the API for performing the following operations with tasks:
Sample task in JSON
{
"id":"95199da8-77f4-4ceb-8316-2916a84b4641",
"origin_task_id": "e3da7fe1-828d-4d9c-b49d-42c0eb5fcfde",
"pool_id": "1",
"input_values": {
"image_url": "www.images/image1.ru"
},
"known_solutions": [
{
"output_values": {
"result": "OK",
"like": false
},
"correctness_weight": 0.8
},
{
"output_values": {
"result": "OK",
"like": true
},
"correctness_weight": 1
},
],
"baseline_solutions": [
{
"output_values": {
"result": "OK",
"like": false
},
"confidence_weight": 0.8
},
{
"output_values": {
"result": "OK",
"like": true
},
"confidence_weight": 1
}
],
"message_on_unknown_solution": "The cat is happy",
"overlap": 3,
"infinite_overlap": false,
"reserved_for": [],
"unavailable_for": [],
"created": "2016-09-07T08:49:27.272"
}
Keys
Key | Value | Data type | Required during creation |
---|---|---|---|
| Task ID. | String | Not used |
| The ID of a task from another pool from which this task was copied for majority vote verification. | String | Not used |
| The ID of the pool that the task is uploaded to. | String | Yes |
| Input data for a task. List of pairs: | JSON | Yes |
| Correct responses for control and training tasks. You can specify several options for a correct task response. If one option is more correct than another, you can assign different weights to the response options. To do this, use the | Array | No |
| Output data values to check. You should specify values for all required output data fields. | JSON | Yes |
| The weight of a correct response in the range from 0 to 1. Lets you count a response as partially correct. This is convenient when there is no single right response to the task. This works like awarding points: if you need to complete one control task correctly to get a skill (receive 1 point), you may complete one task with a weight of 1 or two tasks with a weight of 0.5 or higher. The default value is 1. | Floating-point number | No |
| Preliminary responses. This data simulates performer responses when calculating Define For example, you ask performers to label whether an image shows a cat or a dog. Suppose your neural network already determined that the image might show a dog with a probability of 80% and a cat with a probability of 40%. Let's say you set dynamic overlap from 1 to 3 and the minimum response confidence at 85%. If the performer answers "Dog" and the confidence in their response is high, the overlap most likely won't increase because one response is enough. If the performer answers "Cat", the confidence is most likely not high enough and the overlap will increase further. Can't be used when creating a task suite: an error with code 400 saying | Array | No |
| Output data values for preliminary responses. | JSON | Yes |
| Confidence in a response, from 0 to 1. The default value is 1. | Floating-point number | Yes |
| Hint for the task (for training tasks). | String | No |
| Task suite overlap. | Number | No, if the parameter is used when creating tasks and the overlap is specified in the pool parameters (key defaults.default_overlap_for_new_tasks) |
| Assigns tasks with infinite overlap. For instance, you can use this for training tasks when you want to assign them to all users:
The default value is | Boolean | No |
| ID of the task it was copied from | String | No |
| IDs of users who will have access to the task. | Array | No |
| IDs of users who shouldn't have access to the task. | Array | No |
| The UTC date and time when the task suite was created, in ISO 8601 format: YYYY-MM-DDThh:mm:ss[.sss]. | String | Not used |
Key | Value | Data type | Required during creation |
---|---|---|---|
| Task ID. | String | Not used |
| The ID of a task from another pool from which this task was copied for majority vote verification. | String | Not used |
| The ID of the pool that the task is uploaded to. | String | Yes |
| Input data for a task. List of pairs: | JSON | Yes |
| Correct responses for control and training tasks. You can specify several options for a correct task response. If one option is more correct than another, you can assign different weights to the response options. To do this, use the | Array | No |
| Output data values to check. You should specify values for all required output data fields. | JSON | Yes |
| The weight of a correct response in the range from 0 to 1. Lets you count a response as partially correct. This is convenient when there is no single right response to the task. This works like awarding points: if you need to complete one control task correctly to get a skill (receive 1 point), you may complete one task with a weight of 1 or two tasks with a weight of 0.5 or higher. The default value is 1. | Floating-point number | No |
| Preliminary responses. This data simulates performer responses when calculating Define For example, you ask performers to label whether an image shows a cat or a dog. Suppose your neural network already determined that the image might show a dog with a probability of 80% and a cat with a probability of 40%. Let's say you set dynamic overlap from 1 to 3 and the minimum response confidence at 85%. If the performer answers "Dog" and the confidence in their response is high, the overlap most likely won't increase because one response is enough. If the performer answers "Cat", the confidence is most likely not high enough and the overlap will increase further. Can't be used when creating a task suite: an error with code 400 saying | Array | No |
| Output data values for preliminary responses. | JSON | Yes |
| Confidence in a response, from 0 to 1. The default value is 1. | Floating-point number | Yes |
| Hint for the task (for training tasks). | String | No |
| Task suite overlap. | Number | No, if the parameter is used when creating tasks and the overlap is specified in the pool parameters (key defaults.default_overlap_for_new_tasks) |
| Assigns tasks with infinite overlap. For instance, you can use this for training tasks when you want to assign them to all users:
The default value is | Boolean | No |
| ID of the task it was copied from | String | No |
| IDs of users who will have access to the task. | Array | No |
| IDs of users who shouldn't have access to the task. | Array | No |
| The UTC date and time when the task suite was created, in ISO 8601 format: YYYY-MM-DDThh:mm:ss[.sss]. | String | Not used |