Task suites

A task suite is a set of tasks grouped on a single web page. To create a task suite, upload JSON containing:

  • Input data for tasks (such as text or image URLs).

  • Correct responses (for control tasks).

  • Hints (for training tasks).

  • Parameters for assigning tasks.

A sample task suite in JSON format with key descriptions is shown below.

You can use the API to perform the following operations with a task suite:

Sample task suites in JSON

{
    "id": "63614047-38c3-4ad4-8a86-99c5c651a9b8",
    "pool_id": "1",
    "tasks": [
        {"id": "49a333ea-2728-4c1c-ab1f-8ab1bfe4ee7e",
         "origin_task_id": "e3da7fe1-828d-4d9c-b49d-42c0eb5fcfde",
         "input_values": {
                "image_url": "www.image1.ru"
            },
            "known_solutions": [
                {
                    "correctness_weight": 0.95,
                    "output_values": {
                        "colour": "black"
                    }
                },
                {
                    "correctness_weight": 0.7,
                    "output_values": {
                        "colour": "gray"
                    }
                }
            ],
            "message_on_unknown_solution": "The elephant is black"
        },
        {
            "input_values": {
                "image_url": "www.image2.ru"
            },
            "known_solutions": [
                {
                    "correctness_weight": 1,
                    "output_values": {
                        "colour": "white"
                    }
                }
            ],
            "message_on_unknown_solution": "The elephant is white"
        }
    ],
    "overlap": 5,
    "infinite_overlap": false,
    "remaining_overlap": 3,
    "reserved_for": [],
    "unavailable_for": [],
    "issuing_order_override": 3,
    "mixed": true,
    "automerged": false,
    "created": "2016-04-18T12:43:04.988"
}

Keys

Key

Value

Data type

Required during creation

id

ID of a task suite.

String

Not used

pool_id

The ID of the pool that tasks are uploaded to.

String

Yes

tasks[]

Data for the tasks.

Array

Yes

tasks[].id

Task ID if the task suite is created automatically with the “smart mixing” option (for more information, see the Requester's guide).

String

Not used

tasks[].origin_task_id
The ID of a task from another pool from which this task was copied for majority vote verification. String Not used
tasks[].input_values

Input data for a task. List of pairs:

"<ID of field 1>": "<value of field 1>",
"<ID of field 2>": "<value of field 2>",
...
"<ID of field N>": "<value of field N>"

JSON

Yes

tasks[].known_solutions

Responses and hints for control tasks and training tasks.

JSON

No

tasks[].known_solutions.output_values

Correct responses in a task (for control tasks). If multiple correct responses are possible, define output_values for each possible response and set the weight for the correct response (the correctness_weight key).

"<ID of field 1>": "<correct response>",
"<ID of field 2>": "<correct response>",
...
"<ID of field N>": "<correct response N>"            

JSON

No

tasks[].known_solutions.correctness_weight

The weight of a correct response. Allows you to set multiple versions of correct responses and rank them by correctness. For example, if a correct response is weighted 0.5, it counts as half of a mistake for the user. The more correct the response in correctValues, the higher its weight.

Floating-point number

No

tasks[].message_on_unknown_solution

Hint for the task (for training tasks).

String

No

overlap

Task suite overlap.

Number

No, if a task suite is created with the parameter
allow_defaults=true
and the overlap is specified in the pool parameters (the defaults.default_overlap_for_new_tasks key).
infinite_overlap
Assigning a task suite with infinite overlap. This option is used, for instance, for suites of training tasks when you want to assign them to all users:
  • true — Use infinite overlap.

  • false — Use the overlap that is set for the task suite or pool.

The default value is false.

Boolean

No

remaining_overlap

The remaining overlap for each task. For example, if the task has an overlap of 5 but was completed by two people, remaining_overlap is 3.

If the task is under review or active, the key is 0.

Number

Not used

reserved_for[]

IDs of users who will have access to the task suites.

Array

No

unavailable_for[]

IDs of users who shouldn't have access to the task suite.

Array

No

issuing_order_override

The priority of a task suite among other suites in the pool. Defines the order in which task suites are assigned to performers. The larger the parameter value, the higher the priority.

This parameter can be used if the pool has issue_task_suites_in_creation_order: true.

Allowed values: from -99999.99999 to 99999.99999.

By default the value is 0 .

Floating-point number

No

longitude
The longitude of the point on the map for the task suite.

Floating-point number

Yes, if tasks are chosen on the map. Otherwise, not used.

latitude

The latitude of the point on the map for the task suite.

Floating-point number

Yes, if tasks are chosen on the map. Otherwise, not used.

mixed

Type of operation for creating a task suite:

  • true — Automatically with the “smart mixing” option (for more information, see the Requester's guide).

  • false— Manually.

Boolean

No

automerged
Flag for whether a task suite is created after merging tasks. Value:
  • true — The task suite is generated as a result of merging identical tasks.
  • false — A standard task suite created by “smart mixing” or by the requester.
Boolean Not used
created

The UTC date and time when the task suite was created, in ISO 8601 format: YYYY-MM-DDThh:mm:ss[.sss].

String

Not used

Key

Value

Data type

Required during creation

id

ID of a task suite.

String

Not used

pool_id

The ID of the pool that tasks are uploaded to.

String

Yes

tasks[]

Data for the tasks.

Array

Yes

tasks[].id

Task ID if the task suite is created automatically with the “smart mixing” option (for more information, see the Requester's guide).

String

Not used

tasks[].origin_task_id
The ID of a task from another pool from which this task was copied for majority vote verification. String Not used
tasks[].input_values

Input data for a task. List of pairs:

"<ID of field 1>": "<value of field 1>",
"<ID of field 2>": "<value of field 2>",
...
"<ID of field N>": "<value of field N>"

JSON

Yes

tasks[].known_solutions

Responses and hints for control tasks and training tasks.

JSON

No

tasks[].known_solutions.output_values

Correct responses in a task (for control tasks). If multiple correct responses are possible, define output_values for each possible response and set the weight for the correct response (the correctness_weight key).

"<ID of field 1>": "<correct response>",
"<ID of field 2>": "<correct response>",
...
"<ID of field N>": "<correct response N>"            

JSON

No

tasks[].known_solutions.correctness_weight

The weight of a correct response. Allows you to set multiple versions of correct responses and rank them by correctness. For example, if a correct response is weighted 0.5, it counts as half of a mistake for the user. The more correct the response in correctValues, the higher its weight.

Floating-point number

No

tasks[].message_on_unknown_solution

Hint for the task (for training tasks).

String

No

overlap

Task suite overlap.

Number

No, if a task suite is created with the parameter
allow_defaults=true
and the overlap is specified in the pool parameters (the defaults.default_overlap_for_new_tasks key).
infinite_overlap
Assigning a task suite with infinite overlap. This option is used, for instance, for suites of training tasks when you want to assign them to all users:
  • true — Use infinite overlap.

  • false — Use the overlap that is set for the task suite or pool.

The default value is false.

Boolean

No

remaining_overlap

The remaining overlap for each task. For example, if the task has an overlap of 5 but was completed by two people, remaining_overlap is 3.

If the task is under review or active, the key is 0.

Number

Not used

reserved_for[]

IDs of users who will have access to the task suites.

Array

No

unavailable_for[]

IDs of users who shouldn't have access to the task suite.

Array

No

issuing_order_override

The priority of a task suite among other suites in the pool. Defines the order in which task suites are assigned to performers. The larger the parameter value, the higher the priority.

This parameter can be used if the pool has issue_task_suites_in_creation_order: true.

Allowed values: from -99999.99999 to 99999.99999.

By default the value is 0 .

Floating-point number

No

longitude
The longitude of the point on the map for the task suite.

Floating-point number

Yes, if tasks are chosen on the map. Otherwise, not used.

latitude

The latitude of the point on the map for the task suite.

Floating-point number

Yes, if tasks are chosen on the map. Otherwise, not used.

mixed

Type of operation for creating a task suite:

  • true — Automatically with the “smart mixing” option (for more information, see the Requester's guide).

  • false— Manually.

Boolean

No

automerged
Flag for whether a task suite is created after merging tasks. Value:
  • true — The task suite is generated as a result of merging identical tasks.
  • false — A standard task suite created by “smart mixing” or by the requester.
Boolean Not used
created

The UTC date and time when the task suite was created, in ISO 8601 format: YYYY-MM-DDThh:mm:ss[.sss].

String

Not used