Skipped assignments

Skipping tasks is considered an indirect indicator of the quality of responses. You can block access to a pool or project if a Toloker skips multiple task suites in a row. To do this, set key values in the quality_control.configs array in the pool settings.

Request body

You can block a Toloker from accessing the project for a given number of days, hours, minutes (at a time or in total), or permanently, if they skip 10 task suites in a row.

Ban for 10 days

{
"configs": [
{
"collector_config": {
"type": "SKIPPED_IN_ROW_ASSIGNMENTS"
},
"rules": [
{
"conditions": [
{
"key": "skipped_in_row_count",
"operator": "GTE",
"value": 10
}
],
"action": {
"type": "RESTRICTION_V2",
"parameters": {
"scope": "PROJECT",
"duration_unit": "DAYS",
"duration": 10,
"private_comment": "Skipped more than 10 pages in a row"
}
}
}
]
}
]
}

To set a different ban period, change the

duration_unit
and
duration
parameters for the action key:
for 12 hours
for 30 minutes
permanently
{
...
"action": {
"type": "RESTRICTION_V2",
"parameters": {
"scope": "PROJECT",
"duration_unit": "HOURS",
"duration": 12,
"private_comment": "Skipped more than 10 pages in a row"
}
}
...
}
ParameterOverview
configs[]array of objects | required

Array of quality control settings.
configs[].collector_configobject | required

Parameters for collecting statistics (for example, the number of tasks skipped in the pool).
configs[].collector_config.typestring | required

Criteria for the quality control rule:

  • GOLDEN_SET — The number of correct and incorrect control task responses.
  • MAJORITY_VOTE — The percentage of task responses that matched the majority vote.
  • INCOME — Payment for tasks completed by the Toloker over the past 24 hours.
  • SKIPPED_IN_ROW_ASSIGNMENTS — The number of task suites skipped in a row.
  • ANSWER_COUNT — The number of task suites completed by the Toloker in the pool.
  • ASSIGNMENT_SUBMIT_TIME — The number of "fast" task suite responses (the minimum response speed is set in the parameters).
  • ACCEPTANCE_RATE — The percentage of Toloker responses that were rejected during manual review.
  • ASSIGNMENTS_ASSESSMENT — The number of assignments accepted or rejected with manual review enabled.
  • USERS_ASSESSMENT — The Toloker's skill value and their bans.
configs[].collector_config.parameters

object | required if

Required if configs[].collector_config.type is equal to one of the values:

  • GOLDEN_SET
  • MAJORITY_VOTE
  • ASSIGNMENT_SUBMIT_TIME

Parameters for collecting data (depends on the quality control rule specified in the type key).

configs[].rulesobject | required

Parameters for the control rule conditions and actions.

configs[].rules.conditionsobject | required

Conditions (for example, 10 task suites skipped in a row). Multiple conditions are combined with the AND operator.
configs[].rules.conditions.key

string | required

Values that are checked in the condition:

  • skipped_in_row_count — The number of tasks skipped in a row.
configs[].rules.conditions.operator

string | required

Comparison operator (the key data is compared with the threshold value from value):

  • EQ — Equal to.
  • NE — Not equal to.
  • GT — Greater than.
  • LT — Less than.
  • GTE — Greater than or equal to.
  • LTE — Less than or equal to.
configs[].rules.conditions.valueinteger | required

The threshold value of the variable specified in key.
configs[].rules.actionobject | required

The action to perform if conditions are met (for example, block access to the project).
configs[].rules.action.type

string | required

Type of action:

  • RESTRICTION — Ban access to projects or pools.

  • SET_SKILL_FROM_OUTPUT_FIELD — Set the "percentage of correct responses" as the skill value (used in control tasks and majority vote rules).

    You can use the skill value for filtering Tolokers.

  • CHANGE_OVERLAP — Change the overlap. For example, to re-assign a task suite to other Tolokers or cancel the recompletion of already accepted assignments.

  • REJECT_ALL_ASSIGNMENTS — Reject all Toloker responses. For example, after a certain number of Toloker responses, it became clear that the Toloker completed tasks poorly.

  • APPROVE_ALL_ASSIGNMENTS — Accept all Toloker responses. For example, if the Toloker completes most tasks well and you are satisfied with this result.

  • SET_SKILL — Assign the specified constant value to the skill.

configs[].rules.action.parametersobject | required

Action parameters.
configs[].rules.action.parameters.scope

string | required

Scope:

  • POOL — pool. Affects the Toloker's rating.
  • PROJECT — The project. Affects the Toloker's rating.
  • ALL_PROJECTS — All the requester's projects.
configs[].rules.action.parameters.skill_idstring | required if

Required if type=SET_SKILL_FROM_OUTPUT_FIELD.

ID of the skill to update as tasks are completed.
configs[].rules.action.parameters.from_field

string | required if

Required if type=SET_SKILL_FROM_OUTPUT_FIELD.

The value to assign to the skill:

  • correct_answers_rate — The percentage of correct responses.
  • wrong_answers_rate — The percentage of incorrect responses.
configs[].rules.action.parameters.skill_valueinteger | required if

Required if type=SET_SKILL_FROM_OUTPUT_FIELD.

A fixed value to assign to the skill (a number from 0 to 100).
configs[].rules.action.parameters.deltainteger | required if

Required if type=CHANGE_OVERLAP.

The value determines the amount to change the overlap by.
configs[].rules.action.parameters.public_commentstring | required if

Required if type=REJECT_ALL_ASSIGNMENTS.

Comments (the reason for rejecting responses). Available to the requester and the Toloker.
configs[].rules.action.parameters.duration_unitstring

Ban duration unit:

  • MINUTES — Minutes.
  • HOURS — Hours.
  • DAYS — Days.
  • PERMANENT — Permanent ban.
configs[].rules.action.parameters.durationinteger

Ban duration.

configs[].rules.action.parameters.private_commentstring

Comments (the reason for blocking access). Visible only to the requester.
configs[].rules.action.parameters.open_pool

boolean

Determines whether to re-open a closed pool:

  • true — Open the pool after making changes if it is closed.
  • false — Don't open the pool after making changes, if it is closed.

See also

Contact support

Last updated: August 8, 2023

Toloka API
OverviewAccessing the APIRate limiting
Quick start
Configuration for Template BuilderToloker selection
Quality control rules
Filters
Task actions
Subscriptions to events
Reference
Project
Pool
Training
Webhook-subscription
Skill
User-restriction
Task
Task-suite
Operation
Assignment
Attachment
Aggregated-solution
User-bonus
Message-thread
User-metadata
Requester
All classes and methods