Control tasks

Control tasks contain correct responses. They are used for calculating the user's percentage of correct responses. Depending on the result, you can either change the user's skill value, or block access to tasks.

Add tasks with correct responses to the pool. Set key values in the quality_control.configs array in the pool settings.

Example for a JSON block

Let's say that the following settings are made in the project:

  • Calculating the skill value equal to the percentage of correct responses in control tasks. You can use the skill value for filtering users.

  • Denying access to the project if less than 75% of the performer's responses are correct.

The first skill calculation is made when 7 control tasks are completed. The calculation uses the last 10 responses from the project's control tasks.

You can ban a performer from accessing the tasks in the project for a given number of days, hours, minutes (at a time or in total) or forever.

Ban for 10 days
{
   "configs": [
      {
         "collector_config": {
            "type": "GOLDEN_SET",
            "parameters": {
               "history_size": 10
            }
         },
         "rules": [
            {
               "conditions": [
                  {
                     "key": "golden_set_answers_count",
                     "operator": "GT",
                     "value": 7
                  }
               ],
               "action": {
                  "type": "SET_SKILL_FROM_OUTPUT_FIELD",
                  "parameters": {
                     "skill_id": "42",
                     "from_field": "golden_set_correct_answers_rate"
                  }
               }
            },
            {
               "conditions": [
                  {
                     "key": "golden_set_answers_count",
                     "operator": "GT",
                     "value": 7
                  },
                  {
                     "key": "golden_set_correct_answers_rate",
                     "operator": "LT",
                     "value": 75.0
                  }
               ],
               "action": {
                  "type": "RESTRICTION_V2",
                  "parameters": {
                     "scope": "PROJECT",
                     "duration_unit": "DAYS",
                     "duration": 10,
                     "private_comment": "Control tasks not passed"
                  }
               }
            }
         ]
      }
   ]
}

To set a different ban period, change the applicable parameter for the action key:

{
   ...
               "action": {
                  "type": "RESTRICTION_V2",
                  "parameters": {
                     "scope": "PROJECT",
                     "duration_unit": "HOURS",
                     "duration": 12,
                     "private_comment": "Control tasks not passed"
                  }
               }
   ...
}

Keys

Key

Value

Data type

Required during creation

collector_сonfig.parameters.history_size

The maximum number of the user's recent responses in the project to use for calculating the percentage of correct responses.

If this field is omitted, the calculation is based on all the user's responses in the pool.

Integer

No

rules.conditions.key

Parameters that are checked in the condition:

  • total_answers_count — The number of completed control tasks.

  • correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).

  • incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

  • golden_set_answers_count — The number of completed control tasks.
  • golden_set_correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).
  • golden_set_incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

String

Yes

Key

Value

Data type

Required during creation

collector_сonfig.parameters.history_size

The maximum number of the user's recent responses in the project to use for calculating the percentage of correct responses.

If this field is omitted, the calculation is based on all the user's responses in the pool.

Integer

No

rules.conditions.key

Parameters that are checked in the condition:

  • total_answers_count — The number of completed control tasks.

  • correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).

  • incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

  • golden_set_answers_count — The number of completed control tasks.
  • golden_set_correct_answers_rate — The percentage of correct responses in training and control tasks (from 0 to 100).
  • golden_set_incorrect_answers_rate — The percentage of incorrect responses in training and control tasks (from 0 to 100).

String

Yes