Majority vote

Majority vote is a quality control method based on matching responses from the majority of performers who complete the same task. For example, if a task has an overlap of “5” and three performers selected the same answer, this is considered indirect confirmation of the correct response. Depending on the percentage of correct responses, you can increase the value of the performer's skill or block their access to tasks.

The rule takes effect when the number of responses for the task is equal to the overlap. To get the required number of responses faster, enable the Keep task order option in the pool settings.

Rule settings


Field	Description
Accept as majority	The number of matching responses that is considered the “majority vote” (for example, 3). The value must be less than or equal to the overlap.
History size	The number of the performer's recent responses. To calculate task responses from all project pools, fill in the field in the rule for each pool. If this field is not filled in, the calculation includes only task responses in the pool to which the rule applies.
If	A condition for performing the action in the then field: number of responses — The number of completed tasks. % correct answers — The percentage of correct responses, meaning responses that matched the majority opinion (from 0 to 100). % incorrect answers — The percentage of incorrect responses, meaning responses that didn't match the majority opinion (from 0 to 100). To add multiple conditions, click .
then	Action to perform for the condition: assign skill value — Assign a fixed value to the skill. suspend — Suspend the performer's access to the pool for the specified number of days. Only the requester can view the reason for the ban. assign skill from the field — Save the percentage of the performer's correct responses in tasks as a skill value. ban — Block access to the project or all requester's projects for the specified number of days. Only the requester can view the reason for the ban. If access to tasks is blocked temporarily (for example, for 7 days), the history of the performer's responses is not saved after the ban is lifted. The skill value is based on new responses.


Field	Description
Accept as majority	The number of matching responses that is considered the “majority vote” (for example, 3). The value must be less than or equal to the overlap.
History size	The number of the performer's recent responses. To calculate task responses from all project pools, fill in the field in the rule for each pool. If this field is not filled in, the calculation includes only task responses in the pool to which the rule applies.
If	A condition for performing the action in the then field: number of responses — The number of completed tasks. % correct answers — The percentage of correct responses, meaning responses that matched the majority opinion (from 0 to 100). % incorrect answers — The percentage of incorrect responses, meaning responses that didn't match the majority opinion (from 0 to 100). To add multiple conditions, click .
then	Action to perform for the condition: assign skill value — Assign a fixed value to the skill. suspend — Suspend the performer's access to the pool for the specified number of days. Only the requester can view the reason for the ban. assign skill from the field — Save the percentage of the performer's correct responses in tasks as a skill value. ban — Block access to the project or all requester's projects for the specified number of days. Only the requester can view the reason for the ban. If access to tasks is blocked temporarily (for example, for 7 days), the history of the performer's responses is not saved after the ban is lifted. The skill value is based on new responses.

Examples of rules

Examples are provided for simple classification. There are 10 tasks per page.

Calculating a skill and banning for incorrect responses

Both rules work independently:

If the performer gives at least 3 responses to the tasks, the percentage of correct answers is written as the skill value.
If the performer gives at least 3 answers to tasks and the percentage of correct answers is less than 65%, they are banned on the project.

The calculation uses up to 10 performer's responses to the project tasks.

The performer is blocked after the first incorrect response to the first, second or third task. The skill is not assigned. Since the ban reason is not specified, there is no way to find out why the performer is banned.

All rules are applied independently:

If the performer gives at least 3 responses to the tasks, the percentage of correct answers is written as the skill value.
If the performer gives 2 incorrect responses to 3 tasks, they are blocked in the pool for 10 days.
If the performer gives 2 incorrect responses to 4 tasks, they are blocked in the pool for 10 days.
If the performer gives 5 or more responses to tasks and the percentage of correct responses is less than 80%, they are blocked in the pool for 10 days.

A set of rules like this prevents performers from being banned for one incorrect response and lets you maintain high accuracy.

Setting the skill

If the performer completes 3 tasks, the skill is set to the percentage of correct answers.

Use the skill value to set access to other pools with filters.

Example of filter settings

This rule will never take effect because the number of responses counted (Recent values to use) is less than the number of responses in the rule (number of responses).

Blocking for incorrect responses

If the percentage of correct responses is less than 40%, the performer is blocked on the project for 30 days.

If the percentage of correct responses is less than 40%, the performer is blocked on the project for 30 days. The rule will be applied once, after the fifth response.

Contact support