Review results

If is set in the pool<, add a rule to:

Set the performer's skill based on their responses.
Block access for performers who give incorrect responses.

To have rejected tasks redistributed to other performers, set up the Recompletion of rejected assignments rule.

Attention.

The If and then fields in this rule are mandatory. If you don't fill in at least one field, you won't be able to save the rule.

Rule settings


Field	Description
Recent values to use	The number of the performer's recent responses. To calculate task responses from all project pools, fill in the field in the rule for each pool. If this field is not filled in, the calculation includes only task responses in the pool to which the rule applies.
If	A condition for performing the action in the then field: total reviewed responses — The number of the performer's assignments that were reviewed. accepted responses (%) — The percentage of the performer's assignments that were accepted (from 0 to 100). rejected responses (%) — The percentage of the performer's assignments that were rejected (from 0 to 100). To add multiple conditions, click .
then	Action to perform for the condition: assign skill from the field — Save the percentage of performer's accepted responses as the skill value. Ban — Block access to the project or all requester's projects for the specified number of days. Only the requester can view the reason for the ban. If access to tasks is blocked temporarily (for example, for 7 days), the history of the performer's responses is not saved after the ban is lifted. The skill value is based on new responses. Assign skill value — Assign a fixed value to the skill. accept user's answers — Requires the non-automatic acceptance option to be set. Useful if the performer completes most tasks well. Example: the toloker completed more than 80% of the tasks correctly and you are satisfied with this result. The rule will work automatically — all responses in the pool will be accepted. decline user's answers — Requires the non-automatic acceptance option to be set. Useful if you need several responses from the performer to evaluate the quality of the results. Example: after ten responses, it became clear that the performer completes tasks poorly. The configured action will work — all responses in the pool will be rejected. Suspend — Suspend the performer's access to the pool for the specified number of days. Only the requester can view the reason for the ban.


Field	Description
Recent values to use	The number of the performer's recent responses. To calculate task responses from all project pools, fill in the field in the rule for each pool. If this field is not filled in, the calculation includes only task responses in the pool to which the rule applies.
If	A condition for performing the action in the then field: total reviewed responses — The number of the performer's assignments that were reviewed. accepted responses (%) — The percentage of the performer's assignments that were accepted (from 0 to 100). rejected responses (%) — The percentage of the performer's assignments that were rejected (from 0 to 100). To add multiple conditions, click .
then	Action to perform for the condition: assign skill from the field — Save the percentage of performer's accepted responses as the skill value. Ban — Block access to the project or all requester's projects for the specified number of days. Only the requester can view the reason for the ban. If access to tasks is blocked temporarily (for example, for 7 days), the history of the performer's responses is not saved after the ban is lifted. The skill value is based on new responses. Assign skill value — Assign a fixed value to the skill. accept user's answers — Requires the non-automatic acceptance option to be set. Useful if the performer completes most tasks well. Example: the toloker completed more than 80% of the tasks correctly and you are satisfied with this result. The rule will work automatically — all responses in the pool will be accepted. decline user's answers — Requires the non-automatic acceptance option to be set. Useful if you need several responses from the performer to evaluate the quality of the results. Example: after ten responses, it became clear that the performer completes tasks poorly. The configured action will work — all responses in the pool will be rejected. Suspend — Suspend the performer's access to the pool for the specified number of days. Only the requester can view the reason for the ban.

Examples of rules

Blocking for incorrect responses

If more than 50% of performer's responses are rejected, the performer is blocked and can't complete your tasks for 15 days.

The performer is blocked after the first rejected response, because the number of the reviewed tasks is not specified.

Setting the skill

After 3 tasks are reviewed, the skill is set to the percentage of correct responses. Use the skill value to set access to other pools with filters.

Example of filter settings

Troubleshooting

Should I create a skill for every pool?

It is better to use one skill in a project. You can choose the way to calculate the skill:

Calculate the skill for each pool separately. The current skill value is the value of the skill in the pool the user completed last. This option is convenient if:
- The pools are intended for different groups of performers (for example, there are filters by city or country).
- Pools are started one by one and you don't want to take into account the responses in the previous pools to calculate the skill in the current pool.
This calculation method is used by default when adding a quality control rule to a pool. For the control tasks block, leave the Recent values to use field empty.
Calculate skill based on all tasks in a project This option is good if the pools are small and you don't need to have skill calculated for each pool.

This option is available only for skills on control tasks. To use it, fill in the Recent values to use field in quality control rules in pools.

Can I use a skill beyond a particular pool or project and apply it to other projects as well?

Yes, of course — you can use the same skill for different projects. But most often, a skill is intended for a specific project. If the performer completes a certain task well, this doesn't mean that they will complete other ones successfully. Another disadvantage is that if you filter by skills that were set long ago, you will artificially limit the number of available performers.

Can I disable tasks for performers who complete the tasks poorly?

You can deny access to the pool if the performer's responses are too fast, if they don't match the majority opinion or if the performer makes many mistakes in control tasks. Tasks completed by such performers can be given to other performers.

Can I ask a performer to redo the task if they made mistakes in it?

No. After sending a task, the performer can't make any changes to it. You can add tasks that were completed incorrectly to a new pool.

Contact support