Setting up quality control

Quality control lets you get more accurate responses and restrict access to tasks for cheating performers. Quality control consists of rules. All rules work independently.

Attention.

Quality control rules that you set in the project are applied to all project pools, so you can't change them in just one of the pools.

To set up quality control:
  1. Go to the pool editing page
  2. If you already have a pool with the appropriate quality control settings, you can copy it along with the audience settings. To do this, go to Users filter and click Copy settings from... and then Add Quality Control Rule.

  3. Under Quality Control, choose the rules you want to use.

    If you aren't sure what quality control rules you need, select a quality control preset with default settings.
  4. Make settings for the rules you added. Below is a list of rules with links to detailed information about the rule settings.
  5. Save the pool.

List of rules

  • To keep track of how often performers make mistakes:
    • Control tasks: Use them to assign a skill to performers based on their responses to control tasks and ban performers who submit incorrect responses.
    • Majority vote: Quality is based on matching the response from the majority of performers who complete the same task.
    • Results of checking: Evaluate performers based on the number of accepted and rejected responses.
  • To protect your project from fake users (robots) and cheaters:
    • Fast responses: Control the minimum time that must be spent on a task page.
    • Captcha: Show a captcha from time to time to make sure tasks aren't completed by robots.
    • Skipped assignments: Restrict access to pool tasks for performers who skip several task pages in a row.
  • To attract a variety of performers:
    • Earnings: Limit the amount each performer can earn in the pool per day.
    • Submitted assignments: Limit how many assignments each performer can submit in the pool per day.
  • To allow recompletion of certain assignments: