Selective majority vote check

Before you start

To set up selective majority vote check, you need a project, a pool, and a file with tasks.

Theory

This function allows you to set up majority vote check for only a portion of tasks.

Depending on how many tasks the Toloker completed in N days, overlap for certain tasks will be higher or lower than in the pool settings.

How it works

When to use

Use selective majority vote check if:

  • There are few or no control tasks in the project.

  • You have experienced Tolokers whom you trust.

How verification tasks are assigned

  • In the open pool, the verification tasks that haven't gained enough overlap are assigned to a Toloker.

  • If there are no tasks of this kind in the open pool, they are transferred there from archived and closed pools that belong to the same project.

  • If in the project there are no verification tasks with an overlap that is less than the specified value, a random task from the current open pool is assigned as a verification task.

Practice

To add selective majority vote check:

  1. Upload tasks to the pool using smart mixing.

  2. Switch to pool editing mode.

  3. Under Smart mixing settings, click Selective increase in overlap for general tasks.

  4. Specify how many days the calculation should take into account, Tolokers, and overlap for majority vote tasks.

  5. Set the intervals of completed tasks and the number of review tasks for each interval.

Example

Tips and recommendations

  • Don't use selective majority vote check if:

    • There are many questions and many response options in the task.

    • Tolokers need to attach a file to their assignment.

    • Tolokers need to transcribe text or select objects in a photo, or other tasks where the Tolokers can't possibly provide the same responses.

  • Use selective majority vote check together with the fast responses and skipped assignments quality control rules. This way, you will eliminate dishonest Tolokers who skip tasks or complete them too quickly, which directly affects the number of completed tasks and overlap.

What's next

See also

Contact support

Last updated: June 23, 2023

Introduction
Getting started
Important tips
Useful recommendations
Working with Toloka
Projects
Pools
Tasks
Results
Project analysis
Toloka settings
Task interface
Template Builder
HTML/CSS/JS editor
Help and support
FAQTroubleshootingSupportGlossary