- Yes — Aggregate responses by skill (
- No — Aggregate responses using the Dawid-Skene method (
The Dawid-Skene aggregation model works with control and training tasks as well as with main tasks. Responses specified for control tasks might not match the actual responses to the task if performers often submit an incorrect response to it.
The Dawid-Skene aggregation model automatically evaluates
|L|²parameters for each performer, where
Lis the number of different aggregation values. Note that these parameters are determined automatically and are only used in calculations.
Because the Dawid-Skene method evaluates
|L|²parameters for each performer, we don't recommend using it when the performer labels
< |L|²tasks. In this case, the quality of aggregation may be poor.
Aggregation via the interface and via the API works differently. When using the Dawid-Skene aggregation model, you can't aggregate more than one field via the API. You can only do that via the interface.