Online survey

Tip.

Run the project in the Sandbox first. This helps you avoid making mistakes and spending money on a task that isn't working right.

  1. Create a project
  2. Add a task pool
  3. Upload tasks
  4. Set up quality control
  5. Start the pool and get the results
  6. Troubleshooting

Surveys are designed to collect information for research purposes. The number of respondents is set in overlap, and the requirements for them are set with filters.

You may need additional projects for your task, such as dataset pre-check or checking performers' responses. Learn more about this in the Designing the solution architecture section.

Suppose you want to know where Russian-speaking respondents have breakfast and what they prefer. To do this, create a single task on a page and in the pool, so the performer sees several questions with response options.

Example of a prepared task

To run tasks and get responses:

Create a project

The project defines what the task will look like for a performer.

  1. Click the + Create project button and choose Blank.

  2. We recommend including the word "survey" and the survey topic in the Project name. Briefly explain in the Description what the performer has to do in the task. Performers will see this in the task list.
  3. For a survey, instructions aren't required. Place the recommendations for filling out the survey in the task interface.
    Note. Don't leave your contact information. This contradicts the Customer Service Agreement.
  4. Note. This tutorial shows how to create a task interface in Yandex.Toloka. You can also try creating a task interface in the Template Builder.
    Define which objects you are going to pass to the performers and receive from them in response. To do this, add input and output fields in the Specifications block.
    What are input and output data?

    Input data is types of objects that are passed to the performer for completing the task. For example, this could be a text, an image, or geographic coordinates.

    Output data is types of objects that you receive after the task is completed. For example, this could be one of several response options, typed text, or an uploaded file.

    Learn more about input and output data fields.

    Add the fields:

    • Input data field — The theme string to upload a survey. It is not used in the interface and can contain any text.
    • Output data fields:
      • The required place string to record the performer's response to the question "Where do you usually have breakfast?".;
      • The required string food to record the performer's response to the question "What do you eat for breakfast most often?".;
      • The required string drink to record the performer's response to the question "What do you drink for breakfast (besides water)?".
  5. Create the task interface in the HTML block. It describes how the task elements should be arranged in the task.

    You can use standard HTML tags and special expressions in double curly brackets for input and output data fields.

    <div>
      <div>Where do you usually have breakfast?</div>
    {{field type="radio" name="place" value="home" label="At home"}}
    {{field type="radio" name="place" value="university" label="At school"}}
      {{field type="radio" name="place" value="work" label="At work"}}
      {{field type="radio" name="place" value="cafe" label="In a cafe"}}<br/><br/><br/>
    </div>
    <div>
      <div>What do you eat for breakfast most often?</div>
      {{#field type="select" name="food" placeholder="" width="300px"}}
      {{select_item value="kasha" text="Porridge/Muesli/Cold cereal"}}
      {{select_item value="egg" text="Eggs (fried/boiled/omelet)"}}
      {{select_item value="sandwich" text="Sandwich/Pastry/Pancakes/Toast"}}
      {{select_item value="curd" text="Cottage cheese/Yogurt/Other dairy"}}
    {{select_item value="other" text="Other"}}
      {{/field}}<br/><br/><br/>
    </div>
    <div>
      <div>What do you drink at breakfast?</div>
      {{field class="option" type="radio" name="drink" value="coffee" label="Coffee/Tea"}}
      {{field class="option" type="radio" name="drink" value="milk" label="Milk"}}
      {{field class="option" type="radio" name="drink" value="juice" label="Juice"}}
      {{field class="option" type="radio" name="drink" value="other" label="Other"}}<br/><br/><br/>
    </div>
    <div>
      Thank you for responding to the survey!
    </div>
    This means that the performer will see three questions with response options.

    Clear the JavaScript block. Leave the CSS block empty.

  6. Click the Preview button to see the performer's view of the task.
    Note. The project preview shows one task with standard data. You can define the number of tasks to show on the page later.
  7. Save the project.

Add a task pool

A pool is a set of paid tasks sent out for completion at the same time.

  1. Open the project and click Add pool.
  2. Give the pool any convenient name and description. The pool info is only available to you. Performers can view only the project name and description.
  3. Set the price per task page (for instance, $0.01). In a survey, a task page must contain one task. If you need help rating tasks, read about setting up pricing and see examples of prices for different types of tasks.
  4. Specify the Time for task completion (for instance, 600 seconds).
  5. Set the Overlap, which is the number of performers to complete the same task. At first, set a small number like 100. If you see that this is not enough for sampling, increase the overlap.
  6. Add Filters to choose performers.
    Run the task both in the web version and in mobile apps to speed up task completion. To do this, add the Client filter.
  7. Save the pool.

Upload tasks

Prepare your own task file. Check out the example in a demo TSV file. You can find it on the pool page. At the top-left of the page, there are links to TSV files with regular, control, and training tasks.
  1. Click Upload. In the window that opens, you can also download a sample TSV file by clicking Sample file for uploading tasks.
    What is TSV?
    A TSV file presents a table as a text file in which columns are separated by tabs.

    You can work with it both in a table editor and a text editor, and then save it to the desired format. More about working with a TSV file. There is a CSV format that is similar to TSV, but you should use a TSV file for uploading.

  2. Add input data in it. The header of the input data column contains the word INPUT. To upload a task, add any text to the INPUT column. The string from the input data field isn&apos;t used in the interface. Leave the other columns empty.
  3. Load tasks by choosing Set manually and setting 1 task per page.

Set up quality control

Quality control rules allow you to filter out inattentive performers. You can configure quality control both in the project and in the pool.

    Go to pool editing (the Edit button in the upper-right corner of the page) and click Add Quality Control Rule.

    You can copy quality control settings from another pool. To do this, click Copy settings from in the Users filter section.

  1. Add the Fast responses rule and enter the following values:

    This means that a user who completes a task page in less than 10 seconds will be banned for 10 days and won't be able to complete project tasks.

  2. Add the Banned users' reassessment block.

    For surveys, always add both rules, since they're only effective when paired up for such tasks. Surveys usually only have one task page. This rule complements fast responses so that you get the necessary number of responses, and not just ban those who respond too quickly.

    Attention. Recompletion of assignments from banned users increases the overlap, hence the budget, too.

    Reject assignments according to the conditions described in the instructions. To maintain the reputation of a good requester, don't automatically reject fast responses like that. It's easy to fill out a simple surveys in less than 20 seconds.

    This means that if a performer is banned for fast responses and the responses aren't taken into account, this task is reassigned to another user for completion.

Start the pool and get the results

  1. Start the pool by clicking .
  2. Track the completion of tasks in the Pool statistics section.
  3. When the pool is completed, launch aggregation of results. To do this, find the Download results button and click  → Dawid-Skene aggregation model next to it.

    Aggregation of responses is necessary to get a complete picture of all results. Learn more about aggregation.

  4. Track the aggregation progress on the Operations page. When the process is completed, click Download.
  5. If the performers&apos; responses aren&apos;t enough for analysis, increase the Overlap to receive more responses.
    How do I increase overlap?
    1. Stop the pool by clicking .
    2. In the upper-right corner, click Edit.
    3. Change the value in the Overlap block.
    4. Click the Save button and start the pool.

Troubleshooting

How do I create a simple survey with no options, where the user writes an arbitrary response?
  1. Create a project from an empty template.
  2. Write your question in the HTML block.
  3. If you need an extensive response, add the required number of text entry components.
  4. If you need a short response, add the required number of string entry components.
  5. Come up with a name for each of the components and create a matching number of string-type output fields with the same names. They will be used to save responses.
  6. Make all the output fields mandatory.
  7. Make the input field auxiliary. It will only be used to create a TSV file with tasks.
  8. Come up with a name for the input field and set the string type for it (see the step-by-step guide on creating a survey).