Project 1. Does the photo contain an item?

In this project, you'll ask performers if there are shoes in the photo.

  1. Create a project
  2. Create a pool
  3. Upload a TSV file with tasks.
  4. Receive the results

Create a project

To create a project:

  1. Click + Create project+ Create project.

  2. In the window that opens, select the Image categorization template. To do this, hover over the template and click Choose.

  3. In the window that opens, fill in the project parameters.

    1. In the Project title field, enter Are there shoes in the photo?.

    2. In the Description field, enter Look at the picture and tell if it has shoes..

    3. In the Instructions field, enter the instructions and add an image. To add an image, click and provide a link to the image you want to use as an example.

      Instruction text:

      Look at the picture and determine if there are shoes in the picture. 
      If there are, click Yes. 
      If there isn's, click No.
      For example, there are shoes in the photo, so the correct answer is Yes.
  4. In the Specifications block, delete the like field:

    1. Point the cursor at the field and click the button.

    2. Click the Remove button.

    Read more about the Specification parameters in the Input and output data section.

  5. In the Task interface, edit the html block:

    1. Delete the row responsible for the like field:

      {{field type="checkbox" name="like" label="Do you like the photo?" hotkey="q"}}
    2. After the image line, add the question:

      <div>Are there <b>shoes</b> in the picture?<div>
    3. Change the labels on the response options: GoodYes, BadNo:

      {{img src=image width="100%" height="400px"}}
      <div>Are there <b>shoes</b> in the picture?<div>
      
      {{field type= "radio" name= "result" value= "OK" label="Yes" hotkey="1"}}
      {{field type= "radio" name="result" value= "BAD" label="No" hotkey="2"}}
      {{field type= "radio" name="result" value= "404" label="Loading error" hotkey="3"}}
  6. Leave the JS and CSS blocks unchanged.

  7. Click the Preview button.

  8. In the window that opens, check if the task options work correctly.

    1. Answer the question.

    2. In the lower right corner, click Send.

  9. Exit the preview mode.

    1. In the lower left corner, click Exit.

    2. In the window that opens, click Exit.

  10. In the window that opens, click Finish editing.

Read more about working with the project in the Project section.

Create a pool

To create a pool:

  1. Open the page of the project titled Are there shoes in the photo?.

  2. Click the Add pool button.

  3. Specify the pool name.

  4. (optional) Enter the Private description.

    1. Turn on the Add a private description option.

    2. In the Private description field, enter the pool description. This information is available only to you.

  5. In the Price per task page section, in the Price in US dollars field, specify 0.01.

  6. In the Performers section, filter performers by language:

    1. Click Add filter.

    2. Find the Profile block in the list and choose the Languages skill.

    3. In the ? field specify =.

    4. In the Value field, specify Russian.

  7. In the Quality control section, specify quality control settings for the pool:

    1. Click Add quality control rule.

    2. Find the Rules block in the list and choose Control tasks.

    3. Set a rule for control tasks: If the number of responses to control questions is greater than or equal to 3 and the percentage of correct responses to control questions is less than 60, then restrict the performer's access to the project for 10 days. Specify the Control task as a reason.

      Read more about quality control in the Quality control section.

  8. In the Overlap section, go to the Overlap field and enter 3.

  9. (optional) In the Speed/quality section, specify the percentage of top-rated users who will get access to the pool. Read more about this in Speed/quality balance.

  10. In the Parameters section, go to Time per task page in seconds and specify 600.

  11. Click the Save button.

Upload a TSV file with tasks.

Prepare your own task file or use the demo TSV file.

To upload tasks:

  1. Click Upload.

  2. In the window that opens, configure the file upload settings:

    1. Choose Smart mixing.

    2. In the Main tasks field, specify 9.

    3. In the Training tasks field, specify 0.

    4. In the Control tasks field, specify 1.

    5. Click Upload.

    6. In the window that opens, choose the TSV file with tasks to upload and click Open.

    7. In the window that opens, check the number of tasks and click Add.

  3. Create a control task.

    1. Click Edit.

    2. In the window that opens, click Create control tasks.

    3. In the window that opens, in the column on the left, turn on the result option.

    4. Choose the correct answer to the question.

    5. Click Save and go to next.

    6. Click Are there shoes in the photo? to exit the task markup mode.

    Note.

    In small pools, control tasks should be 10-20% of all tasks. Include different versions of correct responses in equal amounts. See the distribution of responses on the Edit tasks page, Control tasks tab.

  4. Click to start the pool.

    Attention.

    The tasks will be completed by real performers in Toloka. Recheck your project setup before you start the pool.

Receive the results

To get the results:

  1. Next to the Download results button, click .

  2. Choose Dawid-Skene aggregation model. Read more about Result aggregation based on the Dawid-Skene model.

  3. In the window that opens, click Yes.

  4. At the top of the page, click View operations list.

    Note.

    To track the progress, refresh the page from time to time. Aggregation takes from 5 to 20 minutes. You can start designing another project meanwhile.

  5. When the operation is complete, download the TSV file with the results. To do this, click Download in the Files column.

  6. Use the TSV file with the results in project 2.