Project 1. Does the image contain a specific object?

In this project, you ask the performers if a photo contains traffic signs.

  1. Create a project
  2. Create a pool
  3. Upload a TSV file with tasks.
  4. Get the results

Create a project

Choose a template
  1. Click the + Create project button.

  2. In the window that opens, find the Image classification template and click Select.

Provide general information
  1. In the Name to show performers field, enter Are there traffic signs in the picture?.

  2. In the Description for performers field, enter Look at the picture and answer whether or not it shows any traffic signs.

  3. Click Save.
Edit the task interface
Note. This tutorial shows how to create a task interface in the HTML/JS/CSS editor. You can also try creating a task interface in the Template Builder.

In the Task interface section, configure the display of controls in the html block:

  1. Delete the row responsible for the like field:

    {{field type="checkbox" name="like" label="I like the photo" hotkey="q"}}
  2. After the line with the image, enter the question:

    <div>Are there  <b>traffic signs</b> in the picture?<div>
  3. Change the labels on the response options: GoodYes, BadNo:

    {{img src=image width="100%" height="400px"}}
    <div>Are there <b>traffic signs</b> in the picture?<div>
    
    {{field type= "radio" name= "result" value= " OK "label=" Yes "hotkey=" 1"}}
    {{field type= "radio" name="result" value= " BAD "label=" No "hotkey=" 2"}}
    {{field type= "radio" name="result" value= " 404 "label=" Loading error" hotkey="3"}}
  4. Leave the JS and CSS blocks unchanged.

  5. Set up Specifications. Delete the like field:

    1. Point the cursor at the field and click the button.

    2. Click Delete.

    Learn more about Specifications in Input and output data.

  6. Click Save.

In the upper-right corner of the tab, click for preview.

Tip.

If the task isn't displayed in the preview mode, disable extensions and ad blockers in the browser.

In the window that opens, check if the task options work correctly:

  1. Answer the question.

  2. In the lower-right corner, click Submit.

Exit preview mode:

  1. In the lower-left corner, click Exit.

  2. In the list that opens, click Exit.

Write instructions for performers

In the Instructions for performers field, enter the instructions and add an image. To add an image, click and provide a link to the image you want to use as an example.

Instructions:

Look at the image and answer whether there are any traffic signs in it.
If there are, click Yes.
If there aren't, click No.
For example, there are traffic signs in the image, so the correct answer is Yes.

Click Save.

In the upper-right corner, click Finish

Learn more about working with the project in Project.

Create a pool

  1. Open the page of the project titled Are there traffic signs on the picture?.

  2. Click the Add a pool button.

  3. Specify the Pool name.

  4. (optional) Enter a private description.

    1. Turn on the Add a private description option.

    2. In the Private description field, enter the pool description. This information is available only to you.

  5. In the Price per task page section, the Price in US dollars field, specify the price. For example, 0.01. You can see more examples of pricing here.

  6. In the Performers section, filter performers by the language:

    1. Click Add filter.

    2. Find the Audience presets block in the list and choose the Russian speaking users skill.

  7. In the Quality control section, specify quality control settings for the pool:

    1. Click Add Quality Control Rule.

    2. Find the Rules block in the list and choose Control tasks.

    3. Set a rule for control task: if the number of responses to the control questions is ≥ 3 and correct responses (%) to the control questions is < 60, then ban the performer on project for 10 days. Specify the Control task as a reason.

      Learn more in Quality control.

  8. In the Overlap section, specify 3 in the Overlap field.

  9. (optional) In the Speed/quality ratio section, specify the percentage of top-rated users who will get access to the pool. Learn more in the Speed/quality balance section.

  10. In the Parameters section, specify 600 in the Time for completing a task page in seconds field.

  11. Click Save.

Upload a TSV file with tasks.

Prepare your own task file or use the demo TSV file. You can also find it on the pool page. At the top left of the page, there are links to the TSV file with regular, control, and training tasks.

  1. On the pool page, click Upload. In the window that opens, you can also download a sample TSV file by clicking Sample file for uploading tasks.

  2. In the window that opens, configure the file upload settings:

    1. Choose Smart mixing.

    2. In the Main tasks field, specify 9.

    3. In the Training tasks field, specify 0.

    4. In the Control tasks field, specify 1.

    5. Click Upload.

    6. In the window that opens, choose the TSV file with tasks to upload and click Open.

    7. In the window that opens, check the number of tasks and click Add.

  3. Create a control task.

    1. Click Edit.

      Note. If you selected something else instead of smart mixing, click Edit. If this button is missing, delete the file and upload it again.
    2. In the window that opens, click Create control tasks.

    3. In the window that opens, turn on the result option in the Create control task section.

    4. Choose the correct answer to the question.

    5. Click Save and go to next.

    6. Click Are there traffic signs in the photo? to exit the task markup mode.

    Note.

    In small pools, control tasks should be 10-20% of all tasks. Include different versions of correct responses in equal amounts. View the distribution of responses on the Edit tasks page, Control tasks tab.

  4. Click to start the pool.

    Attention.

    The tasks will be completed by real performers in Toloka. Recheck your project setup before you start the pool.

Get the results

  1. Next to the Download results button, click .

  2. Choose Dawid-Skene aggregation model from the list. Read more about Result aggregation based on the Dawid-Skene model.

  3. In the window that opens, click Yes.

  4. At the top of the page, click View the list of operations.

    Note.

    To track the progress, refresh the page from time to time. Aggregation takes from 5 to 20 minutes. You can start designing another project meanwhile.

  5. When the operation is complete, download the TSV file with the results. To do this, click Download in the Files column.

  6. Use the TSV file with the results in project 2.

What's next