Audio transcription

You can publish tasks for transcribing short audio recordings. We recommend that all the recordings in a pool are the same length. It is best to launch transcription tasks in the Toloka web version so that performers can use the keyboard for typing.

Let's say you need to transcribe poems recited by children. To do this, create a task that provides an audio recording in the built-in player. The performer has to type the text they hear on the recording.

Example of a prepared task

To run tasks and get responses:

Create a project

The project defines what the task will look like for a performer.

  1. Click the + Create project button and choose the Audio transcription template.

  2. Enter a clear name and a short description for the project. Performers will see this in the task list.

  3. Write short and clear guidelines (see the recommendations).

  4. Define which objects you are going to pass to the performers and receive from them in response. To do this, add input and output fields in the Specifications block.
    What are input and output data?

    Input data is types of objects that are passed to the performer for completing the task. For example, this could be a text, an image, or geographic coordinates.

    Output data is types of objects that you receive after the task is completed. For example, this could be one of several response options, typed text, or an uploaded file.

    Learn more about input and output data fields.

    The template includes the fields:

    • Input data field — The audio link to an audio file.

      Change the data type to string to upload audio files stored on Yandex.Disk.

    • Output data field — The output string for saving the text entered by the performer.
  5. Create the task interface in the HTML block. It describes how the task elements should be arranged in the task.

    You can use standard HTML tags and special expressions in double curly brackets for input and output data fields.

    <div>
      <audio src={{proxy audio}} controls controlsList="nodownload">
     Unable to play
      </audio><br /><br />
    </div>
    <div>
      <div>The poem text</div>
      {{field type="textarea" name="output" width="300px" rows="6"}}
    </div>
    This notation describes the following task design:
    • The audio recording in the player.
    • Text input field.

    Clear the CSS block.

    Leave the JavaScript unchanged. It is configured to check the record playback in the player. The performer won't be able to send the response without listening to all audio recordings in the task.

  6. Click the Preview button to see the performer's view of the task.
    Note. The project preview shows one task with standard data. You can define the number of tasks to show on the page later.
  7. Save the changes. To switch to the Projects page, click Finish editing.

Add a task pool

A pool is a set of paid tasks sent out for completion at the same time.

  1. Open the project and click Add pool.
  2. Give the pool any convenient name and description. The pool info is only available to you. Performers can view only the project name and description.
  3. Set the price per task page (for instance, $0.05). The price depends on the length of the audio recordings.
    What is a task page?

    A page can contain one or several tasks. If the tasks are simple, you can add 10-20 tasks per page. Don't make pages too long because it slows down loading speed for performers.

    Performers get paid for completing the whole page.

    The number of tasks on the page is set when uploading tasks.

    What is the fair price for a task page?

    The general rule of pricing is the more time the performer spends to complete the task, the higher the price is.

    You can register in Toloka as a performer and find out how much other requesters pay for tasks.

  4. Add Filters to choose performers.
  5. Turn on the Non-automatic acceptance option and enter the number of days for checking the task in the Deadline field (for example, 7).
    What is offline accept?

    The review of assignments option allows you to review completed tasks before accepting them and paying for them. If the performer didn't follow instructions, you can reject the task. The maximum allowed period for the review is set in the Deadline field.

  6. Set the Overlap, which is the number of performers to complete the same task. For the speech transcription, it is 1, as a rule.
  7. Set the Time allowed for completing a task page. This time should be enough to read the instructions, load the task, listen to audio recordings, and type text. (for example, 1200 seconds).
  8. Save the pool.

Upload tasks

  1. Download the File example for task uploading (tsv) in the pool.
    What is TSV?
    A TSV file presents a table as a text file in which columns are separated by tabs.
    You can work with it both in a table editor and a text editor, and then save it to the desired format. More about working with a TSV file.
    Note. Before uploading the file, make sure it is saved in UTF-8 encoding.
  2. Add input data, like links to files on Yandex.Disk in the format <unique name>/image1.jpg, where "unique name" is the name of your proxy (learn more). The header of the input data column contains the word INPUT.

    A link should look like this: <unique name>/audio1.jpg. The unique name is the name of your proxy. Learn more about using files from Yandex.Disk.

  3. Upload the tasks: choose Set manually and set the number of tasks (for example, 4 tasks per page). This means that there will be 4 audio recordings per page, each recording with a text field for transcription.
  4. Click Add to upload your tasks to the pool.

Set up quality control

Quality control rules allow you to filter out inattentive performers. You can configure quality control both in the project and in the pool.

Attention.

Quality control settings are applied to all project pools, so you can't change them in just one of the pools.

  1. Pool

    Go to pool editing (the Edit button in the upper-right corner of the page) and click Add Quality Control Rule.

    You can copy quality control settings from another pool. To do this, click Copy settings from in the Users filter section.

    Project

    Open the project page, open the Quality control tab and click Set quality control. Then click + Add Quality Control Rule.

    The rules are applied to all project pools, so you can't change settings in just one of the pools.

    Restriction.

    When you clone a project, its quality control settings aren't transferred.

  2. Add a restriction for fast responses and specify the following values:

    This means that a user who completes a task page in less than 20 seconds will be blocked for ten days and won't be able to complete your tasks.

  3. Add the Review results quality control rule and enter the following values:

    This means that if 35% or more of a performer's responses are rejected, the performer is banned and can't access your tasks for 15 days. The rule takes effect after 3 responses of the performer are reviewed.

  4. Create a skill. To do this, go to the Skills page, click the +Add skill button and enter the skill name, for example, "Transcriber".
    What is a skill?
    A skill is an assessment of some aspect of the performer's work (a number from 0 to 100). A skill can be awarded to the performer for correct responses in control tasks. It can be appointed arbitrarily as well.

    You can use the skill value when choosing performers.

  5. Add the Submitted answers section and enter the following values:

    This means that the skill is appointed to the performer if they completed at least one task and the result was accepted.

Start the pool and get the results

  1. Start the pool by clicking .
  2. Track the completion of tasks in the Pool statistics section.
  3. When the first results are received, you can start the review . After the specified time period, all responses are automatically accepted, regardless of their quality.

    To review assignments, go to the pool and click Review assignments.

Let performers check the responses

Send the results to performers for the review as tasks. To make these tasks available to performers who didn't transcribe audio recordings, set the filter.

  1. Go to the pool and click Download results.
  2. Create a project with the classification type.
    Example of a prepared task
  3. Create a task interface that shows:
    • An audio recording in the audio player.
    • A transcript.
    • Radio buttons with answer options.
      • The text fully matches the audio recording.
      • Minor mistakes were made in the text.
      • The audio recording is not transcribed fully.
      • The text doesn't match the audio recording.
  4. Add a pool and set Overlap to 3 in it.
  5. Add a filter to choose performers without skill:
  6. Upload tasks to the pool and start it.
  7. When the pool is fully completed, start aggregation of results.
  8. Accept transcription tasks without errors. Reject the rest, specifying the reason.
  9. Rejected tasks can be submitted for completion again.