How Yandex indexes sites
A site on search results page

Site structure

Shows sections of the site that make up more than 1% of the total number of pages loaded to the search engine database. Sections are displayed in a tree hierarchy according to the structure of the site's URLs. Information is available for each site section on the number of pages that were visited by the Yandex robot and that participate in Search.

The data presented can help you determine if the robot is visiting the site sections that you believe are of interest to your users. To disallow visiting pages from particular sections of the site, use the robots.txt file.

Adding a section

You can get indexing information about all the site sections that interest you, even if they don't make up 1% of the total pages loaded to the search engine database. To get this data, click Add section and enter the desired section, beginning with "/". For example, "/catalog1".

You can use the * operator to specify a non-exact match to the name of the site section:

catalog1*Adds all sections with URLs beginning with "catalog1"
*catalog1Adds all sections with URLs ending with "catalog1"
*catalog1*Adds all sections with URLs containing "catalog1"

Changes will take effect after a few days.

Questions and answers

What does “visited” mean?

The number of pages on your site that the Yandex robot accessed. They may include indexed pages as well as pages that returned an error.

What does “pages available on search” mean?

The pages that participate in Yandex Search. A page might not be used in search results if it duplicates the content of another page, contains spam or viruses, redirects to another URL, or an error occurred when loading it.

To find out why a page is missing in Yandex Search, we recommend using the URL testing tool.

Why aren't all the pages in a section indexed?

Most likely, not enough time has passed since the robot found out about the site. We recommend updating your Sitemap and waiting until the robot indexes it.

How can I stop the Yandex robot from visiting pages in a particular section?

To prohibit indexing a site section, enter the following in the robots.txt file:

User-agent: Yandex
Disallow: /section/

At the next site indexing, the robot will take this into account and remove the section. Exceptions for the robots.txt file are described in Exceptions.

Rate this article
Thank you for your feedback!