How Yandex indexes sites
A site on search results page


Information provided on this page allows you to:

  • Get information about the date and time when the site was last accessed by the robot.
  • Track the quantity and quality of visited pages and pages in search.
  • Check pages that were excluded from the index and get information about the possible reason for this.

The service not only allows you to get the number of site pages that are in the Yandex robot's database, but also to track trends as it changes over time.

Note. Data is available starting from September 1, 2015.

Selecting a site section

You can specify which section of the site interests you at this time. The sections are listed according to the site structure that Yandex is aware of.

Selecting page categories

By default, information is available for all site pages that are known to the robot — Summary.

The service allows you to quickly switch to information for a specific category of site pages (for example, to view pages in search).

Viewing area

You can use the icon to expand the page content to the width of the browser window.

Displaying data on a diagram

The diagram shows changes in the number of pages over time. You can point the cursor at a particular section to see data on the number of certain pages on a specific day.

By default, the diagram shows values for all categories of pages that are available in this section. To remove the diagram so it isn't displayed, click .

Pages visited

Clicking the link gives you information about site pages that the Yandex robot has accessed. This might be indexed pages or pages that returned an error.

You can use this data to find out which pages were successfully loaded by the robot, or to determine why loading failed.

Excluded pages

Clicking the link gives you information about possible reasons for excluding pages from the index:

  • Blocked or non-existent — Check whether the robots.txt file is set up correctly, and whether the page has the noindex tag in its HTML code.
  • Server error — When the page was loaded or processed by the robot, the server response contained a 3XX, 4XX, or 5XX HTTP code.
  • Not supported — The page is not canonical or redirects the user to other pages.

Pages available on search

Clicking the link gives you information about pages that can participate in Yandex Search. A page might not be shown in search results if it duplicates the content of another page, contains spam or viruses, redirects to a different URL, or an error occurs when downloading it.

To find out why a page is missing in Yandex Search, we recommend using the URL testing tool.

Changing the time period on the diagram

By default, the diagram displays data for the current month. You can move the slider to change the zoom on the diagram area.

Downloading data in an archive

To download information about visited pages and pages excluded from the index, you can export data in TSV format. Besides the information available in the service's interface (the page URL, the HTTP code, and the date and time of the robot's last visit), the file provides information about whether the page was in search results: 1 — yes; 0 — no.

Rate this article
Thank you for your feedback!