How Yandex indexes sites
A site on search results page

Using HTML tags


Metatags allow the webmasters to specify data that describe a site page. Certain metatags are used by the search engine robots as well. Yandex robot takes into account the content of the following tags:

  • <meta name="Keywords" content="..."/> — may be taken into account when the robot determines if a page matches a search query;

  • <meta name="Description" content="..."/> — contents of this tag may be used in snippets (site descriptions that appear on the search result page);

  • <meta http-equiv="Content-Type" content="type; charset=..."/> — may be taken into account when the robot determines the type and the encoding of a document;

  • <meta http-equiv="refresh" content="N;url=_адрес цели перенаправления_"/> — redirects user to the page with the specified address after N seconds of the user staying on this page. At small values of N, this is considered a temporary redirect equivalent to server redirects with HTTP code of 302/303/307.

Yandex robot also takes into account the directives from the following tags:

  • <meta name="robots" content="all"/> — the text and links on the page may be indexed, similar to <meta name="robots" content="index, follow"/>

  • <meta name="robots" content="noindex"/> — the text on the page should not be indexed,

  • <meta name="robots" content="nofollow"/> — the robot should not follow the links on the page or index them,

  • <meta name="robots" content="none"/> — neither the text nor the links on the page may be indexed, similar to <meta name="robots" content="noindex, nofollow"/>

  • <meta name="robots" content="noarchive"/> — the search result page must not display a link to a cached copy.

The robot will not know about the documents if they are linked to from the page containing a metatag with nofollow attribute. However, they may be indexed if some other page links to them without the nofollow specified.

If the metatags are omitted, by default they have positive values (index, follow, archive). If there are conflicting metatags, the robot will select the default value.


<meta name="robots" content="all"/>

<meta name="robots" content="noindex, follow"/>

The robot will select "all"; text and links will be indexed.

The all tag value does not apply to noarchive.


<meta name="robots" content="all"/>

<meta name="robots" content="noarchive"/>

Text and links will be indexed but the search result page will not display a link to a cached copy.

<noindex> tag

You can use the <noindex> tag to prohibit indexing of some auxiliary text fragments. This tag is similar to the noindex metatag but it only applies to content enclosed in the tag, using the following syntax:

<noindex>this text will not be indexed</noindex>

The noindex tag is not affected by nesting level (i.e. it can reside anywhere inside the HTML code of the page). If you want the source code of the site to be valid, you can use the following syntax of the tag:

<!--noindex-->this text will not be indexed<!--/noindex-->

Attribute rel="nofollow" of the <a> tag

The rel="nofollow" attribute is used in the <a> tag with the following syntax:

<a href="url" rel="nofollow">link text</a>

This tag is similar to the nofollow metatag but it only applies to the link for which it is specified.

Attribute rel="canonical" of the <link> tag

If your site contains groups of webpages that are similar in content, webmaster can select the preferred (canonical) URL to appear in Yandex search results. For example, the following webpage is located at two URL addresses:


If the preferred url — /blog, is located at a non-canonical url, you just need to specify the following:

<link rel="canonical" href="http://www.examplesite.ru/blog"/>
  • A canonical url should be available for indexing, so the Yandex robot takes the rel="canonical";

  • The Yandex indexing robot will not include an attribute if the URL specified is from another host or host subdomain.

  • A webpage cannot have more than one canonical URL.

  • We do not recommend creating chains of more than two elements (source, target), where each element indicates that the canonical URL is the next in the chain.