Sitemap for localized pages

Use the Sitemap file in the XML format to inform Yandex robots about the language versions of pages that need to be indexed on your site. Check out the requirements to the Sitemap file.

Use ISO codes to specify the language and region:

For example, the site may have an English-language page for English-speaking users from all over the world, with two options for German-speaking users: for visitors from Switzerland and from other countries:

  • www.example.com/english/page.html — For English-speaking audience.
  • www.example.com/deutsch/page.html — For those who speak German.
  • www.example.com/schweiz-deutsch/page.html — For users from Switzerland who speak German.
Example of a Sitemap entry
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <url>
    <loc>http://www.example.com/english/page.html</loc>
    <xhtml:link 
               rel="alternate"
               hreflang="de"
               href="http://www.example.com/deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="de-ch"
               href="http://www.example.com/schweiz-deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="en"
               href="http://www.example.com/english/page.html"/>
  </url>
  <url>
    <loc>http://www.example.com/deutsch/page.html</loc>
    <xhtml:link 
               rel="alternate"
               hreflang="de"
               href="http://www.example.com/deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="de-ch"
               href="http://www.example.com/schweiz-deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="en"
               href="http://www.example.com/english/page.html"/>
  </url>
  <url>
    <loc>http://www.example.com/schweiz-deutsch/page.html</loc>
    <xhtml:link 
               rel="alternate"
               hreflang="de"
               href="http://www.example.com/deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="de-ch"
               href="http://www.example.com/schweiz-deutsch/page.html"/>
    <xhtml:link 
               rel="alternate"
               hreflang="en"
               href="http://www.example.com/english/page.html"/>
  </url>
</urlset>

Tag description

Tag Required Description
url Yes Parent tag for each site page
loc Yes Page URL
xhtml:link Yes Links to all language and regional versions of the page, including the main one

FAQ

How do I set up a Sitemap for a large site?

The Sitemap usually includes all significant site pages. If the site has many pages, you can delete the pages known to Yandex from Sitemap and add only new ones or frequently updated ones to the file.

If you use the Sitemap index file mark the frequently updated Sitemap files with the lastmod tag.

To find out which site pages are already known to Yandex, go to the Indexing → Crawl statistics page in Yandex.Webmaster.

Sitemap is displayed in the service as an excluded page with the “Invalid document format” status.

The Sitemap file (and other XML files) are displayed as excluded pages. This is, however, only for information purposes and doesn't affect the site indexing or the Sitemap processing.

The Sitemap file can be displayed in a group of excluded pages if the robot tried to index it as a regular page. Despite this, the Sitemap files don't participate in the search.

The “Unknown tag” error occurred while processing Sitemap

The Sitemap file may contain only certain XML elements. If Yandex.Webmaster finds other elements in the file (for example, a reference to the mobile version), it displays the "Unknown tag" error. The robot ignores unsupported elements when processing Sitemap and takes into account the data from the supported elements. Therefore, you don't need to change the Sitemap file.

If the file content is changed, the information in Yandex.Webmaster is updated within two weeks.

The Sitemap file has the “Not indexed” status.

The Sitemap file may be not indexed for several reasons:

  • The robot crawled the Sitemap file recently and didn't process it yet. Wait for two weeks. If you use the Sitemap index file with multiple files, they may take longer to process than a single Sitemap file.
  • The site was previously unavailable for the indexing robot. You need to wait for the next site visit by the robot.
  • Access to Sitemap is prohibited in the robots.txt file. The server returns an HTTP code other than 200 when the robot tries accessing Sitemap. Check if the file is available to the robot. To do this, use the Server response check tool. Specify the full path to the file.

If there is an access problem, contact the administrator of the site or the server where it is located. If the file is compiled correctly, it will be re-processed within two weeks from the last request by the robot. You can check the file with the Sitemap file analysis tool.

The Sitemap file is in the “Document contains no text” status

This status is displayed in the Server response check tool and notifies that the robot can't index the file as a regular site page and include its contents in search results. This status doesn't affect the Sitemap processing. To check that a Sitemap is correct, use the Sitemap validator tool in Yandex.Webmaster.

The Sitemap file has the status “URL has been excluded from indexing with the "noindex" tag”

This status is displayed in the Server response check tool and reports that the file is closed for indexing — as a rule, the prohibition is written in the HTTP header X-Robots-Tag. The directive prohibits a file from appearing in search results, but doesn't affect the processing of the Sitemap content. To check the Sitemap for correctness, use the Sitemap validator tool in Yandex.Webmaster.

Sitemap isn't processed.

Usually, the Sitemap file is processed by the robot within two weeks after adding it to Yandex.Webmaster. If this doesn't happen, check:

  • That the Sitemap belongs to the site for which you added it in Yandex.Webmaster. Make sure that the site address matches, including the protocol (HTTP/HTTPS) and the www prefix (if it is included).



  • The server response to the Sitemap request. If access to Sitemap is prohibited in the robots.txt file or an HTTP code other than 200 is returned when the robot tries accessing it, the file will not be processed by the robot. If the check displays “Forbidden by the noindex meta tag”, this doesn't indicate an error. The noindex meta tag prevents the file from being included in the search database, but doesn't prevent the robot from processing its contents.
  • The Sitemap file with the Sitemap file analysis tool. Correct the mistakes if they are found.