Robots.txt analysis error manual

    Errors

    The list of errors that may be encountered while parsing robots.txt.

    Error Yandex extension? Description
    A rule begins with a character that is neither '/' nor a '*'. Yes

    A rule can begin with '/' or a '*' only.

    Several matching rules found for User-agent: *. No

    Only one rule of this type is allowed.

    Several Host directives discovered. Yes

    Only one Host directive is allowed.

    The robots.txt file is too long. Yes

    The number of rules in the file exceeds 1024.

    A rule is not preceded by User-agent directive. No

    A rule must always be preceded by the User-agent directive. It is possible that an empty line break follows the User-agent directive.

    A rule is too long. Yes

    The rule exceeds acceptable length (1024 characters).

    Incorrect name of the main mirror of the site. Yes

    The name of the main mirror of the site in the Host directive contains a syntax error.

    Badly formed URL of the Sitemap file. Yes

    URL of the Sitemap file must be specified as a complete URL, including http://. E.g., http://www.mysite.ru/sitemap.xml

    Wrong format of Crawl-delay directive. Yes

    Wrong time specified in the Crawl-delay directive.

    Several Crawl-delay directives have been discovered. Yes

    Only one Crawl-delay directive is allowed.

    Wrong format of Clean-param directive. Yes

    The Clean-param directive specifies one or more parameters that the robot will disregard, as well as the path prefix. The parameters are delimited with '&' and separated from the path prefix by a space.

    Warnings

    The list of warnings that may be generated while parsing robots.txt.

    Warning Yandex extension? Description
    An invalid character may have been used. Yes

    A special character different from "*" and "$" was discovered.

    Unknown directive was discovered. Yes

    A directive not described in the rules for using robots.txt was discovered. It is possible that this directive is used by robots that belong to other search engines.

    A syntax error. Yes

    A string cannot be interpreted as a robots.txt directive.

    Unknown error. Yes

    Analysis of robots.txt discovered an unknown error. Contact technical support.

    URL validation errors

    The list of URL validation errors that may be encountered while parsing robots.txt.

    Error Description
    A syntax error.

    URL syntax error.

    This URL does not belong to your domain

    The specified URL does not belong to the site for which the robots.txt file is analyzed. You may have specified a URL for one of your site mirrors or misspelled the domain name.