Restricting site indexing

Site indexing may be restricted for technical reasons (for example, pages contain errors, and their content cannot be indexed correctly, or the site indexing is forbidden using robots.txt file), or for using dishonest methods of site promotion in search engines (search spam).