Pages in search results and the site structure
- What information is provided in the archive of loaded pages?
The archive of loaded pages is a file listing all the pages on the site that the robot has visited. It contains:
- URLs of pages known to the robot.
- The HTTP code returned to the robot during the last visit.
- The time of the robot's last visit in Unix-time format.
- Information about whether the page was shown in search results: 1 — yes, 0 — no.