An XML sitemap lists the URLs of a site in a structured file intended for search engines. For a user who wants to quickly explore all the articles and resources of a domain, this file serves as a more direct entry point than traditional navigation. The question arises: among the different methods of locating a sitemap, which one provides the most reliable result, and above all, what can we actually derive from it once the file is found?
XML Sitemap, robots.txt, and Internal Navigation: What Each Method Reveals
| Method | Access Required | Type of URLs Found | Main Limitation |
|---|---|---|---|
| Adding /sitemap.xml to the domain URL | None (public access) | Pages declared by the site editor | The file may not exist or may be renamed |
| Reading the robots.txt file | None (public access) | Address(es) of the declared sitemap(s) | Some sites do not declare their sitemap in robots.txt |
| Google Search Console | Owner or delegated access | Submitted sitemaps + indexed pages | Restricted to site administrators |
| Internal navigation (menu, links) | None | Pages linked from other pages | Orphan pages remain invisible |
| SEO Crawler (Screaming Frog, etc.) | None (third-party tool) | All pages accessible by following links | Does not detect pages absent from the internal linking |
The sitemap and the SEO crawler do not cover the same scopes. The former reflects a declaration of intent from the editor. The latter maps what is actually linked. The gap between the two often reveals orphan pages, missing from navigation but present in the sitemap.
To illustrate this approach, one can access Michelle Dastier’s sitemap and see that the file lists all the articles published on the domain, sorted by modification date.
Orphan Pages in the Sitemap: A Gap Most Guides Ignore
A page listed in an XML sitemap is not necessarily accessible from the site’s menu or internal links. The sitemap does not guarantee either the indexing or the visibility of a page. Google treats it as a discovery signal, not as proof of relevance.
SEO audit analyses published in 2026 show that pages perfectly present in the sitemap, returning an HTTP 200 code, received no internal links. These orphan pages are heavily penalized in terms of crawl and ranking.
This finding has a direct consequence for those using a sitemap as an exploration tool: some URLs in the sitemap lead to content that cannot be found through traditional navigation. This is both an advantage (hidden resources are discovered) and a warning signal (these pages lack context for search engines).
How to Identify Orphan Pages in a Sitemap
- Open the site’s XML sitemap and export the complete list of URLs (a sitemap index points to several sub-sitemaps, each grouping a type of content: articles, pages, images).
- Run a crawl of the same domain with a tool like Screaming Frog starting only from the homepage, without injecting the sitemap as a source.
- Compare the two lists: URLs present in the sitemap but absent from the crawl are orphan pages, lacking internal linking.
Locating a Site’s Sitemap When the Standard URL Does Not Work
The majority of sites use the path /sitemap.xml. When this path returns a 404 error, several alternatives exist, and their effectiveness varies depending on the CMS used.
The robots.txt file remains the most reliable option after the standard URL. Simply add /robots.txt to the domain name. A line starting with “Sitemap:” indicates the exact address of the file. Some sites declare multiple sitemaps in this file.
Alternative Paths by CMS
WordPress often generates a sitemap at /sitemap_index.xml when extensions like Yoast SEO or Rank Math are installed. Prestashop frequently uses /1_index_sitemap.xml. Shopify places its sitemap at the root under /sitemap.xml, but the sub-sitemaps follow a specific naming convention by content type (products, collections, pages).
If none of these paths work, a Google search using the operator site:domainname.com filetype:xml may bring up the file in the results. However, this method depends on the sitemap itself being indexed by Google, which is not always the case.
Reading an XML Sitemap: Usable Information to Find Resources
An XML sitemap contains at least the <loc> tag with the URL of each page. Depending on the configuration, it may also include the last modification date (<lastmod>), the estimated update frequency (<changefreq>), and a relative priority (<priority>).
The lastmod tag is the most useful for identifying recent content. By sorting the URLs by descending modification date, one can directly access the latest articles published or updated, without navigating through the site’s structure.
On the other hand, the changefreq and priority tags are largely ignored by Google. They reflect the editor’s wishes, not the actual behavior of the search engine. Their presence in the file has no predictive value regarding crawl frequency.
- An index sitemap (<sitemapindex>) groups several sub-sitemaps, each limited to a defined number of URLs. For a large site, browsing the index allows targeting a specific type of content (blog articles, product sheets, category pages).
- Image and video sitemaps exist alongside the main sitemap. They list media resources with their metadata (caption, title, duration for videos).
- A Google News sitemap, reserved for eligible sites, contains only articles published in the last 48 hours.
The sitemap remains a tool for reading the declared structure of a site, not a faithful mirror of its accessible content. Cross-referencing the sitemap with an internal crawl provides the most complete view of all available resources. URLs found in the sitemap but absent from the internal linking deserve special attention: they represent either deliberately discreet content or a structural flaw that the editor has not corrected.
