A sitemap index is an XML file that lists other sitemap files. It helps you find page URLs spread across several sitemaps, but it does not list those page URLs directly.

Large websites often split their URLs across many sitemap files. The sitemap index acts like a table of contents.

Example:

<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/post-sitemap.xml</loc>
  </sitemap>
  <sitemap>
    <loc>https://example.com/product-sitemap.xml</loc>
  </sitemap>
</sitemapindex>

Sitemap vs sitemap index

A sitemap lists page URLs.

A sitemap index lists sitemap files.

To recognize an index, look for a <sitemapindex> root element and <sitemap><loc> entries that point to sitemap files. A regular sitemap instead uses <urlset> and <url><loc> entries for page URLs. A scraper that follows the index can collect URLs from its linked sitemap files too. The Sitemaps protocol defines both formats.

📌 Short version

If a website publishes a sitemap index, start there to discover its linked sitemap files before extracting page URLs.

Where to find sitemap indexes

First open https://example.com/robots.txt and look for a Sitemap: line. If there is no listed location, try these common paths:

  • https://example.com/sitemap.xml
  • https://example.com/sitemap_index.xml

The filename alone does not tell you which format the file uses. Open it and check its root element. If neither path exists, the site may use another location or may not publish a sitemap index.

Datablist sitemap index workflow

The Sitemap Scraper can follow sitemap indexes and import URLs from the linked sitemap files.

After import, use regex filters to keep sections such as /blog/, /products/, /docs/, or /case-studies/. Then run metadata extraction, AI web scraping, or export the URL list to CSV.