What a sitemap does
An XML sitemap lists the URLs you want search engines to crawl. It does not force anything into the index, but it helps crawlers find new and deep pages faster, and an accurate <lastmod> tells them which pages changed and are worth recrawling.
What this checker looks at
- Discovery. Is the sitemap listed in robots.txt with a
Sitemap:line? That is how every crawler finds it, not just the ones you submit to by hand. - Structure. The root must be
<urlset>(a list of pages) or<sitemapindex>(a list of sitemaps), with the sitemaps.org namespace, and the file must be complete XML. - Limits. One sitemap can hold 50,000 URLs and 50 MB uncompressed. Bigger sites split into several files under an index.
- lastmod coverage. Google uses
<lastmod>when it is consistently accurate. Set it from the real content change date, not the build time. - Hosts. URLs should be on the sitemap's own host.
- Gzip. Compressed
.xml.gzsitemaps are decompressed and checked like any other.
Common sitemap problems
- HTML instead of XML. A catch-all route answers
/sitemap.xmlwith your homepage and a 200 status. - Listing URLs you do not want indexed: redirects, 404s, noindex pages or non-canonical duplicates. Every URL in a sitemap should return 200 and be the canonical version.
- A stale sitemap that was generated once and never updated.
- Truncated files from a generator that timed out halfway.
Next steps
Pull every URL into a list or CSV with the sitemap URL extractor. No sitemap yet? Build one with the sitemap generator, then add it to robots.txt and check the file with the robots.txt tester.