Build an XML sitemap that only lists the right pages
A clean sitemap lists every page you want found: live, final and indexable. This guide covers creating one, keeping it accurate and fixing orphan pages.
The problem
There is no XML sitemap, or the sitemap lists URLs that redirect, return errors or are excluded with noindex, or important pages are missing from it.
How to spot it
/sitemap.xmlreturns 404 and robots.txt has noSitemap:line.- The audit lists "Sitemap URLs that are not indexable", "Indexable pages missing from the sitemap" or "Orphan pages".
Why it matters
Search engines use sitemaps to discover pages and to decide what to crawl. A sitemap full of redirects and errors teaches them that your sitemap is unreliable, while pages missing from it may be found late or not at all.
What happens if you leave it
New or deeply nested pages take longer to appear in search, and crawl effort is wasted on URLs that should not be there.
How to fix it
-
Generate the sitemap automatically. Use your CMS or a plugin so it updates when pages change. Avoid hand-maintained files.
-
Include only canonical, indexable URLs that return HTTP 200: no redirects, no
noindexpages, no parameter or filtered duplicates. -
Use absolute URLs on the same host and protocol as the live site, for example
https://www.example.com/services/. -
Keep each sitemap under 50,000 URLs and 50 MB uncompressed; use a sitemap index to combine several files.
-
Reference it from robots.txt:
Sitemap: https://www.example.com/sitemap.xml -
Fix orphan pages: for each page listed in the sitemap that no other page links to, either add internal links (from navigation, a hub page or related content) or, if the page is obsolete, remove it from the sitemap and redirect or retire it.
-
Submit the sitemap in Google Search Console and Bing Webmaster Tools.
Platform-specific notes
- WordPress: core provides
/wp-sitemap.xml; SEO plugins provide their own. Use one, not both. - Shopify / Wix / Squarespace: a sitemap is generated at
/sitemap.xmlautomatically. Fix errors at the source (unpublish or redirect pages).
Common mistakes
- Listing both
http://andhttps://, orwwwand non-wwwversions. - Leaving staging, thank-you or search-result pages in the sitemap.
- Excluding pages from the sitemap and assuming that removes them from search. It does not.
How to check the fix worked
- Open the sitemap in a browser and spot-check URLs: each should load directly without redirecting.
- In Search Console, the sitemap report should show it as "Success".
- Re-run the audit; sitemap errors and orphan pages should drop.
References and further reading
Related tutorials
Create and check your robots.txt file
robots.txt tells crawlers which parts of your site they may fetch and where your sitemap is. Here is how to pu...
Fix broken links and pages that return errors
Find and repair links to missing pages, restore or redirect dead URLs, and deal with server errors and unreach...
Remove redirect chains and update links to redirected URLs
Redirect old URLs straight to their final destination in one hop, and point your own links at final URLs.