Sitemap
What is a Sitemap? XML file to communicate website structure to search engine crawlers.
Sitemap is a structured file (usually in XML format) that tells search engines all the URLs of a website so they can crawl and index them. A sitemap is not for users, but for search engine crawlers. It functions like a map: "Here are all the important places on my website, please visit these." In technical SEO, a well-maintained sitemap is essential for large websites and signals to Google which URLs are important.
Most small to medium websites don't strictly need a sitemap - Google finds URLs through internal links. But for complex websites, large catalogs (e-commerce), new websites, or websites with many JavaScript-based pages, a sitemap is critical.
What is a Sitemap?
A sitemap is an XML file, typically at /sitemap.xml, that lists all URLs of a website with additional metadata:
Example sitemap structure:
"<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <url> <loc>https://leadanic.com/</loc> <lastmod>2026-03-04</lastmod> <changefreq>weekly</changefreq> <priority>1.0</priority> </url> <url> <loc>https://leadanic.com/glossary/marketing-automation/</loc> <lastmod>2026-02-28</lastmod> <changefreq>monthly</changefreq> <priority>0.8</priority> </url> </urlset>"
Sitemap attributes:
- loc: The complete URL of the page
- lastmod: When was this page last updated? (Optional, but recommended)
- changefreq: How often does this page change? ("always", "hourly", "daily", "weekly", "monthly", "yearly", "never")
- priority: Relative priority (0.0 to 1.0). Default is 0.5
Types of sitemaps:
- XML sitemap: For search engines, machine-readable
- HTML sitemap: For users, a page index as HTML
- Video sitemap: Specialized sitemap for videos
- News sitemap: Specialized sitemap for news sites
- Image sitemap: Specialized sitemap for images
Most focus is on XML sitemaps.
Sitemap in a B2B Context
B2B websites need special sitemap considerations:
Dynamic URLs and parameters
Many B2B pages generate URLs dynamically (e.g., filters in product catalogs):
/products?category=marketing-automation&budget=10k-50k&industry=technology
Sitemaps should not contain every URL variant (would become too large), but rather the canonical, most important version:
/products/marketing-automation
Gated content and authentication
URLs behind login (e.g., "/my-account/", "/dashboard/") should typically not be in the sitemap, as Google cannot crawl these.
Large catalogs (> 50,000 URLs)
Google accepts sitemaps up to 50,000 URLs and 50 MB. For larger catalogs, you need a sitemap index:
"<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"> <sitemap> <loc>https://leadanic.com/sitemap-products-1.xml</loc> </sitemap> <sitemap> <loc>https://leadanic.com/sitemap-products-2.xml</loc> </sitemap> </sitemapindex>"
This creates multiple sitemaps (e.g., "sitemap-products-1.xml", "sitemap-products-2.xml") and an index.
Creating a sitemap
Option 1: Automatically via CMS
Most modern CMS platforms (WordPress, Webflow, HubSpot) generate sitemaps automatically:
- WordPress: The Yoast SEO plugin or a similar one generates it automatically
- Webflow: Automatically generated on each publish
- HubSpot: CMS generates automatically
This is often the best option because the sitemap is updated whenever new pages are added.
Option 2: XML sitemap generator (online tools)
Tools like:
- XML-Sitemaps.com (free, up to 500 URLs)
- Screaming Frog SEO Spider (Paid, unlimited)
- Sitemap Generator Tools
These crawl the website and generate a sitemap XML file.
Option 3: Manual or programmatic
For larger catalogs, generate the sitemap programmatically from a database:
- Database query: SELECT url, lastupdate FROM pages
- Generate XML file based on data
- Upload file to /sitemap.xml
- Reference in robots.txt
Sitemap best practices
1. Reference in robots.txt
Your /robots.txt should reference the sitemap:
"Sitemap: https://leadanic.com/sitemap.xml"
This tells Google where the sitemap is.
2. Include only crawlable URLs
Don't include URLs that are behind a login, return a 404 or get in the way of users:
- URLs that are not publicly accessible
- Pages with noindex tag
- Duplicate/canonical URLs (only the canonical)
3. Update lastmod
The lastmod date should be updated when a page actually changes. Google uses it to decide how often to re-crawl:
- Page was updated: update lastmod
- Only typo fixes: don't update lastmod (not necessary)
4. Use changefreq thoughtfully
changefreq is a hint, not a guarantee. Google often ignores it and uses historical data instead.
- Homepage: "weekly" or "daily" (frequent updates)
- Blog articles: "monthly" or "yearly" (not often updated after publishing)
- Old archives: "never" (not updated)
5. Set priority realistically
Priority tells Google which pages are relatively more important:
- Homepage: 1.0 (most important)
- Top-level service pages: 0.8 - 0.9
- Blog articles: 0.6 - 0.7
- Archive/old content: 0.3 - 0.5
Don't set all pages to 1.0 - that doesn't mean anything.
6. Test sitemap in Google Search Console
After adding a new sitemap:
- Go to Google Search Console
- Go to "Sitemaps"
- Enter your sitemap URL
- Google validates and shows status
Search Console will show:
- How many URLs in the sitemap
- How many of them are indexed
- Any errors
Sitemap errors and problems
URLs in sitemap but not indexed
If Google sees the sitemap but does not index the pages, possible reasons are:
- robots.txt blocks the URL
- A noindex tag on the page
- The page is canonicalized to another URL
- Very slow page speed
- The page has few backlinks and little authority
Sitemap too large (> 50 MB or > 50,000 URLs)
Google does not accept more than that. Solution:
- Create several smaller sitemaps (sitemap index)
- Remove unimportant URLs (e.g., archives)
- Consolidate paginated URLs (first page only)
Outdated sitemap
A sitemap that is not kept up to date becomes an SEO liability:
- Dead links in the sitemap
- New pages are missing
- Inaccurate lastmod
Solution: automate it (CMS or programmatic) instead of maintaining it by hand.
Sitemap Size and Limits
| Limit | Value | What to do when exceeded |
|---|---|---|
| Max URLs per sitemap | 50,000 | Sitemap index + several sitemaps |
| Max file size | 50 MB | Sitemap index + several sitemaps |
| Max sitemaps in an index | No hard limit, but 1,000+ is not ideal | Consolidate pages, remove old archives |
Sitemap vs. Internal Linking
A common question: "Do I need a sitemap if I have good internal links?"
Answer: For small websites (< 1,000 URLs) with good internal linking, a sitemap is optional. For larger or more complex websites, a sitemap plus good internal links is the best combination.
A sitemap and internal links complement each other. They are not alternatives.
A well-maintained sitemap costs nothing and helps Google crawl and index your website. For larger websites, or ones with a lot of new content, a sitemap should be standard. For small websites it is less critical, but having one does no harm.