X-Robots-Tag HTTP header contains noindex: what it is, why it matters, and how to fix it

The server sends an X-Robots-Tag HTTP header with a noindex directive. This instructs search engines not to index the page, even if the HTML contains no noindex meta tag.

The X-Robots-Tag is processed before the page body is parsed, making it an authoritative signal that overrides on-page directives. If set unintentionally (often by server misconfiguration), it silently de-indexes pages without any visible indication in the HTML source, making it hard to diagnose.

Remove the noindex directive from the X-Robots-Tag HTTP header in your server or CDN configuration. Check .htaccess, nginx config, and any middleware that modifies response headers. Verify with curl -I to confirm the header is no longer sent.

This is a critical issue: it can remove pages from search results entirely or block them from being crawled at all. Treat it as an outage, not a backlog item — every day it persists is compounding lost visibility. It belongs to the crawlability family of checks.

Run a free Dr Urls audit: it crawls your site, detects x-robots-tag http header contains noindex on every affected page, shows example URLs, and generates a ready-to-use fix task. Re-scan after fixing to verify the issue is gone.