How search ranking works

What happens between your site and a results page

Search engines work in three stages. First they crawl: automated software follows links and sitemaps to fetch pages. Then they index: fetched pages are parsed, understood and stored. Only then do they rank: for each query, indexed pages are ordered by hundreds of signals: relevance to the words used, content quality and depth, page experience, and what the rest of the web says about you.

The crucial insight: the first two stages are almost completely under your control. If crawlers are blocked in robots.txt, if pages carry a noindex tag, or if your site has no sitemap and poor internal links, nothing downstream can work; the best content on earth ranks nowhere if it was never indexed.

What this site measures

Our search-visibility category checks the controllable foundation: robots.txt access, noindex directives, an XML sitemap, canonical URLs, HTTPS redirects, one canonical host, and real 404 handling. These are pass/fail facts, not opinions; each check shows exactly what our crawler saw on your site.

How to improve

• Make sure robots.txt doesn't block sections you want found.
• Publish an XML sitemap and reference it from robots.txt.
• Declare one canonical host (www or bare domain) and redirect the other.
• Return real 404 status codes for missing pages so error pages aren't indexed.
• Link every important page from your navigation or homepage.
• Use Google Search Console (free) to see exactly how Google crawls and indexes you.

Common mistakes

A staging noindex tag that ships to production and silently removes the whole site from results. A blanket 'Disallow: /' left in robots.txt. Both www and non-www serving the same content as duplicates. Soft 404s, error pages that return 200 and get indexed as real content.