webzod
account

Let search engines crawl the homepage on WordPress

seo.indexability.robots_blocked · effort: minutes

Why it matters

robots.txt tells Googlebot or all crawlers not to fetch the homepage, so search engines cannot read it and will show little or nothing for it, although the address can still be listed without a description. This is often left over from a staging site.

Staged change. Do this in stages and check each one before the next, so a mistake does not break your site or your mail.

The fix

  1. Check that it is not deliberate, then open Settings, Reading and clear Discourage search engines from indexing this site. Older WordPress versions also wrote a Disallow rule for it.
  2. Check the robots.txt editor of your SEO plugin and any physical robots.txt in the web root for a Disallow line, and clear caches.

Read the documentation (opens in a new tab)

Scan your domain to see whether this applies to you.