AI search readiness

AI crawlers are blocked from this site

One of the 136 checks Lala SEO runs on every page of a site in a free audit.

How serious: Worth knowing. Small, or only matters in some cases. Read it and decide.
Score: Reported, but it does not lower your site's score.
Based on: Industry standard: Google does not document this one, so the check follows established practice.

What is wrong

Your robots.txt file or meta tags are set to block AI crawlers like Google's Gemini, OpenAI's GPT crawler, and similar systems. This is a deliberate choice that prevents these tools from accessing your site content to train their models or answer user questions about your business.

Why it costs you

The consequence depends on what you want. If you block AI crawlers, your site will not appear in AI-powered search results or chatbots when people ask questions that your pages could answer. You lose visibility in those emerging channels. However, if you prefer to keep your content private and prevent it being used to train AI systems, blocking them protects that intent.

How to fix it

To change this, edit your robots.txt file (in your site root) and remove the rules that disallow these crawlers, or add them to an allow list. Common rules to remove include those naming Googlebot-Extended, GPTBot, or Claude-Web. If you are using meta tags on individual pages, look for `<meta name="robots" content="noimageindex">` or similar directives and modify or delete them. This is a setting you control; there is no dependency on your hosting provider or CMS vendor.

What the audit tests

robots.txt lets GPTBot (ChatGPT) fetch the site's start page
robots.txt lets ClaudeBot (Claude) fetch the site's start page
robots.txt lets PerplexityBot (Perplexity) fetch the site's start page
robots.txt lets Google-Extended (Google's AI) fetch the site's start page
robots.txt lets CCBot (Common Crawl) fetch the site's start page

Check your site

Run a free audit of up to 300 pages. No card needed.

Start a free audit

Or check a single address with the free tool.