The #1 Reason your website isn't getting cited by AI (probably)
Part 1 of 3 one in our series: The #1 Reason your website isn't getting cited by AI (Probably) Also in this series: What Smart People Miss, How to fix it.
The short version
AI tools can only recommend sites they can read.
Three different "doors" can be locked: a robots.txt rule, a platform setting, or a firewall.
Blocking the wrong bot can remove you from AI answers, even if you only meant to protect your content.
This is the fastest fix on the list, usually one afternoon.
Time to read: 3 minutes.
Skill level: none.
Evidence: solid. A bot that can't read your page can't cite it.
Your website looks great. Your Google rankings are fine. But when someone asks ChatGPT or Perplexity for the best option in your area, you're nowhere.
Before you rewrite a single page, check the simplest cause: is your site telling AI to go away?
Why it happens: three locked doors
Each door is controlled by a different person or tool, which is why nobody notices.
Door 1: robots.txt. Every site has a small text file that tells automated visitors what they may read. Someone (a developer, an SEO plugin) may have added a rule during the "block the AI crawlers" wave of 2024. Some of those rules were broad enough to catch bots you actually want.
Door 2: a platform setting. Website builders have switches like "discourage search engines." They're meant for sites under construction and are easy to leave on after launch.
Door 3: a firewall or CDN. Services like Cloudflare sit in front of your site and can block unfamiliar bots automatically. A firewall block happens before a bot ever reads your robots.txt, so it overrides any Allow rule in that file.
What to do next
Before you unlock anything, there's a catch: not all AI bots do the same job, and "block them all" can backfire. Part 2 explains the difference in three minutes, Part 3 walks you through the 15-minute check, with steps for WordPress, Wix, Webflow, Squarespace, Shopify and Cloudflare.
Would you rather have someone check it for you? Bot access is the first thing we test in the AI Report Card. See what's included
Comments