How do I check whether AI bots can access my website?
AI bots, the programs ChatGPT, Perplexity, Claude and Google use to read web pages, get onto your site if nothing stops them along the…
ChatGPT most often can't see a site that's in Google because it searches other search engines, including Bing. If Bing doesn't know the site or OpenAI's bot can't get onto it, ChatGPT usually won't show it, even when it ranks first in Google.
Below are the 5 most common causes: how to check each one and what to do about it.
When ChatGPT looks something up online, it uses the search engines it partners with, and one of them is Microsoft’s Bing. It turns the question into a few short queries, sends them, looks at the results and sometimes follows up with a more specific search. That’s how OpenAI’s help center describes it. For a site owner, one thing follows: ranking high in Google doesn’t guarantee that ChatGPT will see your site at all.
Think of two libraries in the same town. A book can sit on the shelf in one, while in the other nobody has heard of it.
ChatGPT often describes a site using what the search engine has stored about it: the title and a short description. If Bing last visited the site six months ago, ChatGPT will tell you the six-month-old version. A bit like the friend who asks at every get-together about the job you quit three years ago.
That’s why, when I check a site’s visibility in ChatGPT, I start with Bing.
If ChatGPT can't see your site, I'll find the stage where it drops out.
Book a free callMost often ChatGPT can’t see a site because Bing doesn’t know it yet. This applies especially to new sites and new pages. Google and Bing keep separate indexes and visit sites at their own pace, so a new URL may already be in Google but not yet in Bing. A page that no other page on the site links to may never get into Bing at all, because the bot has no way to reach it.
How to check:
site:yourdomain.com into bing.com. Bing will show every page from your domain that it knows;What to do if the site isn’t there:
When Bing remembers an old version of a page, ChatGPT knows the page but says things about it that are months out of date: the old title, the old offer, a sentence that disappeared long ago. The page is in Bing, but the bot hasn’t been back since the changes. ChatGPT has no way of learning about them, so it repeats what Bing remembered.
How to check: search for the page on bing.com and compare its title and description with what you see in Google and on the page itself. Bing Webmaster Tools also shows when the bot last visited.
What to do: after every important change, such as a new title, a new offer or new company details, submit the URL through IndexNow or in Bing Webmaster Tools. Test ChatGPT only after Bing has visited the page again. Before that, you’re testing Bing’s memory, not your site.
A bot is a program that visits pages in place of a person and reads what’s on them. Any site can shut the door on some bots, most often in the robots.txt file, a short list of „who can come in and who can’t.” OpenAI has several bots, and each has a different job (OpenAI documentation):
robots.txt doesn’t let it in, the site won’t appear in search answers. After a change in the file, allow about 24 hours;robots.txt;Bing has its own bot too (Bingbot). If it can’t get in, the site won’t make it into Bing, and ChatGPT loses one of the main routes it uses to find it.
robots.txt isn’t the only thing that shuts the door. Cloudflare, a service that a lot of websites route their traffic through, has separate settings for AI bots. Domains added to Cloudflare from September 15, 2026 block agent bots and model-training bots by default on pages with ads, and the block can also catch bots that search on the side (Cloudflare, July 2026). Server security can do the same: OpenAI states plainly that your host has to allow traffic from its bot’s IP addresses.
How to check: type yourdomain.com/robots.txt into your browser and look for the bot names from the list above. If your site uses Cloudflare, check its AI bot settings. Only the server logs, the record of every visit, give you certainty: they show whether OpenAI’s bots come in and whether they get the page or an error. I laid out every step, from robots.txt to the logs, in the guide how to check whether AI bots can access your site.
Some pages only assemble themselves in the browser after a moment. The server sends an empty skeleton, and a JavaScript script adds the text. A person doesn’t notice, because the browser does it in a fraction of a second, and Google usually copes too. The AI bots in Vercel’s December 2024 study, including OpenAI’s and Anthropic’s, didn’t run JavaScript. A bot like that sees a frame without a picture: the address is there, the content isn’t.
How to check: right-click on the page and choose „View page source” (not „Inspect”). That’s roughly what the bot gets. If you don’t see the heading and the first paragraphs there, only script code, the bot sees a blank page.
What to do: the most important things, the heading, the first sentences, the offer and the company details, have to be in what the server sends. In WordPress, that’s usually the case. The trouble shows up more often with some site builders and stores built on JavaScript. Then it’s a job for a developer, and the phrase to use with them is „server-side rendering.”
If the site is in Bing, the bots can get in and the text is in the code from the start, the technical side is fine. Then the content decides whether ChatGPT shows the site. ChatGPT breaks the question into several smaller ones and builds its answer from the passages that best answer each of them. In my test on October 11, 2026, it also ran a separate site: search to check whether the page stated outright what the user had asked about, for example how many years someone has been running campaigns.
If that information isn’t on the page as a sentence, the page drops out, even though technically everything is fine. Experience that only shows up in a graphic or an animated counter doesn’t exist for ChatGPT.
How to check: run the clean test described below and compare the words from the question with what the page says as text.
What to do: add the missing information in plain sentences, in the customer’s words. I collected all the points I check in a text like that in the article How to appear in AI answers: a checklist for writing website copy. How that checklist worked on one of my own articles, I show in the case study: first place in Google and six citations in an AI Overview a week after publication.
A ChatGPT test is easy to spoil: one follow-up question and the result shows what we fed it ourselves. So I set up the conditions first and then check two things:
Checking whether your site is in Bing and whether Bing knows its current version is an afternoon’s job, and you can do it yourself. Bot blocks and blank pages usually need access to the server, Cloudflare or the code, so this is where someone technical comes in handy. The most work goes into the case where ChatGPT sees the site but picks others: that’s content work, meaning what the page says outright, in which words, and whether it has the proof AI is looking for.
If you’d like me to look at this on your site, from the logs and Bing to tests in ChatGPT, here’s how it works: SEO and AI search visibility.
Author - Michał Wojciechowski
I have run Google Ads accounts myself since 2019 and I create the ads and copy for them. I have run campaigns on 9 markets and worked on projects on 8 more.
I started in graphic design and film, so I look at an ad the way a viewer does. I work directly with business owners: regular calls, decisions based on data, and work on the offer and the landing page, not only on the ads.
One hour on a call is usually enough to see why the sales are not coming in.