How to Check Whether GPTBot Can Access Your Website
Use the GPTBot Checker to see whether your robots.txt file blocks or allows GPTBot and 70 other AI crawlers from reading your site.
GPTBot is the web crawler OpenAI uses to gather publicly available content for training its AI models. Whether GPTBot can read your site is controlled by your robots.txt file, and it's easy to check without any technical setup.
What GPTBot is
GPTBot is OpenAI's crawler, identified by the user-agent token GPTBot in your server logs and robots.txt rules. Site owners can allow or disallow it independently of other search engine or AI crawlers.
GPTBot vs. OAI-SearchBot — a common mix-up
GPTBot and OAI-SearchBot are separate, independent controls. GPTBot governs whether your content can be used to train OpenAI's models — allowing or blocking it has no effect on whether your site appears in ChatGPT's search results. Visibility in ChatGPT Search is controlled separately by OAI-SearchBot: per OpenAI's documentation, sites that block OAI-SearchBot generally won't appear in ChatGPT search answers, though they may still show as a navigational link. Allowing GPTBot does not, by itself, make a site show up in ChatGPT Search — the GPTBot Checker reports on both crawlers independently, so check the one that matches what you actually want to control.
How to check GPTBot access
- Open the GPTBot Checker.
- Enter your website's URL and run a check.
- The tool fetches your site's
robots.txt, evaluates it against GPTBot's user-agent token using the same longest-match, allow-wins-ties logic browsers and crawlers use, and reports whether GPTBot is allowed or blocked. - Along with GPTBot, the same scan checks 70 AI crawler tokens across four categories — model training bots, AI search index bots, user-triggered fetch bots, and agent/research bots — so you can see your site's full AI-crawler access picture in one pass, not just GPTBot.
If GPTBot is blocked and you want to allow it
Add or edit a rule in your robots.txt file:
User-agent: GPTBot
Allow: /
If you want to block it instead, use Disallow: / under the same User-agent: GPTBot line. Changes to robots.txt typically take effect the next time a crawler re-fetches the file — there's no submission step required.
Frequently Asked Questions
What is GPTBot?
GPTBot is the web crawler OpenAI uses to collect publicly available content for training its AI models. It identifies itself with the user-agent token GPTBot.
How can I check if GPTBot is blocked on my site?
Use the GPTBot Checker — enter your URL and it evaluates your robots.txt against GPTBot's user-agent token, along with 70 other AI crawler tokens.
Can robots.txt block AI crawlers?
Yes — robots.txt rules can allow or disallow specific crawlers by user-agent token, including GPTBot and other AI crawlers, the same way it controls access for traditional search engine bots.
Does allowing GPTBot make my site show up in ChatGPT Search?
No — GPTBot controls whether your content can be used to train OpenAI's models, not whether your site appears in ChatGPT Search. Search visibility in ChatGPT is controlled separately by OAI-SearchBot. The two are independent, and blocking one doesn't affect the other.