ChatGPT Crawler Simulator & GPTBot Tester
Last updated
Simulate a ChatGPT or GPTBot visit to any URL on your website. Crawl the page and inspect the HTML OpenAI's bots get, so ChatGPT can read and cite it.
Simulate a ChatGPT Fetch visit
Enter any URL on your website to crawl it as ChatGPT Fetch and inspect the HTML it gets.
How to use the ChatGPT Fetch simulator
- Step 1
Enter a URL
Paste the full URL of any page on your website, such as a product page, a blog post, or the home page.
- Step 2
Select the ChatGPT Fetch user agent
The tester sends the request with the ChatGPT Fetch user-agent string, so your server answers the way it answers a real ChatGPT Fetch visit.
- Step 3
Run the crawl
The simulator requests the page as ChatGPT Fetch and follows redirects, the same way the crawler does.
- Step 4
Inspect the HTML
Check the status code, redirects, response headers, raw HTML, word count, and Open Graph tags ChatGPT Fetch gets. If your main content is missing from the HTML, ChatGPT Fetch cannot read it.
What is ChatGPT Fetch?
ChatGPT uses several crawlers to access web content: GPTBot for training data collection, ChatGPT-User for real-time web browsing in ChatGPT Plus, and OAI-SearchBot for SearchGPT. When users ask ChatGPT questions, it may fetch your website content to provide accurate, up-to-date answers. If your site blocks these crawlers or serves JavaScript-only content, ChatGPT won't be able to cite your website in its responses.
Why allow ChatGPT Fetch?
ChatGPT Fetch robots.txt configuration
Control how ChatGPT Fetch crawls your website using robots.txt directives. Add these rules to your robots.txt file at the root of your domain, then run the simulator again on the same URL to confirm the change.
Allow ChatGPT Fetch
# Allow ChatGPT and OpenAI crawlers User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: OAI-SearchBot Allow: /
Block ChatGPT Fetch
# Block ChatGPT and OpenAI crawlers User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: OAI-SearchBot Disallow: /
User-Agent String: Mozilla/5.0 (compatible; ChatGPT-User/1.0; +https://openai.com/)
ChatGPT Fetch simulator FAQ
Related Crawler Simulators
More Free SEO Tools
Continue auditing your site with these companion tools.
30 checks for crawlability, SEO, and AI readiness
See what 100+ search and AI bots get from your page
Fetch your page as Googlebot Smartphone or Desktop
Check which crawlers are allowed or blocked
Preview social media link cards
Audit meta tags and generate a fix prompt
Check if ChatGPT, Claude, and Perplexity can read your site
Related Guides
Dive deeper with these step-by-step guides and tutorials.
Make your site crawlable
JavaScript sites often have indexing issues. Encited pre-renders your SPA into crawler-friendly HTML so ChatGPT Fetch and other bots can read your content.
