🤖 Free tool

PerplexityBot Robots.txt Tester

See whether PerplexityBot can find, read, and recommend your website. The PerplexityBot Robots.txt Tester runs in your browser — enter a URL and we email you the report with the exact fixes.

Free · No signup · Report emailed in ~1 minute

What is the PerplexityBot Robots.txt Tester?

The PerplexityBot Robots.txt Tester fetches your robots.txt and checks whether AI crawlers — GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, and Google-Extended — are allowed, blocked, or unmentioned. It also verifies your sitemap reference and flags rules that quietly hide you from AI search. Results are emailed to you.

What the PerplexityBot Robots.txt Tester checks

  • Whether robots.txt exists and parses correctly
  • Access for each major AI bot: GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended
  • Traditional crawlers: Googlebot and Bingbot
  • Whether a sitemap.xml is declared and reachable
  • Overly broad Disallow rules that block AI discovery
  • Meta robots and X-Robots-Tag directives that conflict with robots.txt

How it works

  1. 1

    Enter your URL

    Paste any page on your domain — we read robots.txt from the root.

  2. 2

    We test every AI bot

    We evaluate each crawler individually against your rules to see who can reach you.

  3. 3

    Get your report by email

    You receive a per-bot allowed/blocked breakdown and the fixes to unblock AI search.

Frequently asked questions

Which AI crawlers should robots.txt allow?

To be discoverable in AI search, most sites allow GPTBot and ChatGPT-User (OpenAI), ClaudeBot (Anthropic), PerplexityBot (Perplexity), and Google-Extended (Google AI). Blocking them removes you from those engines.

Will this change my robots.txt?

No. The tester only reads and evaluates your existing robots.txt. It shows you exactly which rules to change, but never modifies your site.

What happens if I block GPTBot?

If GPTBot is disallowed, OpenAI’s crawler will not index your pages, so ChatGPT and SearchGPT are far less likely to cite or recommend your site.

Is PerplexityBot allowed to crawl my site?

The PerplexityBot Robots.txt Tester checks your robots.txt specifically for PerplexityBot and reports whether it is allowed, blocked, or unmentioned — then shows you the rule to change if it cannot reach you.

Also known as

The same free tool, by the names people search for it:

Robots.txt TesterRobots.txt CheckerRobots.txt ValidatorRobots.txt AnalyzerRobots.txt AuditRobots.txt AI Crawler TesterAI Crawler TesterAI Bot TesterGPTBot Robots.txt TesterClaudeBot Robots.txt TesterGoogle-Extended Robots.txt TesterFree Robots.txt TesterOnline Robots.txt TesterInstant Robots.txt TesterFree Robots.txt CheckerFree Robots.txt Validator

More free AI visibility tools

🛰️

AI Visibility Checker

See how ChatGPT, Perplexity, Gemini, and Google AI Overviews find, read, and recommend your website — scored 0–100.

📄

llms.txt Checker

Check whether your site publishes a valid llms.txt file with the sections AI systems expect — and what to add.

🧩

Schema Markup Checker

Validate the JSON-LD structured data on your page — Organization, FAQ, Product, Article, and more.

📖

AI Readability Checker

Measure whether AI models can cleanly extract answers from your content — headings, structure, and clarity.

Fix what the scan finds — automatically

CookMyRank turns every issue this tool surfaces into a deployable fix — schema, metadata, llms.txt, and content — that you approve before it ships.

See how it works