Robots.txt Tester: Check if a URL Is Blocked

Paste your robots.txt rules and a URL to see whether that URL is allowed or blocked for Googlebot and other crawlers. Catch a rule that blocks important pages before it costs you traffic.

🤖 Robots.txt Tester

Paste your robots.txt and a URL to see whether Googlebot or Bingbot is allowed to crawl it — and exactly which rule decides it.

How to use the Robots.txt Tester

Paste the contents of your robots.txt file.
Enter the URL path and choose the user agent.
Read the result and fix any rule that blocks pages you want indexed.

Why it helps your SEO

Prevent accidents: one wrong Disallow line can hide a whole site.
Before launch: test changes before they go live.
Debug indexing: rule out robots.txt when pages are not indexed.

Frequently asked questions

Does robots.txt remove pages from Google?

No. It blocks crawling, not indexing. Use a noindex tag to keep a page out of results.

Where is my robots.txt?

At yoursite.com/robots.txt.

More free tools: see all SEO tools.