Probador de robots.txt

1. Paste the actual rules: Copy your robots.txt content into the input. The tool does not fetch a live file. 2. Choose the crawler and paths: Enter a crawler product token such as Googlebot and up to 50 paths or full URLs. Test rules for the site the file belongs to. 3. Review the winning rule: Run the test and inspect the selected groups, allowed status and winning line. Fix your real file separately.

No. This checks pasted Allow and Disallow rules only. It does not inspect robots meta tags, X-Robots-Tag headers, canonical tags, HTTP responses or Google indexing.

The tester selects the longest matching crawler token prefix and merges equally specific groups. It supports case-sensitive paths, query strings, * wildcards, terminal $ anchors, percent-encoded paths and Allow winning an equal-length conflict. It is a local approximation, not a Google certification or live crawler test.

Empty Allow and Disallow values are ignored. Sitemap lines do not affect crawling rules. Unsupported directives such as Crawl-delay are reported but not evaluated. Invalid path rules are flagged and skipped; review warnings before relying on the result.

Up to 500 KB of robots text, 500 groups, 500 non-empty rules and 50 paths, each up to 2,048 characters. Complex matching can stop with an error instead of freezing your browser.