Changed robots.txt? See how each AI crawler reads it

One line in the wrong group can block ChatGPT search while you meant to block only AI training. Check the result for 16 crawlers.

Free · No signup · Results in 30 seconds

Which group applies

A crawler follows the group that names it, not the * group, which surprises many site owners.

Search versus training

Block training crawlers if you wish, while keeping the ones that put you in AI search answers.

The exact fix

The lines to change, ready to paste, and a recheck to confirm.

Get the complete package

After moving to Cloudflare · After a redesign · After moving to Shopify