Amazonbot
Amazonbot is Amazon's crawler for improving its products and services, and Amazon says its data may be used to train Amazon AI models. It respects robots.txt. Amazon runs separate agents for search, Amzn-SearchBot, and for user requests such as Alexa queries, Amzn-User, each with its own token.
What it does
Amazon says Amazonbot is used to improve its products and services, and that its data may be used to train Amazon AI models. Amazon states that its automated crawling respects the Robots Exclusion Protocol, including user-agent and allow or disallow rules.
Amazon caches robots.txt for up to 30 days and does not support Crawl-delay. Blocking Amazonbot does not cover Amzn-SearchBot or Amzn-User, which have their own tokens.
robots.txt token
Amazonbot
User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36How to allow it
User-agent: Amazonbot
Allow: /How to block it
User-agent: Amazonbot
Disallow: /How to verify real visits
Amazon publishes the IP addresses Amazonbot uses. Match the source IP of a request against that list; Amazon does not document a reverse DNS check and asks publishers with questions to contact amazonbot@amazon.com.