# Amazonbot: Amazon Crawler User Agent and robots.txt

> Amazonbot, Amzn-SearchBot and Amzn-User: what each Amazon agent does, user agent strings, IP lists, and how to block them in robots.txt.

Amazonbot is the web crawler of Amazon. Amazon says that it uses Amazonbot to improve its products and services, and that the data can be used to train Amazon AI models. Amazon also runs Amzn-SearchBot and Amzn-User, and you control each of the three agents separately in robots.txt.

## Amazon user agents

Amazon describes its three agents on the [Amazonbot page](https://developer.amazon.com/amazonbot):

| Agent | What Amazon uses it for | AI model training |
|---|---|---|
| Amazonbot | Improves Amazon products and services | Amazon says the data can be used for training |
| Amzn-SearchBot | Improves search in Amazon products and services, such as Alexa | Amazon says it does not crawl for generative AI training |
| Amzn-User | Gets live information from the web for user actions, for example an Alexa question | Amazon says it does not crawl for generative AI training |

Amazon gives these example user agent strings. Amazon writes the Chrome version in them as `W.X.Y.Z`.

```
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amazonbot/0.1) Chrome/W.X.Y.Z Safari/537.36
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-SearchBot/0.1) Chrome/W.X.Y.Z Safari/537.36
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Amzn-User/0.1) Chrome/W.X.Y.Z Safari/537.36
```

The robots.txt tokens are `Amazonbot`, `Amzn-SearchBot` and `Amzn-User`.

## How Amazon reads robots.txt

Amazon says that its agents follow the Robots Exclusion Protocol. They read the `User-agent`, `Allow` and `Disallow` lines. Amazon gives these other rules:

- The agents read robots.txt for each host. For example, `site.example.com` uses its own file, not the file of `example.com`.
- The agents can use a cached copy of robots.txt that is up to 30 days old.
- If the agents cannot get the file, they act as if the file does not exist.
- The agents do not support `Crawl-delay`.
- If robots.txt does not name Amzn-SearchBot but allows other search bots, Amzn-SearchBot follows the rules for those bots.
- Amzn-User can skip some robots.txt rules, because a user starts its requests.
- A change can take about 24 hours to take effect.

The agents also read robots meta tags. Amazon supports `noarchive` (do not use the page for model training), `noindex`, `none`, and the link attribute `rel=nofollow`.

## Block Amazonbot in robots.txt

If you want to stop Amazonbot but keep Amzn-SearchBot, add a group for Amazonbot only:

```
User-agent: Amazonbot
Disallow: /
```

Amazon says that access for Amzn-SearchBot makes your content eligible for search experiences such as Alexa.

If you want to keep a page open to Amazon crawlers but out of model training, add the `noarchive` tag to the page. Read the [meta robots guide](/html-tags/meta-robots/) for the syntax of this tag.

```html

```

If you want to block all three Amazon agents, add one group for each name:

```
User-agent: Amazonbot
Disallow: /

User-agent: Amzn-SearchBot
Disallow: /

User-agent: Amzn-User
Disallow: /
```

Amazon's page also says that sites that allow Amazonbot can be eligible for its Amazon Content Partners program.

## Verify Amazon requests

Amazon publishes a separate IP address list for each agent:

- Amazonbot: [developer.amazon.com/amazonbot/ip-addresses/](https://developer.amazon.com/amazonbot/ip-addresses/)
- Amzn-SearchBot: [developer.amazon.com/amazonbot/searchbot-ip-addresses/](https://developer.amazon.com/amazonbot/searchbot-ip-addresses/)
- Amzn-User: [developer.amazon.com/amazonbot/live-ip-addresses/](https://developer.amazon.com/amazonbot/live-ip-addresses/)

If a request has an Amazon user agent, make sure that its source IP address is in the list for that agent. Amazon does not describe a reverse DNS check on its page.

If you block these IP addresses at your firewall, the agents cannot get any page, robots.txt included. Use robots.txt if you want to block one agent and keep the others. For questions, Amazon gives the address amazonbot@amazon.com and asks you to include your domain names.

---

[View the canonical page](https://aichatwatch.com/ai-crawler-bot/amazonbot/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)