Claude-User: Anthropic Agent for User Requests

Claude-User is the Anthropic user agent that opens a web page when a person asks Claude a question that needs it. Anthropic describes these visits as user-initiated requests, not as an automatic crawl. If you want Claude to read your pages for its users, allow Claude-User. If you do not, block it in robots.txt.

What Claude-User does

The crawler help page says that Claude can access websites with the Claude-User agent when people ask it questions. The token lets site owners decide which sites Claude can open for these requests. Cloudflare puts Claude-User in the “AI Assistant” category of its AI Crawl Control bot list.

Two other tokens cover other jobs: ClaudeBot for model training and Claude-SearchBot for the search index.

robots.txt token and user agent

The robots.txt token is Claude-User. robots.txt is a text file at the site root with rules for crawlers. No full user-agent string is published for this agent. To find it in your logs, search for Claude-User in the User-Agent field. These requests follow the questions of users, not a crawl schedule.

Does Claude-User obey robots.txt?

Yes. The help page says that the Anthropic bots obey robots.txt, and Claude-User is one of the three bots on that page. The API documentation for the web fetch tool also lists robots.txt among the reasons that a fetch can be refused.

Other operators handle this differently. OpenAI does not promise that its user-initiated agent, ChatGPT-User, obeys robots.txt.

Block Claude-User

If you want to stop Claude from opening any page on your site, add this group to the robots.txt file in the top-level directory:

User-agent: Claude-User
Disallow: /

If you want to block only part of the site, disallow that path:

User-agent: Claude-User
Disallow: /members/

Add the group on each subdomain that you want to opt out.

What a block changes

The help page says that a block stops the system from getting your content in response to a user query. This can reduce the visibility of your site when users search the web through Claude. Claude also cannot read your page for a user who pastes its URL into a chat.

The block covers only this agent. Training and search indexing keep their own rules.

Verify requests

The IP ranges of the Anthropic bots are at claude.com/crawling/bots.json. A Claude-User request from an address outside this list is unverified.

The help page advises against an IP block as an opt-out, because the block also stops the bots from reading your robots.txt. If you think that Claude-User does not obey your rules, write to [email protected] from an address at your domain.

For Claude as a product, see the Claude guide.