# Feedfetcher-Google: Google Feed Fetcher and Blocking

> FeedFetcher-Google fetches RSS and Atom feeds for Google News and WebSub. It ignores robots.txt. User agent, IP list and how to block it.

Feedfetcher is how Google fetches RSS and Atom feeds for Google News and WebSub. It acts for the users who add a feed, so it ignores robots.txt. If you want to stop it, your server must return an error status, such as `404` or `410`, to the `Feedfetcher-Google` user agent.

## User-agent string

Google's [list of user-triggered fetchers](https://developers.google.com/crawling/docs/crawlers-fetchers/google-user-triggered-fetchers) gives this string:

```text
FeedFetcher-Google; (+http://www.google.com/feedfetcher.html)
```

Google's [Feedfetcher page](https://developers.google.com/crawling/docs/crawlers-fetchers/feedfetcher) writes the name as `Feedfetcher-Google`. If you match it in logs or server rules, ignore case.

## What Feedfetcher does

Google lists Feedfetcher as a user-triggered fetcher. A fetcher usually makes a single request for a user. A crawler finds and scans sites automatically. Google's Feedfetcher page gives these facts:

1. It fetches RSS and Atom feeds for Google News and [WebSub](https://www.w3.org/TR/websub/). WebSub is a standard that tells subscribers when a feed changes.
2. It stores and refreshes feeds that users of an app or service requested.
3. It does not follow links. It fetches only the feed URL that users gave it.
4. It requests a common feed only once for all users who asked for it.
5. For most sites, it fetches a feed no more than once every hour on average. It can fetch sites that change often more frequently.

Google also says that only podcast feeds get indexed in Google Search.

## robots.txt does not apply

Google says Feedfetcher acts as a direct agent of a human user, not as a robot. For this reason, it ignores robots.txt rules. A `User-agent: Feedfetcher-Google` group in your robots.txt has no effect.

## Block Feedfetcher on the server

Google gives one method for this. If your feed is public, configure your server to return `404`, `410` or another error status to the `Feedfetcher-Google` user agent. For example, in nginx, add this to the `server` block:

```nginx
if ($http_user_agent ~* "feedfetcher-google") {
    return 410;
}
```

If a blog host or another hosting service serves your feed, ask that service to restrict access.

After the block, Google services that use Feedfetcher cannot get your feed. The rule matches only Feedfetcher, so [Googlebot](/ai-crawler-bot/googlebot/) still crawls your web pages for Search.

## Verify requests

Google publishes the Feedfetcher IP addresses in [user-triggered-fetchers-google.json](https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json). Google's [verification guide](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests) says that IP addresses in this list resolve to a `google.com` hostname.

1. Run a reverse DNS lookup on the IP address from your log.
2. Make sure that the hostname ends in `google.com`.
3. Run a forward DNS lookup on that hostname.
4. Make sure that the result is the original IP address.

## Related fetchers

Google Publisher Center uses a different fetcher, `GoogleProducer`, for feeds that publishers supply for Google News landing pages. Other feed readers run their own fetchers. For example, [Feedly](/ai-crawler-bot/feedlybot/) identifies itself as `Feedly/1.0`.

---

[View the canonical page](https://aichatwatch.com/ai-crawler-bot/feedfetcher-google/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)