GoogleOther: Google Generic Crawler User Agent and Blocking

GoogleOther is a generic Google crawler. Google product teams use it to fetch public content, for example for one-off crawls for internal research and development. Rules for GoogleOther do not affect Google Search or any other specific product, so you can block it without a Search effect.

User-agent strings

Google’s list of common crawlers gives two strings for GoogleOther:

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; GoogleOther)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GoogleOther) Chrome/W.X.Y.Z Safari/537.36

Chrome/W.X.Y.Z is a placeholder for the current Chrome version. If you match these strings in logs or server rules, use a wildcard for the version.

GoogleOther has two variants:

Crawler	User-agent string	robots.txt tokens	Use
GoogleOther	See above	GoogleOther	Generic crawls of public pages
GoogleOther-Image	GoogleOther-Image/1.0	GoogleOther-Image, GoogleOther	Public image URLs
GoogleOther-Video	GoogleOther-Video/1.0	GoogleOther-Video, GoogleOther	Public video URLs

What is known and what is not

Google says that rules for GoogleOther do not affect any specific product. Google gives internal research and development as one example of its use.

Google does not name the teams that use GoogleOther. Google also does not say what happens to the content after these crawls. If your concern is Gemini training, the documented control is Google-Extended. Google describes it as a control for content that Google crawls from your site.

Block GoogleOther in robots.txt

Google lists GoogleOther among its common crawlers, which always obey robots.txt when they crawl automatically.

If you want to keep GoogleOther out of your whole site, add this group to /robots.txt:

User-agent: GoogleOther
Disallow: /

This group also applies to GoogleOther-Image and GoogleOther-Video if your file has no group for their own tokens. A crawler uses only the group with the most specific matching token (robots.txt specification).

GoogleOther does not accept the Googlebot token. A User-agent: Googlebot group does not apply to it. If you have no GoogleOther group, it follows your User-agent: * group. A GoogleOther block does not change crawling by Googlebot.

Verify requests

Google’s common crawlers generally use the IP ranges in common-crawlers.json. Their hostnames match crawl-***-***-***-***.googlebot.com or geo-crawl-***-***-***-***.geo.googlebot.com. A reverse DNS lookup finds the hostname for an IP address.

1. Run a reverse DNS lookup on the IP address from your log.
2. Make sure that the hostname matches one of the two patterns.
3. Run a forward DNS lookup on that hostname.
4. Make sure that the result is the original IP address.

Google describes both methods in its verification guide.