# Google-Extended: robots.txt Token for Gemini Training

> Google-Extended is a robots.txt token, not a crawler. It controls Gemini training and grounding, not Google Search ranking or AI Overviews.

Google-Extended is a robots.txt token, not a separate crawler. It controls whether Google can use content it crawls from your site to train future Gemini models and to ground Gemini answers. It does not change your inclusion or ranking in Google Search, and it does not control AI Overviews or AI Mode.

## What Google-Extended controls

Google describes Google-Extended in its [list of common crawlers](https://developers.google.com/crawling/docs/crawlers-fetchers/google-common-crawlers#google-extended) as a standalone product token. If you disallow it, Google applies your choice to two uses:

1. Training future generations of Gemini models that power [Gemini Apps](/ai-chat-bot/google-gemini/) and the Vertex AI API for Gemini.
2. Grounding in Gemini Apps and in Grounding with Google Search on Vertex AI.

Grounding is when Google gives the model content from the Google Search index at prompt time. Google says the purpose is better factuality and relevance.

Google announced Google-Extended on [September 28, 2023](https://blog.google/innovation-and-ai/products/an-update-on-web-publisher-controls/). The announcement named Bard and the Vertex AI generative APIs. The current documentation names Gemini Apps and Vertex AI.

## What it does not control

Google says Google-Extended does not affect whether your site is in Google Search. Google also says it is not a ranking signal.

AI Overviews and AI Mode are part of Google Search. For these features, Google's [AI features page](https://developers.google.com/search/docs/appearance/ai-features) names robots.txt rules for [Googlebot](/ai-crawler-bot/googlebot/) as the control. To limit the text that Search shows from a page, use `nosnippet`, `data-nosnippet`, `max-snippet` or `noindex`. A Google-Extended rule does not remove your pages from [AI Overviews](/ai-search-engine/google-ai-overviews/).

## No user-agent string

Google-Extended has no user-agent string of its own. Google crawls with its existing user-agent strings and uses the Google-Extended token only as a control. You will not see `Google-Extended` in your server logs.

For this reason, a server or firewall rule cannot separate Google-Extended from Search crawling. A rule that blocks Google's crawler user agents also blocks Googlebot. Use robots.txt for this control.

## Block Google-Extended in robots.txt

If you want to opt your whole site out of Gemini training and grounding, add this group to `/robots.txt`:

```text
User-agent: Google-Extended
Disallow: /
```

If you want to opt out only one folder, disallow that path:

```text
User-agent: Google-Extended
Disallow: /articles/
```

Keep your `Googlebot` rules as they are. The Google-Extended group does not change them.

Google lists only one token for this control: `Google-Extended`. For this reason, a `User-agent: Googlebot` group does not set your Google-Extended choice. Google caches robots.txt for up to 24 hours ([robots.txt specification](https://developers.google.com/crawling/docs/robots-txt/robots-txt-spec)), so a change can take a day to apply.

## Related Google tokens

| Token | What it controls |
|---|---|
| [`Googlebot`](/ai-crawler-bot/googlebot/) | Google Search, including AI Overviews and AI Mode |
| [`GoogleOther`](/ai-crawler-bot/googleother/) | Crawls by product teams and internal research, no specific product |
| [`Google-CloudVertexBot`](/ai-crawler-bot/google-cloudvertexbot/) | Crawls that site owners request for Vertex AI Agents |

For other companies' AI crawlers, see the [robots.txt guide for AI crawlers](/guide/robots-txt-ai-crawlers/).

---

[View the canonical page](https://aichatwatch.com/ai-crawler-bot/google-extended/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)