# Robots Meta Tag: noindex, nosnippet and Other Rules > Robots meta tag and X-Robots-Tag syntax, the rules Google, Bing and Apple document, and how nosnippet affects Google AI Overviews. The robots meta tag tells search engine crawlers whether they can index a page, follow its links and show a snippet of it. Put it in the page ` `, or send the same rules in an `X-Robots-Tag` HTTP header. Do not block the page in robots.txt, because then crawlers cannot read the tag. ## Syntax ```html ``` The name `robots` applies to all crawlers. If you want a rule for one crawler only, use the name of that crawler: ```html ``` You can put several rules in one tag, separated by commas, or in separate tags. Google says that the name, the content and the rules are case-insensitive. The tag is not part of the HTML standard. [MDN](https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/meta/name/robots) calls it a de facto standard for crawlers. ## Rules that Google supports Google lists these rules in its [robots meta tag specification](https://developers.google.com/search/docs/crawling-indexing/robots-meta-tag): | Rule | What Google does | |---|---| | `noindex` | Does not show the page in search results | | `nofollow` | Does not follow the links on the page | | `none` | Same as `noindex, nofollow` | | `nosnippet` | Shows no text snippet or video preview for the page | | `max-snippet:[number]` | Limits the text snippet to that number of characters. `0` means no snippet, `-1` means no limit | | `max-image-preview:[setting]` | Sets the largest image preview: `none`, `standard` or `large` | | `max-video-preview:[number]` | Limits a video preview to that number of seconds. `-1` means no limit | | `noimageindex` | Does not index the images on the page | | `notranslate` | Does not offer a translation of the title link and snippet | | `indexifembedded` | Together with `noindex`, lets Google index the content when another page embeds it in an iframe | | `unavailable_after:[date]` | Stops showing the page after the date. Use a format such as ISO 8601 or RFC 822 | | `all` | No restrictions. This is the default | If two rules conflict, Google uses the more restrictive rule. For example, `nosnippet` wins over `max-snippet:50`. Google ignores `noarchive`, `nocache` and `nositelinkssearchbox`. For Google Search, the tag accepts two crawler names: `googlebot` for text results and `googlebot-news` for news results. Google ignores other names. ## Google AI Overviews and AI Mode Google says that `nosnippet` applies to all forms of its search results, including AI Overviews and AI Mode. Google also says that `nosnippet` stops the content from being used as a direct input for AI Overviews and AI Mode. `max-snippet` also applies to AI Overviews and AI Mode. Google says that this limit does not apply where a publisher separately gave permission to use the content. If you want to hide only part of a page from snippets, put the `data-nosnippet` attribute on a `span`, `div` or `section` element: ```html This text can appear in a snippet. This text cannot. ``` Google's [AI features page](https://developers.google.com/search/docs/appearance/ai-features) names `nosnippet`, `data-nosnippet`, `max-snippet` and `noindex` as the controls for what Search shows. [Google-Extended](/ai-crawler-bot/google-extended/) is a robots.txt token, not a meta tag rule. Google says that it limits AI training and grounding in some of its other systems. ## Rules that Bing supports In an [April 2020 post](https://blogs.bing.com/webmaster/april-2020/Announcing-new-options-for-webmasters-to-control-their-snippets-at-Bing), Bing announced support for `max-snippet`, `max-image-preview` and `max-video-preview`. The post says that Bing still supports `nosnippet`. Bing reads these rules in the meta tag and in the `X-Robots-Tag` header. In a [September 2023 post](https://blogs.bing.com/webmaster/september-2023/Announcing-new-options-for-webmasters-to-control-usage-of-their-content-in-Bing-Chat), Bing gave two rules a meaning for Bing Chat: 1. `nocache`: Bing Chat can use the page, but answers show only the URL, the title and the snippet. Only those parts can be used to train Microsoft's generative AI foundation models. 2. `noarchive`: Bing Chat does not use the page in answers and does not link to it. Bing does not use the content to train Microsoft's generative AI foundation models. If a page has both rules, Bing treats it as `nocache`. Bing says that pages with either rule still appear in Bing search results. Microsoft later renamed Bing Chat to Copilot. The post does not say how the rules apply to Copilot. ```html ``` ## Rules that Applebot supports Apple says that [Applebot](/ai-crawler-bot/applebot/) supports `noindex`, `nosnippet`, `nofollow`, `none` and `all` in robots meta tags ([Apple support](https://support.apple.com/en-us/119829)). Applebot also reads the `X-Robots-Tag` header. Apple describes only a robots.txt control for [Applebot-Extended](/ai-crawler-bot/applebot-extended/), the token for Apple AI training. ## X-Robots-Tag header If the file is not HTML, for example a PDF or an image, send the rules in an HTTP response header: ```text X-Robots-Tag: noindex X-Robots-Tag: googlebot: nofollow ``` A rule without a crawler name applies to all crawlers. Google recommends the header for PDF, video and image files. ## noai and noimageai DeviantArt added the `noai` and `noimageai` directives in November 2022 to mark art that must not be used to train AI ([TechCrunch](https://techcrunch.com/2022/11/11/deviantart-provides-a-way-for-artists-to-opt-out-of-ai-art-generators)). DeviantArt's terms tell third parties to exclude this content from AI training data sets. The directives are not in Google's list of supported rules. If you want to block an AI crawler, use the robots.txt token that its operator documents, for example [GPTBot](/ai-crawler-bot/gptbot/). ## What the robots meta tag does not do The robots meta tag does not do these things: 1. It does not stop crawling. A crawler must fetch the page to read the tag. Google says that if robots.txt blocks the page, Google does not see the rules and ignores them. 2. It does not protect content. MDN says that only cooperative robots follow these rules. 3. It does not remove a page at once. MDN says that `noindex` takes effect only after the robot visits the page again. 4. In Google Search, `noarchive` and `nocache` have no effect. The [meta tags cheat sheet](/html-tags/meta-tags-cheat-sheet/) lists the other tags that search engines document. --- [View the canonical page](https://aichatwatch.com/html-tags/meta-robots/) ยท [Browse llms.txt](https://aichatwatch.com/llms.txt)