YandexGPT in Yandex Search: AI Answers and Opt-Out

YandexGPT is a family of large language models from Yandex. Yandex Search uses generative AI to write answers from indexed pages, with links to the source pages. If you do not want your pages in these AI answers, add a Disallow rule for YandexAdditional or YandexAdditionalBot in robots.txt.

YandexGPT models

Developers use the models through Yandex AI Studio, a service of Yandex Cloud. Its model list includes YandexGPT Pro 5.1, YandexGPT Pro 5 and YandexGPT Lite 5. The same list includes newer models named Alice AI LLM and Alice AI LLM Flash.

AI answers in search

The Webmaster help describes two kinds of AI answers in search.

The Russian help page on quick answers names “quick answers with YandexGPT”. YandexGPT finds a few sources that match the query in real time. It then summarizes each source and picks the most useful answer. The help page says that these answers appear only in Russia.

The English help calls the second feature Yandex AI. The Russian help calls it Alice AI answers. The help describes it in three steps:

1. It searches the web for information about the query.
2. A generative neural network analyzes the sources in real time.
3. It combines the findings into one answer with references.

All these answers include references to the sources, except solutions to school math problems. The help pages do not say which model writes these answers.

Which pages become sources

The help says that AI answers come automatically from indexed pages. If you want your pages to be sources, let the search crawler index them.

The English help says that a page is more likely to be used if it is well structured, informative and grammatically correct. The Russian help adds these points:

• Alice AI uses pages that rank high in search as sources.
• To select sources, it looks at expertise, usefulness, originality and depth of content.
• It does not copy page text word for word, except quotes of direct speech.
• It does not rank the links. It puts them in the order of the answer, and it can link several pages of one site.

Opt out with YandexAdditional

If you want to keep the full site out of AI answers, add this group to robots.txt:

User-agent: YandexAdditional
Disallow: /

If you want to keep only one page out, use a path. This is the example from the help page:

User-agent: YandexAdditionalBot
Disallow: /cats/mycat.html

The change starts when the robots crawl the page again. This usually takes 2 to 14 days.

The Russian list of robots gives these user agents for the two tokens:

Mozilla/5.0 (compatible; YandexAdditional/1.0; +http://yandex.com/bots)
Mozilla/5.0 (compatible; YandexAdditionalBot/1.0; +http://yandex.com/bots)

The same list says that these tokens are read from robots.txt. They apply to pages that the main indexing robot indexed. The list also says that they do not send indexing requests.

What the opt-out does not change

The help describes the YandexAdditional rule only as a control for AI answers. It does not describe it as a way to remove pages from search results.

If you want to remove a page from search, robots.txt is not the right tool. The help says that pages blocked in robots.txt can still appear in search. Use the noindex directive in the page or in an HTTP header instead. See how to exclude pages from search.

Which bots to allow or block

YandexBot indexes pages for search, and so it decides which pages can be AI answer sources. Its guide gives the user agents, the robots.txt rules and the DNS check for real requests.

YandexAdditional and YandexAdditionalBot are the robots.txt tokens for the AI answer opt-out above.