Naverbot (Yeti): Naver Crawler User Agent and robots.txt
Naver calls its search crawler Yeti. User-agent strings, robots.txt examples, HTTP status handling and how to identify real requests.
Naverbot is a common name for the web crawler of Naver, the Korean search engine. The official name of the crawler is Yeti. If you want to control it, use the token Yeti in robots.txt.
User-agent strings
The crawler guide in Naver Search Advisor (in Korean) lists two user-agent forms for Yeti. The guide says that version details can change without notice. The first form looks like a Chrome browser, and W.X.Y.Z stands for the Chrome version:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Yeti/1.1; +https://naver.me/spd) Chrome/W.X.Y.Z Safari/537.36The second form is short:
Mozilla/5.0 (compatible; Yeti/1.1; +https://naver.me/spd)The name Naverbot appears in the file name of the IP list, naverbot.json. The robots.txt examples use only the token Yeti.
Robots.txt rules for Yeti
The robots.txt guide says that Yeti obeys robots.txt. If a site has no robots.txt file, Yeti treats all content as open to collection.
To block Yeti from the full site, add this group:
User-agent: Yeti
Disallow: /To block paths such as /private-image and /private-video, the guide gives this example:
User-agent: Yeti
Disallow: /private*/The guide also gives an example that allows only Yeti:
User-agent: *
Disallow: /
User-agent: Yeti
Allow: /Yeti reads the HTTP status of your robots.txt file as follows:
| Status | What Yeti does |
|---|---|
| 2xx | Reads the rules. A robots.txt file sent as HTML can count as no file. |
| 3xx | Follows up to 5 redirects. After that, it treats the site as fully allowed. |
| 4xx | Treats the site as fully allowed. |
| 5xx | Treats the site as fully disallowed. It can use an earlier copy of the rules for a time. |
The guide also says that robots for special uses do not always follow robots.txt. Examples are robots under a separate agreement with the site owner, robots that collect advertiser information, and robots that make link previews. The guide tells site owners to protect private content with a login or another method.
Other Naver robots
The crawler guide names two special-purpose robots:
| User agent name | Use |
|---|---|
Ads-Naver |
Collects more information from advertiser pages for ad display |
Blueno |
Collects link previews, for example when a user adds a link in the blog editor |
Identify real requests
The crawler guide documents a two-step DNS method:
- Do a reverse DNS lookup on the IP address of the request. Make sure that the name ends in
.naver.com, for examplecrawl.125-209-235-169.web.naver.com. - Do a forward DNS lookup on that name. Make sure that it returns the original IP address.
You can also compare the IP address with the IP range list. The guide says that the list can change without notice.
AI training and AI answers
The crawler and robots.txt guides do not say whether Yeti data is used to train AI models. They also do not name a separate robots.txt token for AI training. AI Briefing, the AI answer feature in Naver search, uses the search index. The Naver Cue: guide covers it.