# PetalBot: Huawei Petal Search Crawler and robots.txt

> PetalBot crawls for Huawei Petal Search and AI Search. User agents, robots.txt rules, reverse DNS check and what a block changes.

PetalBot is the crawler of Petal Search, the search engine from Huawei. It builds the index for Petal Search and for content suggestions in Huawei Assistant and AI Search. Huawei says that PetalBot obeys robots.txt, so you can block or limit it with a `PetalBot` group.

## What PetalBot does

Huawei's [PetalBot page](https://webmaster.petalsearch.com/site/petalbot) says that PetalBot visits PC and mobile websites and builds an index database. Users can then find your content in Petal Search. Huawei also uses the index for content suggestions in Huawei Assistant and AI Search services, which Petal Search powers.

## User-agent strings

A user agent is the client name in the User-Agent HTTP header. Huawei lists two strings:

| Crawler | Full user agent |
|---|---|
| PetalBot (PC) | `Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot)` |
| PetalBot (mobile) | `Mozilla/5.0 (Linux; Android 7.0;) AppleWebKit/537.36 (KHTML, like Gecko) Mobile Safari/537.36 (compatible; PetalBot;+https://webmaster.petalsearch.com/site/petalbot)` |

The robots.txt token is `PetalBot`.

## Block or limit PetalBot in robots.txt

robots.txt is a text file at the root of your site with rules for crawlers. Huawei states: "PetalBot complies with the Internet robots protocol." If you want to block PetalBot on the whole site, add this group:

```
User-agent: PetalBot
Disallow: /
```

If you want to block only some paths, list them. Huawei gives this example:

```
User-agent: PetalBot
Allow: /w/api/
Disallow: /trap/
```

If PetalBot sends too many requests, Huawei asks site owners to write to petalbot@huawei.com. Huawei says that it adjusts the crawl rate to server capacity, site quality and site updates.

## What a block changes

Huawei warns that a block makes the pages on your site unsearchable in Petal Search. A block does not remove pages that are already in the index at once. Huawei says that removal from the index can take several months. If the removal is urgent, Huawei asks you to write to petalbot@huawei.com.

## Verify requests

Anyone can copy a user-agent string. Huawei recommends a reverse DNS lookup (a query that finds the hostname of an IP address). Do these steps on a host with the `host` command:

1. Run `host ` on the IP address from your logs.
2. Make sure that the hostname ends in `petalsearch.com`.
3. Run `host` on that hostname.
4. Make sure that the result is the same IP address as in your logs.

The example on Huawei's page uses hostnames in the form `petalbot- .petalsearch.com`. The PetalBot page gives no IP list.

## Related Huawei crawler

Third-party bot lists also name [PanguBot](/ai-crawler-bot/pangubot/) as a Huawei crawler. Huawei does not document it on the PetalBot page. A `PetalBot` group in robots.txt does not apply to PanguBot.

---

[View the canonical page](https://aichatwatch.com/ai-crawler-bot/petalbot/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)