# MLBot: Undocumented Crawler User Agent and Blocking

> MLBot has no operator documentation. What public bot lists say, what is unknown, and how to block it with robots.txt and nginx.

MLBot is a user agent that no operator documents today. Public bot lists connect it to a company called Metadata Labs, but that bot page no longer exists. If you see MLBot in your logs and do not want it, block it in robots.txt and at your server.

## What is known about MLBot

The open [crawler-user-agents list](https://github.com/monperrus/crawler-user-agents) on GitHub records this user-agent string:

```
MLBot (www.metadatalabs.com/mlbot)
```

The list describes MLBot as a Metadata Labs web crawler for analysis. On 2026-10-22, the address `www.metadatalabs.com/mlbot` sent visitors to a parked landing page with no information about a bot.

## What is not known

We found no operator page that states any of these facts:

- What MLBot collects, or why.
- Whether MLBot follows robots.txt.
- Which IP addresses or reverse DNS names MLBot uses.

The letters "ML" in the name do not prove a link to machine learning. We found no source that says MLBot collects data to train AI models.

## Block MLBot

To ask MLBot to stay away, add this group to your robots.txt file:

```
User-agent: MLBot
Disallow: /
```

No source confirms that MLBot reads robots.txt. If the requests continue after you add the group, block the user agent at your web server, CDN or firewall. This nginx example returns HTTP 403 to requests whose user agent starts with `MLBot`:

```
if ($http_user_agent ~ "^MLBot") {
    return 403;
}
```

Put the rule in the `server` block of the nginx configuration for your site. The match is case-sensitive and anchored to the start, so it does not catch longer names that contain the same letters.

A user-agent rule stops only clients that send this string. If the same traffic comes back with a different user agent, block its IP addresses instead.

For crawlers that publish their purpose and their robots.txt behavior, see the [introduction to AI crawler bots](/ai-crawler-bot/introduction-to-ai-crawler-bots/).

---

[View the canonical page](https://aichatwatch.com/ai-crawler-bot/mlbot/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)