Tabnine: AI Code Assistant and Its Web Fetch Tool

Tabnine is an AI coding assistant for software teams. It works with code in the team’s own environment, and Tabnine documents no web crawler. Only the Tabnine CLI has an optional Web Fetch tool, which reads URLs that a developer gives it. Tabnine does not document a user agent for these requests.

What Tabnine does

Tabnine describes itself as a private AI coding agent that runs in the customer’s environment and connects to code repositories and development systems (Tabnine docs). Developers use Tabnine Agent in an IDE or in the terminal, and Tabnine Chat answers questions about a codebase. An IDE (integrated development environment) is a program for writing code.

The native tools of the IDE agent read and write project files, run terminal commands and search the local workspace (In-IDE Agent Settings). None of these native tools reads web pages. A team can add more tools through MCP servers. MCP (Model Context Protocol) is a standard way to connect AI agents to other services.

Web Fetch in Tabnine CLI

The Web Fetch documentation gives these facts:

• Web Fetch is disabled by default. A developer turns it on with /settings or in settings.json.
• The agent uses it when a prompt contains URLs and the task needs their content.
• It is not a general web crawler. It reads only the URLs that the developer gives, and sometimes the next or previous page of the same documentation.
• It takes up to 20 URLs per request and is rate limited to 10 requests per minute per host.
• In the default mode, Tabnine sends the URL to the LLM API for processing. An LLM (large language model) is the AI model that writes the answer.
• In the experimental Direct Fetch mode, the CLI fetches the URL itself over HTTP.

This is the setting that turns the tool on:

{
  "tools": {
    "enableWebFetchTool": true
  }
}

Tabnine CLI is in maintenance mode. Tabnine says that it will get only urgent updates through December 31, 2026, and that it will then be deprecated (Tabnine CLI). Tabnine moves CLI users to the Tabnine Plugin for OpenCode.

User agents and blocking

Tabnine does not document a user agent, a robots.txt token or IP ranges for Web Fetch. In the default mode, Tabnine sends the URL to the LLM API, and the documentation does not say which system then fetches the page. In Direct Fetch mode, the CLI fetches the page from the computer of the developer.

Web Fetch reads only pages that a developer asks for. For this reason, a site owner cannot target Tabnine by name. If you want to limit automated reads of your pages, read How to Find and Block AI Crawlers. It shows how to find bots in server logs.