AI SEARCH ENGINES

AI Search Engines

Comprehensive guides to AI-powered search engines and how they discover, crawl, and surface your content.

Algolia: Hosted Site Search and the Algolia Crawler

Algolia is hosted search that companies add to their own sites, not a web search engine. Algolia Crawler user agent, robots.txt and IP.

Andi is a Y Combinator-backed AI search engine that answers with sources. What is known about it, and what it does not document for sites.

Arc Search, the AI mobile browser from The Browser Company, is shutting down. Key dates, what Browse for Me did, and what it means for sites.

Baidu uses ERNIE models in the ERNIE Bot app and for AI answers in Baidu Search. Which crawler collects pages, and the robots.txt controls.

Brave Search uses its own index for AI Answers and Ask Brave. How the Brave crawler follows Googlebot rules and how to remove a page.

CoCounsel is a legal AI assistant from Casetext, now part of Thomson Reuters. It is not a public web search engine. Key dates and facts.

How ChatGPT search finds and cites web pages, which OpenAI bots to allow or block, how to opt out, and how ChatGPT visits show in analytics.

How products appear in ChatGPT shopping: OAI-SearchBot, OpenAI product feeds for approved merchants, feed flags and Instant Checkout.

How Claude finds, reads and cites web pages in its apps and in the API, which Anthropic bots to allow, and how to keep search but block training.