Connected Papers: Data Source and How Graphs Are Built
Connected Papers builds similarity graphs of research papers from the Semantic Scholar corpus. How the graph works and how papers get in.
3 min read
AI SEARCH ENGINES
AI Search Engines - Page 2
Comprehensive guides to AI-powered search engines and how they discover, crawl, and surface your content.
Consensus: AI Search for Research Papers and Sources
Consensus searches over 200 million research papers from Semantic Scholar, OpenAlex and its own crawl. How its search and ranking work.
2 min read
Coveo: Enterprise Search Software and Coveobot Crawler
Coveo is search software that companies run on their own sites. How its crawler Coveobot handles robots.txt, noindex and crawl rate.
3 min read
DeepSeek: Chat Web Search, Search API and Fetch API
DeepSeek runs its own search engine for its chat app and its Search and Fetch APIs. What is documented, and what site owners can control.
3 min read
DuckDuckGo AI Chat (Duck.ai) and Search Assist
Duck.ai gives anonymous access to third-party AI models. How Search Assist answers cite pages, and which DuckDuckGo crawlers to allow.
3 min read
Elasticsearch: Self-Hosted Search and the Elastic Crawler
Elasticsearch is search software you run yourself or on Elastic Cloud. Its semantic search options and the Elastic Open Crawler user agent.
3 min read
Elicit: AI Research Assistant and Its Paper Sources
Elicit searches about 138 million papers from Semantic Scholar, OpenAlex and PubMed. Its sources, search method and web access.
3 min read
Exa AI: Search API, ExaSearchBot Crawler and Opt-Out
Exa is a web search API for AI apps. Its crawler ExaSearchBot follows robots.txt and signs requests. User agents, opt-out and removal.
3 min read
Felo AI: Multilingual AI Search and Web Access
Felo is a multilingual AI search engine from Felo Inc. It answers from real-time web results with citations. No crawler is documented.
2 min read