# Amazon Q: Web Crawler, robots.txt and Site Access > What Amazon Q Developer and Amazon Q Business do with websites, and how to block the Amazon Q web crawler with the amazon-QBusiness token. Amazon Q is the name of two AWS assistants: Amazon Q Developer for software work and Amazon Q Business for company data. AWS does not document a public web search feature for either one. A site owner meets Amazon Q mostly through the Amazon Q Business Web Crawler, which obeys robots.txt rules for the `amazon-QBusiness` token. ## Amazon Q Developer AWS [describes Amazon Q Developer](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/what-is.html) as a conversational assistant that helps you build and operate AWS applications. In an IDE (a code editor), it gives inline code suggestions, chat about code, code generation and security scans. It runs on Amazon Bedrock. AWS will [end support for the Amazon Q Developer IDE plugins](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/q-developer-ide-end-of-support.html) on April 30, 2027. AWS points IDE users to Kiro, which offers agentic coding, chat and MCP support. AWS does not document a crawler or a user agent that Amazon Q Developer uses to visit public websites. Its suggestions can be similar to public open-source code. In that case, the [code reference log](https://docs.aws.amazon.com/amazonq/latest/qdeveloper-ug/code-reference.html) shows information about the source. ## Amazon CodeWhisperer Amazon CodeWhisperer was an AWS assistant that suggested code in a code editor. It became part of Amazon Q Developer on April 30, 2024, as the [AWS Toolkit for JetBrains documentation](https://docs.aws.amazon.com/toolkit-for-jetbrains/latest/userguide/amazonq.html) states. The change includes inline code suggestions and security scans. The [CodeWhisperer user guide](https://docs.aws.amazon.com/codewhisperer/latest/userguide/whisper-legacy.html) says that customers can migrate in place and keep their subscriptions, customizations and tags. The old product page at `aws.amazon.com/codewhisperer/` now redirects to the Amazon Q Developer page. AWS documents no crawler, user agent or robots.txt token for CodeWhisperer. ## Amazon Q Business and its web crawler [Amazon Q Business](https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/what-is.html) answers questions from a company's data sources and gives citations to those sources. AWS says that it is [no longer open to new customers](https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/qbusiness-availability-change.html). Existing customers still get bug fixes and security updates, and AWS recommends Amazon Quick for new projects. One of its data sources is the [Web Crawler connector](https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/connector-webcrawler.html). A customer gives it website addresses, and it crawls and indexes those sites. AWS tells customers to crawl only their own pages or pages that they have permission to index. ## Block the Amazon Q Business crawler robots.txt is a text file at the site root with rules for crawlers. AWS says that the Amazon Q Business Web Crawler [obeys the Allow and Disallow rules](https://docs.aws.amazon.com/amazonq/latest/qbusiness-ug/stop-web-crawler.html) in robots.txt. If you want to stop it on your whole site, add this group: ``` User-agent: amazon-QBusiness Disallow: / ``` If you want to keep it out of one folder only, disallow that path: ``` User-agent: amazon-QBusiness Disallow: /credential-pages/ ``` ## Amazon Quick web crawler The [Amazon Quick Web Crawler](https://docs.aws.amazon.com/quick/latest/userguide/web-crawler-integration.html) identifies itself with the user-agent string `amazon-Quick-on-behalf-of- `. AWS says that it obeys robots.txt user-agent, allow and disallow rules. If it cannot get your robots.txt file, it crawls as if the file does not exist. ## Other Amazon bots Amazonbot, Amzn-SearchBot and Amzn-User are crawlers for Amazon products and services such as Alexa. The [Amazonbot page](https://developer.amazon.com/amazonbot) does not connect them to Amazon Q. The [Amazonbot guide](/ai-crawler-bot/amazonbot/) explains how to control them. --- [View the canonical page](https://aichatwatch.com/ai-chat-bot/amazon-q/) ยท [Browse llms.txt](https://aichatwatch.com/llms.txt)