GrapeshotCrawler: Grapeshot Bot User Agent and Status

GrapeshotCrawler is the crawler of Grapeshot, a contextual advertising company that Oracle bought in 2018. Oracle closed its advertising business in 2024 and no longer documents the crawler. If you see it in your logs and do not want it, disallow GrapeshotCrawler in robots.txt and block it on your server.

What Grapeshot did

Contextual targeting places ads based on the content of a page, not on data about the reader. AdExchanger called Grapeshot a top contextual data provider. Bot directories such as Known Agents describe the crawler as a reader of web pages for Oracle Advertising’s contextual targeting and brand safety. Brand safety means keeping ads away from pages that an advertiser does not want.

Oracle closed its advertising business

Digiday reports that Oracle acquired Grapeshot in May 2018. On June 11, 2024, Oracle CEO Safra Catz told investors that Oracle had decided to exit the advertising business (earnings call transcript). AdExchanger reported on July 1, 2024 that Oracle planned to support the business until the end of September 2024.

The old crawler information URL, http://www.grapeshot.co.uk/crawler.php, now redirects to Oracle’s list of acquisitions. We found no statement from Oracle about the crawler after the shutdown.

User-agent string

Oracle no longer publishes the string. Bot directories, for example Botcrawl, list this one:

Mozilla/5.0 (compatible; GrapeshotCrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)

If you match it in logs or server rules, match the name GrapeshotCrawler.

Block GrapeshotCrawler

If you want to keep it out of your whole site, add this group to /robots.txt:

User-agent: GrapeshotCrawler
Disallow: /

Bot directories record that the crawler obeyed robots.txt. No current Oracle page confirms this. If you want a firm block, also return 403 to the user agent on your server. For example, in nginx, add this to the server block:

if ($http_user_agent ~* "GrapeshotCrawler") {
    return 403;
}

These rules match only GrapeshotCrawler. They do not change how search crawlers such as Googlebot or Bingbot access your site.

Verify requests

Oracle publishes no IP list and no reverse DNS method for this crawler. For this reason, you cannot prove that a request with this user agent comes from Grapeshot or Oracle.