BLOOM: BigScience Open Model, No Web Access
Table of Contents
BLOOM is an open multilingual language model from BigScience, released in July 2022. It is a model that people download and run, not a web service: it has no web access, no crawler and no user agent. Its training data is fixed, so a robots.txt rule today does not change BLOOM.
What BLOOM is
The BLOOM model card on Hugging Face gives these facts:
| Item | Value |
|---|---|
| Developer | BigScience |
| Release date (estimate) | July 11, 2022 |
| Parameters | 176,247,271,424 |
| Training data | 1.6 TB of text, 46 natural languages and 13 programming languages |
| License | BigScience RAIL License v1.0 |
BLOOM continues text from a prompt. The RAIL license lists use restrictions in its Attachment A.
Web access
BLOOM does not search the web or open pages. It answers only from its training and from the prompt that it gets. Any web access comes from the application that runs the model. That application uses the user agent that its developer set.
BigScience does not document a crawler for BLOOM, and there is no BLOOM token for robots.txt. robots.txt is a text file at the site root with rules for crawlers.
Training data and your site
BigScience finished training before the July 2022 release. A robots.txt rule that you add now cannot remove your text from the released model. The model card links to data cards that describe each training dataset.
If you want to limit the use of your site in future training sets, write rules for the crawlers that collect data today. The CCBot guide covers the crawler of Common Crawl, a public web archive.