# GPT-NeoX: EleutherAI Open Model and Training Data

> GPT-NeoX is the EleutherAI training library and 20B open model. It has no web access or crawler. Its training data, the Pile, includes Common Crawl text.

GPT-NeoX is an open-source library from EleutherAI for training large language models. EleutherAI also used it to train GPT-NeoX-20B, an open model. Neither one browses the web or sends requests to your site, so there is nothing to allow or block for GPT-NeoX itself.

## What GPT-NeoX is

The [GPT-NeoX repository](https://github.com/EleutherAI/gpt-neox) describes the project as the EleutherAI library for training large-scale language models on GPUs. The code is under the Apache License 2.0. The README says that more than 40 organizations used it to train more than 50 models. Two examples are Pythia from EleutherAI and [StableLM](/ai-chat-bot/stablelm/) from Stability AI.

[GPT-NeoX-20B](https://huggingface.co/EleutherAI/gpt-neox-20b) is a language model with 20 billion parameters that EleutherAI trained with this library. Its model card says that the model is mainly for research. It is not a product and is not fine-tuned for chat. The card says that it will likely not respond to a prompt the way ChatGPT does. The [GPT-NeoX-20B paper](https://arxiv.org/abs/2204.06745) gives the technical details.

## No web access

GPT-NeoX-20B is a set of model weights. It has no browser, no search tool and no crawler. It works only with the text that it learned in training and the text that a person gives it.

An app built on this model can read web pages. In that case, the app sends the requests, with its own user agent. A user agent is the client name in the User-Agent HTTP header. Your logs show the name of the app, not GPT-NeoX.

## Training data and Common Crawl

EleutherAI trained GPT-NeoX-20B on [the Pile](https://arxiv.org/abs/2101.00027). The Pile is an 825 GiB English text dataset that EleutherAI published in December 2020. One part of it, Pile-CC, is a filtered subset of Common Crawl. Common Crawl is a public archive of web pages that its crawler, [CCBot](/ai-crawler-bot/ccbot/), collects.

A robots.txt rule that you add today does not change the Pile or the models that were trained on it. If you do not want future Common Crawl archives to include your site, the [CCBot guide](/ai-crawler-bot/ccbot/) shows the robots.txt rule.

---

[View the canonical page](https://aichatwatch.com/ai-chat-bot/gpt-neox/) · [Browse llms.txt](https://aichatwatch.com/llms.txt)