User:Stellaathena/sandbox1

EleutherAI is an artificial intelligence research laboratory founded in July 2020. EleutherAI made headlines in 2021 after its GPT-3 replication project produced the most powerful autoregressive language model freely available online.

History
EleutherAI

While EleutherAI initially turned down funding offers, preferring to use Google's TRC program to source their compute, by early 2021 they had accepted funding from CoreWeave (a small cloud computing company) and SpellML (a cloud infrastructure company) in the form of access to powerful GPU clusters that are necessary for large scale machine learning research.

Research and technologies
According to their website, EleutherAI is a "decentralized grassroots collective of volunteer researchers, engineers, and developers focused on AI alignment, scaling, and open source AI research". While they do not sell any of their technologies as products, they publish the results of their research in academic venues, write blog posts detailing their ideas and methodologies, and provide trained models for anyone to use for free.

Massive Language Models
EleutherAI is best known for its pioneering work on developing and publicly releasing large language models. In contrast to other leading labs such as Google and OpenAI, EleutherAI t

As of November 2021, EleutherAI

Actual Cites
- EleutherAI was created with the goal of open sourcing GPT-3 - The Pile was created because no suitable training data existed - EleutherAI uses compute from TRC - The models are freely available on HF - "GPT-Neo was able to generate a coherent, almost-believable article without missing out on the central themes"

- Connor, Leo, and Sid founded EleutherAI - GPT-Neo release

- The Pile was used by MSFT - The Eval Harness was used by MSFT

- "We think that access to large, pretrained models will enable large swathes of research that would not have been possible while such technologies are locked away behind corporate walls. For-profit entities have explicit incentives to downplay risks and discourage security probing. We want to help the wider safety and security communities access and study these new technologies" - The EleutherAI community has now expanded its activity and is working on open-source alternatives in BioML and generative art.

- The EleutherAI language models hosted on Hugging Face has been used ~300,000 times each month.

Biological ML
Following [DeepMind]]'s breakthrough application of transformers to protein folding, EleutherAI began to expand into the space. They have a stated public goal of replicating and releasing the (unreleased) AlphaFold2 model developed by DeepMind. Enroute to that goal, EleutherAI has released several technologies and papers