TL;DR — Questions whether scaling LLMs without regard for cost, data quality, and societal risk produces benefits that outweigh the harms.
Abstract
The authors question whether the trend toward ever larger language models in NLP is actually beneficial, examining environmental costs, financial cost, data issues, and risk of harm.
Cite this paper
BibTeX
APA
MLA
Chicago
@inproceedings{bender2021on, title = {On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?}, author = {Emily M. Bender and Timnit Gebru and Angelina McMillan-Major and Shmargaret Shmitchell}, year = {2021}, booktitle = {FAccT '21}, doi = {10.1145/3442188.3445922}
}
Related papers
1
Model Cards for Model Reporting
Mitchell et al.20191,823 citations
2
Datasheets for Datasets
Gebru et al.20211,456 citations
3
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
Buolamwini & Gebru20182,103 citations
From Bluesky
DK
Dave Krishnamurthy
@dave.science.bsky.social
"On the Dangers of Stochastic Parrots" keeps coming up in every LLM discussion for a reason. Essential reading if you're thinking about responsible AI development.