Stochastic parrot - Wikipedia

Jump to content

Search

Search

Donate

Create account

Log in

Personal tools

Donate

Create account

Log in

Stochastic parrot

14 languages

العربية<br>Català<br>Deutsch<br>Euskara<br>Français<br>Galego<br>Italiano<br>日本語<br>한국어<br>Polski<br>Português<br>Русский<br>Tiếng Việt<br>中文

Edit links

From Wikipedia, the free encyclopedia

Term used in machine learning

In machine learning, the term stochastic parrot is a metaphor that frames large language models as systems that statistically mimic text without real understanding. The word "stochastic" – from the ancient Greek "στοχαστικός" (stokhastikos, 'based on guesswork') – is a term from probability theory meaning "randomly determined".[1] The word "parrot" refers to parrots' ability to mimic human speech.[1]

The term was introduced in a 2021 paper on AI ethics titled "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" that was authored by Timnit Gebru, Emily M. Bender, Angelina McMillan-Major, and Margaret Mitchell.[a] The paper outlined possible risks associated with large language models (LLMs). In December 2020, it was the subject of a workplace dispute between Gebru (then co-leader of Google's Ethical Artificial Intelligence Team) and Google, which had requested the retraction of the paper. The incident culminated in Gebru's controversial departure from the company.

The paper was later presented at the 2021 ACM Conference, and the term "stochastic parrot" has seen widespread use in academic research concerning generative AI and LLMs. The term has been interpreted negatively as an insult towards AI.[1]

Background<br>[edit]

Timnit Gebru is an AI ethics researcher,[2] Emily M. Bender is a linguist specializing in computational linguistics, and Margaret Mitchell is a computer scientist specializing in algorithmic bias. Gebru had joined Google in 2018, where she co-led a team on the ethics of artificial intelligence with Mitchell.

In late 2020, the paper "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜" was co-written by Gebru and five other researchers, four of whom were Google employees. The paper argues that large language models (LLMs) present significant risks such as environmental and financial costs, inscrutability leading to unknown dangerous biases, and potential for deception as LLMs do not understand the concepts underlying what they learn.[3]

The paper states that LLMs are "stitching together sequences of linguistic forms ... observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning." Therefore, they are labeled "stochastic parrots".[4]

Dismissal of Gebru by Google<br>[edit]

Further information: Timnit_Gebru § Exit_from_Google

After the paper was submitted for consideration to the 2021 ACM Conference, Google requested that Gebru either retract the paper from the conference or remove the names of Google employees from it.[5] Gebru refused to do so without further discussion, and emailed Google Research vice president Megan Kacholia that if the company could not explain the request for retraction and address other concerns regarding similar projects, she would plan to resign after a transition period, stating that they could "work on a last date".[6] The following day, on December 2, 2020, Gebru received an email saying that Google was "accepting her resignation".[3] Her abrupt firing sparked protests by Google employees and negative publicity for the company.[7]

Usage<br>[edit]

The phrase has been used by AI skeptics to signify that LLMs lack understanding of the meaning of their outputs.[1]

Sam Altman, CEO of OpenAI, used the term shortly after the release of ChatGPT in December 2022, tweeting "i am a stochastic parrot, and so r u".[1] The term was nominated as the 2023 AI-related Word of the Year by the American Dialect Society.[8][9]

Debate<br>[edit]

Some LLMs, such as ChatGPT, have become capable of interacting with users in convincingly human-like conversations.[10] The development of these new systems has deepened the discussion of the extent to which LLMs understand or are simply "parroting".

According to machine learning researchers Lindholm, Wahlström, Lindsten, and Schön, the term "stochastic parrot" highlights two vital limitations of LLMs:[11][12]

LLMs are limited by the data they are trained on and are simply stochastically repeating contents of datasets.

Because they are just making up outputs based on training data, LLMs do not understand if they are saying something incorrect or inappropriate.

Lindholm et al. noted that, with poor quality datasets and other limitations, a learning machine might produce results that are "dangerously wrong".[11]

Subjective experience<br>[edit]

In the mind of a human being, words and language correspond to things one has experienced.[13] For LLMs, according to proponents of the theory, words correspond only to other words and patterns of usage fed into their training...