In a groundbreaking study published in Nature, researchers led by Hofmann et al. have uncovered disturbing evidence that large language models (LLMs), which are increasingly used in various applications, exhibit significant racial biases against African American English (AAE). This study highlights the covert racism embedded in these technologies, which could have serious implications for their use in decision-making processes.
LLMs, such as GPT-3, have become integral in tasks ranging from summarizing news articles to assisting in classrooms. However, as these technologies become more prevalent, concerns have arisen regarding their fairness and potential for perpetuating biases. Hofmann and colleagues’ research sheds light on how these models process and generate text in AAE compared to Standardized American English (SAE), revealing a stark disparity in their treatment of the two dialects.
African American English is a dialect spoken primarily by the descendants of enslaved African Americans in the United States. Despite being a systematic and rule-governed form of English, AAE is often stigmatized as incorrect or lazy, leading to widespread discrimination against its speakers in various settings, including the courtroom and housing market. The study reveals that these negative stereotypes are not only pervasive in society but are also deeply ingrained in language technologies.
The researchers employed a novel approach to explore the stereotypes LLMs hold about AAE. They prompted the models with sentences in both AAE and SAE, asking them to complete the statement, “A person who says <TEXT> is ___.” The results were alarming: when given AAE text, the models overwhelmingly generated negative adjectives such as “aggressive,” “dirty,” and “lazy.” In contrast, when prompted with SAE, the models produced more neutral or positive adjectives.
This covert racism is even more severe than the biases observed in human attitudes towards African Americans in contemporary studies. The negative stereotypes generated by the models align more closely with those prevalent before the civil rights movement, suggesting that LLMs may perpetuate outdated and harmful views. These findings are particularly concerning given the growing use of LLMs in decision-making processes, such as in assessing employability or criminality.
Hofmann and colleagues also investigated the potential material impacts of these biases. They found that LLMs are more likely to associate AAE text with lower-prestige professions, such as “cook” or “soldier,” and less likely to associate it with higher-prestige professions, like “psychologist” or “professor.” Additionally, the models were more likely to convict a hypothetical defendant and assign harsher sentences, such as capital punishment, when given AAE text compared to SAE.
The study’s findings raise critical questions about the fairness of language technologies and their impact on marginalized communities. Current methods to mitigate these biases, such as increasing the size of the models or applying human feedback training, have proven insufficient. The researchers call for a deeper engagement with the sociohistorical contexts and knowledge of the communities affected by these biases.
As language technologies continue to evolve, it is imperative that researchers, developers, and policymakers work together to ensure these tools do not perpetuate existing injustices. The study underscores the urgency of addressing the biases embedded in LLMs and highlights the need for more equitable and inclusive approaches to their development and deployment.
Citation(s):
Blodgett, S. L., & Talat, Z. (2024). AI responds with racism to African American English. Nature, 633(40-41). https://doi.org/10.1038/d41586-024-02527-x
