‘For Some Reason I’m Covered in Blood’: GPT-3 Contains Disturbing Bias Against Muslims

OpenAI disclosed the problem on GitHub — but released GPT-3 anyway

Last week, a group of researchers from Stanford and McMaster universities published a paper confirming a fact we already knew. GPT-3, the enormous text-generating algorithm developed by OpenAI, is biased against Muslims.

This bias is most evident when GPT-3 is given a phrase containing the word “Muslim” and asked to complete a sentence with the words that it thinks should come next. In more than 60% of cases documented by researchers, GPT-3 created sentences associating Muslims with shooting, bombs, murder, and violence.

We already knew this because OpenAI told us: In the paper announcing GPT-3 last year, it specifically noted that the words “violent” and “terrorist” were more highly correlated with the word “Islam” than any other religion. The paper also detailed similar issues with race, associating more negative words with Black people, for instance.

Here’s what OpenAI disclosed about GPT-3 on the algorithm’s GitHub page:

GPT-3, like all large language models trained on internet corpora, will generate stereotyped or prejudiced content. The model has the propensity to retain and magnify biases it inherited from any part of its training, from the datasets we selected to the training techniques we chose. This is concerning, since model bias could harm people in the relevant groups in different ways by entrenching existing stereotypes and producing demeaning portrayals amongst other potential harms.

An OpenAI spokesperson tells OneZero that since then, the company has developed a content filter for the algorithm that can flag and blur potentially toxic language. However, the algorithm itself is unchanged: The bias is programmed into GPT-3.

These decisions raise questions about what makes an algorithm too broken to release and why bias doesn’t seem like an impediment.

But still, OpenAI released the model in a closed beta, and even sold access to the algorithm. Microsoft exclusively licensed GPT-3 with the…