Objective Algorithms Are a Myth
Shalini Kantayya on her new documentary, ‘Coded Bias,’ and the importance of breaking open the black box of algorithm design
--
The protests across the U.S. and around the globe in the wake of the murder of George Floyd have raised awareness about structural inequalities. Though the specific focus has been on police brutality, scholars, activists, and artists are sounding the alarm on how systemic racism has been amplified in other areas like the tech industry, through communication and surveillance technology.
In Coded Bias, a documentary by Shalini Kantayya, the director follows MIT Media Lab researcher and Algorithmic Justice League founder Joy Buolamwini as she discovers one of the fundamental problems with facial recognition. While working on a facial recognition art project, Buolamwini realizes that the computer vision software was having trouble tracking her face, but it worked fine when she put on a white mask. It was just the latest evidence of the type of bias that’s baked into facial recognition and A.I. systems
Along with Buolamwini, Kantayya interviews authors, researchers, and activists like Cathy O’Neil, Meredith Broussard, Safiya Noble, and Silkie Carlo, unraveling the problems of current technology like facial recognition or crime prediction software. These technologies often connect back to the dark historical practices of racialized surveillance, eugenics, or physiognomy.
The film, which was screened at Sundance, focuses a critical eye on the assumed “objectivity” of algorithms, which O’Neil defines as “using historical information to make a prediction about the future.” While algorithms are often understood as unbiased, objective, and amoral, they can reproduce the biases of the humans that create them. Broussard says that we imbue technology with “magical thinking,” which lauds its benefits but obscures its negative effects.
Coded Bias explains how algorithmic bias can have negative effects on the real…
