OPEN DIALOGUE
Health Care A.I. Needs to Get Real
Unlocking the medical potential of artificial intelligence requires being more realistic about its limitations
--
This is Open Dialogue, an interview series from OneZero about technology and ethics.
I’m excited to talk with Muhammad Aurangzeb Ahmad. Muhammad is the principal research scientist at KenSci, inc., a company specializing in A.I. in health care, and an affiliate professor in the department of computer science at the University of Washington Bothell. I’ve known Muhammad for a long time. When I started teaching philosophy at Rochester Institute of Technology, he was one of my first students. Over the years, we’ve kept in touch as Muhammad went on to get his PhD in computer science and eventually became a notable data scientist and A.I. researcher. Muhammad's interests in philosophy have never wavered. I often turn to him when I need a holistic explanation of technological trends and controversies.
Our conversation has been edited and condensed for clarity.
Evan Selinger: With so much pain occurring during the pandemic, including massive financial challenges, adaptive responses were necessary. Many depended on technological innovation, from online communication platforms facilitating remote teaching and working to mobile robots keeping places clean. Despite high hopes for medical A.I., Stanford Medicine drew negative headlines. It selected software to determine which workers should receive priority for the Pfizer vaccine during the first distribution wave, and the algorithms created poor results. It favored lower-risk doctors over medical residents who worked in close physical proximity to Covid-19 patients. Why did the system choose the wrong people?
Muhammad Ahmad: The press initially misdescribed the software as a machine-learning system. It was, however, a simple rule-based system. In other words, human programmers gave the software rules to follow. These rules created a protocol for who should get vaccinated first. Here’s the interesting thing. If one looks at each rule individually, they’re all sensible. None of them express anything objectionable. However, when the rules are combined, they turn out to be more than the sum of their…
