A.I. Doctors Have a Trust Problem

Ethicists argue that A.I.-based medical services need to be evaluated and regulated in the same way as new drugs

Credit: Hero Images/Getty

Imagine you’re a 59-year old man, and you go to your doctor with chest pains. The doctor thinks it might be a heart attack and orders further tests. Now, imagine you’re a 59-year old woman with the same symptoms. The doctor tells you that you’re probably having a panic attack.

These strikingly different suggestions, however, didn’t come from a doctor, but from a popular health care app called GP at Hand, which uses artificial intelligence to tell you what might be wrong with you based on your symptoms. Babylon Health, which makes the app, is careful not to use the word “diagnose,” describing the app instead as a triage tool. The company batted away concerns about sexism by arguing that it bases its suggestions on “epidemiological data from a huge number of research studies.” Because women are much less likely to suffer heart attacks than men but twice as likely to suffer from anxiety disorders, it argued, the app’s suggestion was correct.

Nonetheless, the story raises difficult questions about how A.I. should be used in health care. The promise of A.I. is that, by analyzing large quantities of data — from patient health care records, laboratory results, scan images, research studies, and DNA databases — it will be possible to create algorithms that make diagnoses and treatment recommendations that are as good, or even better, than those of a human doctor.

“The hope is that if we bring in artificial intelligence, we could make rules that are more objective, and we don’t have to rely on a clinician who hasn’t slept in 24 hours and is just running off caffeine.”

In one example, Moorfields Eye Hospital in London has worked with Google-owned company DeepMind Health to develop an A.I. algorithm trained on thousands of anonymized eye scans. Its 94% accuracy in making correct referral decisions matches that of world-leading eye experts. As Irene Chen, a researcher in computer science at MIT, puts it: “The hope is that if we bring in artificial…