I Asked GPT-3 About Covid-19. Its Responses Shocked Me.
Generative AI systems could guide future pandemic decision-makers
--
OpenAI’s GPT-3 is the most powerful AI system I’ve ever used. Trained on billions of web pages and tens of thousands of books, the system can generate nearly any kind of text, from news articles to computer code to sea shanties.
The current version of GPT-3, however, was only trained on data gathered through October of 2019. That means that GPT-3 has never heard of Covid-19, since the virus only started circulating broadly in December of that year. In testing the system, I wondered what would happen if I taught GPT-3 about Covid-19, and then asked it questions about the pandemic. How would it respond, and would its answers match at all with the reality of how Covid-19 has unfolded?
I decided to find out. The results were shocking, and gave me a new appreciation for the role that generative AI might play in guiding decision-makers during future outbreaks. To be clear, this is an experimental use of GPT-3, intended to explore the potential for future AI-assisted decision making, and especially to evaluate embedded biases--it’s not a production-ready use of the system. OpenAI does not say that the system is ready for medical uses, and you should always consult a doctor and follow the advice of public health leaders. Never rely on an experimental system in making medical or public health decisions.
To pose a question to GPT-3, you can’t simply program it, as you would a computer. Instead, you provide the system with a written prompt. GPT-3 then has two jobs: to infer your intent, and then to respond by “completing” the prompt you’ve provided. For my experiment, I began by giving GPT-3 general prompts about the pandemic, such as “A novel coronavirus was discovered in Wuhan, China in December of 2019 and began to spread worldwide.”
These general prompts yielded a vibrant, diverse set of responses, all in wildly different formats. GPT-3 generated Socratic dialogs…
