How to Avert a Digital Dystopia
Why we need to start challenging the prevailing conversations around tech
“What I find [ominous] is how seldom, today, we see the phrase ‘the 22nd century.’ Almost never. Compare this with the frequency with which the 21st century was evoked in popular culture during, say, the 1920s.”
—William Gibson, famed science-fiction author, in an interview on dystopian fiction.
The 2010s are almost over. And it doesn’t quite feel right.
When the end of 2009 came into view, the end of the 2000s felt like a relatively innocuous milestone. The current moment feels so much more, what’s the word?
Ah, yes: dystopian.
Looking back, “dystopia” might have been the watchword of the 2010s. Black Mirror debuted close to the beginning of the decade, and early in its run, it was sometimes critiqued for how over-the-top it all felt. Now, at the end of the decade, it’s regularly critiqued as made obsolete by reality.
And it’s not just prestige TV like Black Mirror reflecting the decade’s mood of incipient collapse. Of the 2010s top 10 highest-grossing films, by my count at least half involve an apocalypse either narrowly averted or, in fact, taking place (I’m looking at you, Avengers movies).
People have reasons to wallow. I get it. The existential threat of climate change alone — and seeing efforts to mitigate it slow down precisely as it becomes more pressing — could fuel whole libraries of dystopian fiction.
Meanwhile, our current tech landscape — the monopolies, the wild spread of disinformation, the sense that your most private data could go public whenever, with no recourse, all the things that risk making Black Mirror feel quaint — truly feels dystopian.
We enjoy watching distant, imaginary dystopias because they distract us from oncoming, real dystopias.
Since no one in a position to actually do something about our dystopian reality seems to be admitting it — no business leaders, politicians or legacy media — it makes sense that you might get catharsis of acknowledgment from pop culture instead. And yet, the most popular end-of-the-world fiction isn’t about actual imminent threats from climate or tech. It’s about Thanos coming to snap half of life out of existence. Or Voldemort threatening to destroy us Muggles.
Maybe that kind of pop culture, which acknowledges dystopia but not the actual threats we currently face, gives us a feeling of control: Sure, Equifax could leak my social security number and face zero consequences, but there are no Hunger Games. Wow — it really could be so much worse! Maybe we enjoy watching distant, imaginary dystopias because they distract us from oncoming, real dystopias.
But let’s look at those actual potential dystopias for a moment and think about what we need to do to avert them.
I’d suggest the big four U.S. tech giants — Amazon, Facebook, Apple, Google — each have a distinct possible dystopia associated with them. If we don’t turn around our current reality, we will likely get all four — after all, for all the antagonistic rhetoric among the giants, they are rather co-dependent. Let’s look at what we might have, ahem, look forward to — unless we demand the tech giants deliver on the utopia they purportedly set out to achieve when their respective founders raised their rounds of millions. I would argue not only that we can, but that we must hold them accountable.
“Mad Max,” or, slowly then all at once: starring Apple
“‘How did you go bankrupt?’ Bill asked. ‘Two ways,’ Mike said. ‘Gradually and then suddenly.’”
—Ernest Hemingway, The Sun Also Rises.
When you think of Mad Max, you probably think of an irradiated, post-apocalyptic desert hellscape. You’re also not thinking of Mad Max.
In the original 1979 film, the apocalypse hasn’t quite yet happened. There’s been a substantial social breakdown, but things are getting worse in slow motion. There are still functioning towns. Our protagonist, Max, is a working-class cop; and while there’s reason to believe a big crash is coming, or has even begun, society is still hanging on. (It’s only in the sequels that we’re well into the post-apocalyptic landscape people are thinking of when they say “Mad Max.”)
A relatively subtle dystopia, where things gradually decline in the background, is also a good day-to-day description of a society overrun by algorithms, even without the attention-grabbing mega-scandals of a Cambridge Analytica or massive data breach. A kind of dystopia “light” — and Apple is its poster child.
After all, Apple has a genuinely better track record than some of the other tech giants on a few key privacy issues. But it’s also genuinely aware of the value of promulgating that vision of itself — and that can lead Apple users into danger.
In January, Apple purchased a multistory billboard outside the Consumer Electronics Show in Las Vegas, with this message: “What happens on your iPhone, stays on your iPhone.” Sounds great — but it’s deeply misleading, and as journalist Mark Wilson noted, Apple’s mismatch between rhetoric and behavior fuels the nightmare that is our current data security crisis:
“[iPhone] contents are encrypted by default […] But that doesn’t stop the 2 million or so apps in the App Store from spying on iPhone users and selling details of their private lives. “Tens of millions of people have data taken from them — and they don’t have the slightest clue,” says [the] founder of [the] cybersecurity firm Guardian […] The Wall Street Journal studied 70 iOS apps […] and found several that were delivering deeply private information, including heart rate and fertility data, to Facebook.” [Emphasis mine.]
A tech giant that is claiming it’s the path to salvation, while effectively creating a trap for those who believe it, sounds ironically familiar given Apple’s famous evocation of Big Brother.
After all, when people talk about habit-forming technology in terms so terrifying they’ve convinced Silicon Valley executives to limit their children’s access to their own products, let’s be real: They’re talking about iPhones.
When academic child psychology researcher Jean Twenge talks about a possible teenage mental health epidemic fueled by social media, we know what’s at the heart of it: She’s talking about iPhones.
All those aforementioned horror stories, and a huge slice of those algorithms you’ve heard so much about, are likely first reaching you on smartphones that, with world market share above 50%, are largely, you guessed it, iPhones. (And none of these stories even mention Apple workers at overseas at facilities like Foxconn who create our iPhones and who really are living in a kind of explicit dystopia.)
What happens on your iPhone almost certainly doesn’t stay on your iPhone. But who created that surveillance capitalism running it all in the first place?
“Black Mirror:” “Nosedive,” or, welcome to surveillance capitalism: starring Google
“We know where you are. We know where you’ve been. We can more or less know what you’re thinking about.”
—Google’s then-CEO Eric Schmidt, in a 2011 interview.
You’ve probably heard it before: “if you’re not paying, you’re the product.” This is usually in reference to ostensibly “free” services like Facebook or Gmail. It’s a creepy thought. And, according to Shoshana Zuboff, professor emeritus at Harvard and economic analyst of what she’s termed “surveillance capitalism,” the selling of your personal information undermines autonomy. It’s worse than you being the product: “You are not the product. You are the abandoned carcass.”
Google, according to Zuboff, is the original inventor of Surveillance Capitalism. In their early “Don’t Be Evil” days, the idea of accessing people’s private Google searches and selling them was considered unthinkable. Then Google realized it could use search data for targeting purposes — and never stopped creating opportunities to surveil their users:
“Google’s new methods were prized for their ability to find data that users had opted to keep private and to infer extensive personal information that users did not provide. These operations were designed to bypass user awareness. […]In other words, from the very start Google’s breakthrough depended upon a one-way mirror: surveillance.”
Twenty years later, surveillance capitalism has become so ubiquitous that it’s hard to live in Western society without being surveilled constantly by private actors.
As far as I know, no mass popular culture has really yet captured this reality, but one small metaphor that kind of hits on its effects is a Black Mirror episode called “Nosedive.”
In “Nosedive,” everyday people’s lived experience is very clearly the picked-apart carcass for an entire economic and social order; a kind of surveillance-driven social credit score affects every aspect of your daily life, from customer service to government resources to friendships, all based on your app usage and, most creepily, how other people rate you in the app.
If surveillance capitalism has been the engine powering our economy in the background for nearly two decades, it’s now having a coming-out party. Increasingly, Google isn’t just surveilling us in private — with its “designing smart cities” initiatives, the company will literally be making city management decisions instead of citizens: Sidewalk Labs, a Google sister company, plans to develop “the most innovative district in the entire world” in the Quayside neighborhood of Toronto, and Google itself is planning on siphoning every bit of data about how Quayside residents live and breathe and move via ubiquitous monitoring sensors that will likely inform — for a fee naturally — how other cities will develop.
If surveillance capitalism has been the engine powering our economy in the background for nearly two decades, it’s now having its coming-out party.
Much like Apple, Google takes pains to present itself as a conscientious corporate citizen. They might be paternalistic, or antidemocratic — but they have learned it’s important to their brand that they’re seen as responsive to their workers and the broader public, largely thanks to the courageous and persistent effort of their workers and consumer advocates in civil society.
Not so much with Amazon.
“Elysium,” or, dystopia for some, Prime Day for others: starring Amazon
“[The New York Times] claims that our intentional approach is to create a soulless, dystopian workplace where no fun is had and no laughter heard. Again, I don’t recognize this Amazon and I very much hope you don’t either.” —Jeff Bezos, August 17, 2015 letter to staff after the New York Times investigation into working conditions at the company.
In 2015, Jeff Bezos felt the need to set the record straight: The New York Times was wrong about Amazon. Working there did not feel like a dystopia.
The years since have only validated the New York Times story, which focused on life for coders and executives at Amazon. Notably, when the Times and other investigative journalists have probed life for the far more numerous warehouse workers employed by Amazon, Bezos has largely stayed silent.
In fact, the further down the corporate ladder you get at Amazon, the more likely it seems that Jeff Bezos will stay quiet on any controversy. Just this month, in a report published almost exactly four years after Bezos’ “Amazon is not a dystopia” declaration, the New York Times has uncovered almost a dozen previously unreported deaths allegedly caused by Amazon’s decentralized delivery network. Rather than defend itself out loud, Amazon has kept quiet while repeating the same argument in the courts: Those delivery people aren’t Amazon workers at all, and thus Amazon is not liable.
Amazon, like every major tech giant, has a key role in the dystopia of surveillance capitalism — the monopolylike market share of Amazon Web Services, and Amazon’s involvement in increasingly ubiquitous facial recognition software, represent their own deeply dystopian trends. But the most visible dystopia Amazon creates, for all to see, is dystopia in the workplace.
In many ways, Amazon is the single company that best explains the appeal of an Andrew Yang figure to a certain slice of economically alienated young voters. When speaking near Amazon’s HQ in Seattle, Yang explicitly talked about the surveillance of Amazon workers, and how reliable those jobs are in any case:
“All the Amazon employees [here] are like, ‘Oh shit, is Jeff watching me right now?’… [Amazon will] open up a fulfillment warehouse that employs, let’s call it 20,000 people. How many retail workers worked at the malls that went out of business because of Amazon? [The] greatest thing would be if Jeff Bezos just stood up one day and said, ‘Hey, the truth is we are one of the primary organizations automating away millions of American jobs.’ […] I have friends who work at Amazon and they say point-blank that ‘we are told we are going to be trying to get rid of our own jobs.’”
You can flat-out disagree with Yang’s proposed solutions, but a lot of his appeal stems from the fact that he’s diagnosing a problem that broad swaths of people don’t feel is being talked about. Yang validates his supporters’ concerns that they are, in fact, living in a dystopia of the corporate overlord variety.
In the movie Elysium, most work is done in warehouses, under constant surveillance, with workers creating the very automation systems that surveil and punish them. The movie takes place in a company townlike setting, with no such thing as a class system or social mobility. Meanwhile, the ruling class in Elysium lives in space, having left everyone else behind to work on Earth, a planet now fully ravaged by climate change.
That might sound particularly far-fetched, but given Bezos’ explicit intention to colonize space because “we are in the process of destroying this planet,” it suddenly doesn’t feel so off the mark. And in an era where Governors and Mayors openly genuflect to Amazon, preemptively giving up vast swaths of democratic powers for the mere possibility that Amazon might host an office building there, it’s hard not to feel like we’re already in an Elysium-flavored dystopia.
Amazon has their dystopia picked out, flavor and all. But what happens when the biggest social network in the world can’t decide which dystopia it wants to be when it grows up?
Pick a dystopia — any dystopia!: starring Facebook
“Understanding who you serve is always a very important problem, and it only gets harder the more people that you serve.”
—Mark Zuckerberg, 2014 interview with the New York Times.
Ready Player One is one of the more popular recent dystopian novels.
The bleak future it depicts is relatively straightforward: In the face of economic and ecological collapse, the vast majority of human interaction and commercial activity happens over a shared virtual reality space called Oasis.
In Oasis, the downtrodden masses compete in enormous multiplayer video games, hoping to win enough prizes and gain sufficient corporate sponsorship to scrape out a decent existence. Imagine a version of The Matrix, where people choose to constantly log into unreality because actual reality has gotten so unbearably terrible, electing to let the real world waste away. Horrific.
Ready Player One is also the book that Oculus founder and former Facebook employee Palmer Luckey used to give new hires, working on virtual reality to get them “excited” about the “potential” of their work.
Sound beyond parody? In so many ways, Facebook is unique among the tech giants: It’s not hiding the specter of dystopia. It’s amplifying dystopia.
It’s hard to pick a popular dystopia Facebook isn’t invested in.
1984? Sure, Facebook says, quietly patenting technology that lets your phone record you without warning.
Brave New World? Lest we forget, Facebook literally experimented with making depression contagious in 2014.
28 Days Later, or any of the various other mass-violence-as-disease horror movies like The Happening? Facebook has been used to spread mass genocidal panics far more terrifying than any apocalyptic Hollywood film.
What about the seemingly way out there dystopias — something like THX-1138 or a particularly gnarly Black Mirror episode where a brain can have its thoughts directly read, or even electronically implanted? It won’t comfort you to know that Facebook just acquired CTRL-Labs, which is developing a wearable brain-computer interface, raising questions about literal thought rewriting, brain hacking, and psychological “discontinuity.”
Roger McNamee, an early Zuckerberg advisor and arguably its most important early investor, has become unadorned about it: Facebook has become a dystopia. It’s up to the rest of us to catch up.
We spent the 2010s on dystopia—let’s spend the 2020s on utopia instead
“Plan for the worst, hope for the best, and maybe wind up somewhere in the middle.” —Bright Eyes, “Loose Leaves”
People generally seem to think dystopias are possible, but utopias are not. No one ridicules you for conceiving of a dystopia.
I think part of that is because it gives us an easy out. Dystopias paralyze us. They overwhelm. They make us feel small and powerless. Envisioning Dystopia is like getting married anticipating the divorce. All we can do is make sure it’s amicable.
Is there room for a utopian counterweight? There’s not only room, there’s an urgent need if we want to look forward (as opposed to despondently) to the 22nd century. We cannot avert or undo dystopias without believing in their counterparts.
But we need to make the utopian alternative feel real, accessible, and achievable. We need to be rooting not for the lesser of two evils, but for something actually good.
Dystopias — real, about-to-unfold dystopias — have been averted before. The threat of nuclear apocalypse during the Cold War. The shrinking hole in the ozone layer (which is both distinct from, and has lessons to teach us about, the climate crisis). We didn’t land in utopia, but it was only by hitching our wagons to a utopian vision that we averted the worst.
In 2017, cultural historian Jill Lepore penned a kind of goodbye letter to dystopian fiction, calling for a renewal of utopian imagination. “Dystopia,” she lamented, “used to be a fiction of resistance; it’s become a fiction of submission.” Dystopian narratives once served as stark warnings of what might be in store for us if we do nothing, spurring us on to devise a brighter future. Today, dystopian fiction is so prevalent and comes in so many unsavory flavors that our civic imaginations are understandably confined to identifying the one we deem most likely to inevitably happen, and to come to terms with it.
But we don’t have to.
A new decade is on the way. Let’s spend the 2020s exercising our utopian imaginations — the muscles we use to envision dystopia are now all too-well-developed, and a body that only exercises one set of muscles quickly grows off-balance.
Dystopias disempower. We are tiny, inconsequential — how could we do anything about them? Utopias, on the other hand, are rhetorical devices calling upon us to build. They invite our participation. Because a utopia where we don’t matter is a contradiction in terms.
Let’s envision a world where those creating algorithms are thinking not only about their reach, but also about their impact. A world in which we are not the carcass left behind by surveillance capitalism. A world in which calling for ethical norms and standards is in itself a utopian act.
Let’s spend the next decade fighting for what we actually want: A world in which the powerful few are held to a higher standard; an industry in which ethics aren’t an afterthought, and the phrase “unintended consequences” doesn’t absolve actors from the fall out of their very deliberate acts.
Let’s actualize the utopia which, ironically enough, the tech giants themselves so enthusiastically promised us when they set out to change the world.
Let’s spend this next decade asking for what we actually want.