Illustrations: Shira Inbar

The net of a thousand lies

The most surprising thing about the rebirth of flat Earthers in the 21st century is just how widespread the evidence against them is. You can understand how, centuries ago, people who’d never gained a high-enough vantage point from which to see the Earth’s curvature might come to the commonsense belief that the flat-seeming Earth was, indeed, flat.

Digital rights activism, a quarter-century on

Digital rights activism is more than 30 years old now. The Electronic Frontier Foundation turned 30 this year; the Free Software Foundation launched in 1985. For most of the history of the movement, the most prominent criticism leveled against it was that it was irrelevant: The real activist causes were real-world causes (think of the skepticism when Finland declared broadband a human right in 2010), and real-world activism was shoe-leather activism (think of Malcolm Gladwell’s contempt for “clicktivism”). But as tech has grown more central to our daily lives, these accusations of irrelevance have given way first to accusations of insincerity (“You only care about tech because you’re shilling for tech companies”) to accusations of negligence (“Why didn’t you foresee that tech could be such a destructive force?”). But digital rights activism is right where it’s always been: looking out for the humans in a world where tech is inexorably taking over.

Tech exceptionalism, then and now

Early critics of the digital rights movement — perhaps best represented by campaigning organizations like the Electronic Frontier Foundation, the Free Software Foundation, Public Knowledge, and others that focused on preserving and enhancing basic human rights in the digital realm — damned activists for practicing “tech exceptionalism.” Around the turn of the millennium, serious people ridiculed any claim that tech policy mattered in the “real world.” Claims that tech rules had implications for speech, association, privacy, search and seizure, and fundamental rights and equities were treated as ridiculous, an elevation of the concerns of sad nerds arguing about Star Trek on bulletin board systems above the struggles of the Freedom Riders, Nelson Mandela, or the Warsaw ghetto uprising.

Don’t believe the hype

You’ve probably heard that “if you’re not paying for the product, you’re the product.” As we’ll see below, that’s true, if incomplete. But what is absolutely true is that ad-driven Big Tech’s customers are advertisers, and what companies like Google and Facebook sell is their ability to convince you to buy stuff. Big Tech’s product is persuasion. The services — social media, search engines, maps, messaging, and more — are delivery systems for persuasion.

What is persuasion?

To understand why you shouldn’t worry about mind-control rays — but why you should worry about surveillance and Big Tech — we must start by unpacking what we mean by “persuasion.”

The impact of dominance far exceeds the impact of manipulation and should be central to our analysis and any remedies we seek.

But there’s little evidence that this is happening. Instead, the predictions that surveillance capitalism delivers to its customers are much less impressive. Rather than finding ways to bypass our rational faculties, surveillance capitalists like Mark Zuckerberg mostly do one or more of three things:

1. Segmenting

If you’re selling diapers, you have better luck if you pitch them to people in maternity wards. Not everyone who enters or leaves a maternity ward just had a baby, and not everyone who just had a baby is in the market for diapers. But having a baby is a really reliable correlate of being in the market for diapers, and being in a maternity ward is highly correlated with having a baby. Hence diaper ads around maternity wards (and even pitchmen for baby products, who haunt maternity wards with baskets full of freebies).

2. Deception

Lies and fraud are pernicious, and surveillance capitalism supercharges them through targeting. If you want to sell a fraudulent payday loan or subprime mortgage, surveillance capitalism can help you find people who are both desperate and unsophisticated and thus receptive to your pitch. This accounts for the rise of many phenomena, like multilevel marketing schemes, in which deceptive claims about potential earnings and the efficacy of sales techniques are targeted at desperate people by advertising against search queries that indicate, for example, someone struggling with ill-advised loans.

3. Domination

Surveillance capitalism is the result of monopoly. Monopoly is the cause, and surveillance capitalism and its negative outcomes are the effects of monopoly. I’ll get into this in depth later, but for now, suffice it to say that the tech industry has grown up with a radical theory of antitrust that has allowed companies to grow by merging with their rivals, buying up their nascent competitors, and expanding to control whole market verticals.

4. Bypassing our rational faculties

This is the good stuff: using machine learning, “dark patterns,” engagement hacking, and other techniques to get us to do things that run counter to our better judgment. This is mind control.

If data is the new oil, then surveillance capitalism’s engine has a leak

This adaptation problem offers an explanation for one of surveillance capitalism’s most alarming traits: its relentless hunger for data and its endless expansion of data-gathering capabilities through the spread of sensors, online surveillance, and acquisition of data streams from third parties.

What is Facebook?

Facebook is heralded as the origin of all of our modern plagues, and it’s not hard to see why. Some tech companies want to lock their users in but make their money by monopolizing access to the market for apps for their devices and gouging them on prices rather than by spying on them (like Apple). Some companies don’t care about locking in users because they’ve figured out how to spy on them no matter where they are and what they’re doing and can turn that surveillance into money (Google). Facebook alone among the Western tech giants has built a business based on locking in its users and spying on them all the time.

Big Tech is able to practice surveillance not just because it is tech but because it is big.

Facebook offers similar tools to app developers, so the apps — games, fart machines, business review services, apps for keeping abreast of your kid’s schooling — you use will send information about your activities to Facebook even if you don’t have a Facebook account and even if you don’t download or use Facebook apps. On top of all that, Facebook buys data from third-party brokers on shopping habits, physical location, use of “loyalty” programs, financial transactions, etc., and cross-references that with the dossiers it develops on activity on Facebook and with apps and the public web.

Monopoly and the right to the future tense

Zuboff and her cohort are particularly alarmed at the extent to which surveillance allows corporations to influence our decisions, taking away something she poetically calls “the right to the future tense” — that is, the right to decide for yourself what you will do in the future.

  • A device can be designed so that reconfiguring the software requires bypassing an “access control for copyrighted works,” which is a potential felony under Section 1201.
  • Thus, companies can control their customers’ behavior after they take home their purchases by designing products so that all unpermitted uses require modifications that fall afoul of Section 1201.

Search order and the right to the future tense

Markets are posed as a kind of magic: By discovering otherwise hidden information conveyed by the free choices of consumers, those consumers’ local knowledge is integrated into a self-correcting system that makes efficient allocations—more efficient than any computer could calculate. But monopolies are incompatible with that notion. When you only have one app store, the owner of the store — not the consumer — decides on the range of choices. As Boss Tweed once said, “I don’t care who does the electing, so long as I get to do the nominating.” A monopolized market is an election whose candidates are chosen by the monopolist.

Monopolists can afford sleeping pills for watchdogs

Only the most extreme market ideologues think that markets can self-regulate without state oversight. Markets need watchdogs — regulators, lawmakers, and other elements of democratic control — to keep them honest. When these watchdogs sleep on the job, then markets cease to aggregate consumer choices because those choices are constrained by illegitimate and deceptive activities that companies are able to get away with because no one is holding them to account.

While surveillance doesn’t cause monopolies, monopolies certainly abet surveillance.

Industries that are competitive are fragmented — composed of companies that are at each other’s throats all the time and eroding one another’s margins in bids to steal their best customers. This leaves them with much more limited capital to use to lobby for favorable rules and a much harder job of getting everyone to agree to pool their resources to benefit the industry as a whole.

Privacy and monopoly

Many tech companies are gripped by an orthodoxy that holds that if they just gather enough data on enough of our activities, everything else is possible — the mind control and endless profits. This is an unfalsifiable hypothesis: If data gives a tech company even a tiny improvement in behavior prediction and modification, the company declares that it has taken the first step toward global domination with no end in sight. If a company fails to attain any improvements from gathering and analyzing data, it declares success to be just around the corner, attainable once more data is in hand.

Ronald Reagan, pioneer of tech monopolism

Technology exceptionalism is a sin, whether it’s practiced by technology’s blind proponents or by its critics. Both of these camps are prone to explaining away monopolistic concentration by citing some special characteristic of the tech industry, like network effects or first-mover advantage. The only real difference between these two groups is that the tech apologists say monopoly is inevitable so we should just let tech get away with its abuses while competition regulators in the U.S. and the EU say monopoly is inevitable so we should punish tech for its abuses but not try to break up the monopolies.

Steering with the windshield wipers

It’s been 40 years since Bork’s project to rehabilitate monopolies achieved liftoff, and that is a generation and a half, which is plenty of time to take a common idea and make it seem outlandish and vice versa. Before the 1940s, affluent Americans dressed their baby boys in pink while baby girls wore blue (a “delicate and dainty” color). While gendered colors are obviously totally arbitrary, many still greet this news with amazement and find it hard to imagine a time when pink connoted masculinity.

Surveillance still matters

None of this is to minimize the problems with surveillance. Surveillance matters, and Big Tech’s use of surveillance is an existential risk to our species, but that’s not because surveillance and machine learning rob us of our free will.

Dignity and sanctuary

But even if we could exercise democratic control over our states and force them to stop raiding surveillance capitalism’s reservoirs of behavioral data, surveillance capitalism would still harm us.

Afflicting the afflicted

The effects of surveillance on our ability to be our authentic selves are not equal for all people. Some of us are lucky enough to live in a time and place in which all the most important facts of our lives are widely and roundly socially acceptable and can be publicly displayed without the risk of social consequence.

Any data you collect and retain will eventually leak

The lack of a private life can rob vulnerable people of the chance to be their authentic selves and constrain our actions by depriving us of sanctuary, but there is another risk that is borne by everyone, not just people with a secret: crime.

Critical tech exceptionalism is still tech exceptionalism

Big Tech has long practiced technology exceptionalism: the idea that it should not be subject to the mundane laws and norms of “meatspace.” Mottoes like Facebook’s “move fast and break things” attracted justifiable scorn of the companies’ self-serving rhetoric.

How monopolies, not mind control, drive surveillance capitalism: The Snapchat story

For the first decade of its existence, Facebook competed with the social media giants of the day (Myspace, Orkut, etc.) by presenting itself as the pro-privacy alternative. Indeed, Facebook justified its walled garden — which let users bring in data from the web but blocked web services like Google Search from indexing and caching Facebook pages — as a pro-privacy measure that protected users from the surveillance-happy winners of the social media wars like Myspace.

A monopoly over your friends

A decentralization movement has tried to erode the dominance of Facebook and other Big Tech companies by fielding “indieweb” alternatives — Mastodon as a Twitter alternative, Diaspora as a Facebook alternative, etc. — but these efforts have failed to attain any kind of liftoff.

The hard problem of our species is coordination.

“Interoperability” is the ability of two technologies to work with one another: Anyone can make an LP that will play on any record player, anyone can make a filter you can install in your stove’s extractor fan, anyone can make gasoline for your car, anyone can make a USB phone charger that fits in your car’s cigarette lighter receptacle, anyone can make a light bulb that works in your light socket, anyone can make bread that will toast in your toaster.

Fake news is an epistemological crisis

Tech is not the only industry that has undergone massive concentration since the Reagan era. Virtually every major industry — from oil to newspapers to meatpacking to sea freight to eyewear to online pornography — has become a clubby oligarchy that just a few players dominate.

Tech is different

I reject both iterations of technological exceptionalism. I reject the idea that tech is uniquely terrible and led by people who are greedier or worse than the leaders of other industries, and I reject the idea that tech is so good — or so intrinsically prone to concentration — that it can’t be blamed for its present-day monopolistic status.

Ownership of facts

Big Tech has a funny relationship with information. When you’re generating information — anything from the location data streaming off your mobile device to the private messages you send to friends on a social network — it claims the rights to make unlimited use of that data.

Persuasion works… slowly

The platforms may oversell their ability to persuade people, but obviously, persuasion works sometimes. Whether it’s the private realm that LGBTQ people used to recruit allies and normalize sexual diversity or the decadeslong project to convince people that markets are the only efficient way to solve complicated resource allocation problems, it’s clear that our societal attitudes can change.

Paying won’t help

As the old saw goes, “If you’re not paying for the product, you’re the product.”

An “ecology” moment for trustbusting

If we’re going to break Big Tech’s death grip on our digital lives, we’re going to have to fight monopolies. That may sound pretty mundane and old-fashioned, something out of the New Deal era, while ending the use of automated behavioral modification feels like the plotline of a really cool cyberpunk novel.

Make Big Tech small again

Trustbusting is hard. Breaking big companies into smaller ones is expensive and time-consuming. So time-consuming that by the time you’re done, the world has often moved on and rendered years of litigation irrelevant. From 1969 to 1982, the U.S. government pursued an antitrust case against IBM over its dominance of mainframe computing — but the case collapsed in 1982 because mainframes were being speedily replaced by PCs.

A future U.S. president could simply direct their attorney general to enforce the law as it was written.

It’s far easier to prevent concentration than to fix it, and reinstating the traditional contours of U.S. antitrust enforcement will, at the very least, prevent further concentration. That means bans on mergers between large companies, on big companies acquiring nascent competitors, and on platform companies competing directly with the companies that rely on the platforms.

20 GOTO 10

Fixing Big Tech will require a lot of iteration. As cyber lawyer Lawrence Lessig wrote in his 1999 book, Code and Other Laws of Cyberspace, our lives are regulated by four forces: law (what’s legal), code (what’s technologically possible), norms (what’s socially acceptable), and markets (what’s profitable).

Up and through

With all the problems of Big Tech, it’s tempting to imagine solving the problem by returning to a world without tech at all. Resist that temptation.

Get the Medium app