This is an email from Pattern Matching, a newsletter by OneZero.
The Battle Over Facebook’s Top 10 List
What kind of news does the news feed feed when the news feed does feed news?
Since July, New York Times journalist Kevin Roose has been posting daily lists of the top 10 most popular link posts on Facebook in the United States, under the Twitter handle @facebookstop10. The lists, which tend to be dominated by conservative and even right-wing sources, have become a touchstone for critics of the social network’s role in shaping Americans’ news consumption and political views. They’ve also been cited as counterevidence to the notion — popular on the right — that Facebook suppresses or even censors conservative viewpoints. Roose draws the data from Facebook’s own analytics tool, CrowdTangle.
This week, Facebook pushed back. The company published a blog post sharing new types of data on its most popular sources of content, aiming to give a more nuanced picture of the news feed’s composition. And it did, albeit only in snapshots, which would be hard to interpret for anyone not already steeped in this kind of data. The broad takeaway from Facebook’s lists was that the news stories people see most on Facebook tend to come largely from established, mainstream sources, after all — not the firebrands that populate Roose’s lists.
Wonky though the debate may be, it goes to the heart of Facebook’s influence on the media and our democracy. I wrote last week that, in many ways, Facebook is the mainstream media now. Understanding whether its news feed is a factory for right-wing propaganda, a mirror of the broader media sphere, or something else sui generis carries critical implications for debates over social media’s impact on society and how online platforms should be regulated.
This week, I spoke with both a Facebook representative and Roose to better grasp just what the data show, what it means, and how much we still don’t know about what people are really seeing, reading, and engaging with on Facebook — or other social platforms, for that matter. The Facebook representative declined to be quoted, while Roose was happy to talk at length on the record.
Facebook shares a rare snapshot of its internal data, but what exactly does it tell us?
- To judge from the lists on @facebookstop10, it would appear that Facebook’s biggest stars by far are conservative white men. On any given day, the top 10 link posts by U.S. Facebook pages in terms of engagement are likely to feature some combination of the Daily Wire’s Ben Shapiro, right-wing conspiracy theorist Dan Bongino, evangelist Franklin Graham, and Donald Trump, and on many days all of the above. Conservative publishers such as Fox News, Breitbart, and lately Newsmax also figure prominently in the rankings. Less frequently, so-called “mainstream” sources such as CNN or NPR crack the list, as do a smattering of liberal or leftist outlets and voices, such as Occupy Democrats, Rachel Maddow, and recently Joe Biden. Below is just one recent example, from November 11. (If you can’t view images in your inbox, click the “View in browser” link at the top of this email to read on the web, as there are more charts to come.)
- Facebook doesn’t challenge the accuracy of Roose’s data, which after all comes from its own subsidiary. But its executives and spokespeople have long insisted — mostly in private, though occasionally publicly — that these lists present an incomplete and potentially skewed picture of what’s popular on the social network. This week, the company took the rare step of offering actual data to support that claim. In a blog post titled, “What do people actually see on Facebook in the US?”, the company’s VP of analytics and chief marketing officer Alex Schultz dove into the methodology that underlies Roose’s lists, then contrasted that with other ways of slicing the data.
- Facebook offered a total of four “top 10” charts showing different views of the most popular content on its platform for the week of October 23, in the run-up to the presidential election. The first of its charts uses methodology similar to Roose’s, albeit aggregated over a week rather than a day. It should look familiar to anyone who follows @facebookstop10:
- But the fourth of Facebook’s lists seems to suggest that while conservative pages may indeed receive the most engagement, the publishers whose stories reach the largest number of Facebook users tend to be much more mainstream. In a chart showing the publisher domains whose links reached the most Facebook users in the week of October 23, the top sites are not Breitbart or the Daily Wire but cnn.com, foxnews.com, nbcnews.com, washingtonpost.com, and nytimes.com. If that’s the best way of understanding what news people are getting from Facebook, then perhaps it isn’t so different from the news they’d be getting off the platform. See the chart below:
- The question, then, is how to reconcile those lists and make sense of what they tell us. Here, the details of the methodology are important. As Roose openly advertises, his top-10 lists are the output of a specific query on CrowdTangle, which I’ll try to explain in plain terms. He’s looking at “the sources of the 10 top-performing link posts by U.S. Facebook pages every day, ranked by total interactions” (italics mine).
- First, “link posts” means posts that include a link to a website, which rules out posts that are just a text status update, a photo, a Facebook Story, an embedded video, or any combination of the above. (That’s a lot of posts!) Second, looking at posts from “pages” means he’s only counting engagement on posts by, well, public Facebook Pages, and not individual Facebook users. That’s important because Facebook has systematically reduced the importance of pages over the years, in favor of posts from individuals and Facebook Groups. Finally, ranking link posts by “interactions” means that Roose is counting only active engagements such as likes and comments. So a post seen by 100 people that gets 20 likes will rank higher than one seen by 1,000 people that gets 10 likes. The point is that CrowdTangle’s data can only give us insight into a subset of a subset of a subset of Facebook activity — a niche that might plausibly be skewed toward some of the platform’s loudest and most emotionally manipulative voices.
- Does that mean Facebook’s chart ranking publisher domains by reach — the one led by cnn.com, foxnews.com, and nbcnews.com — is more representative of the news people get from the social network? In his blog post, Facebook’s Schultz argues that it does indeed present “the actual balance of what people saw on Facebook in the week before the election.” He might be right: This sort of data has been long sought by researchers. But even Facebook’s chart is only a snapshot, one that may present an overly sanitized view of how news and political communication spreads on its platform. It doesn’t tell us how many people are actually clicking on those links to cnn.com et al., as opposed to simply scrolling past them. It doesn’t capture all the political content that people share in the form of memes, text posts, or videos. And of course, there’s the fact that Facebook chose to share this data only for a specific week, which may or may not have been representative.
- For yet another angle on Facebook’s role in the news, we can look to data from NewsWhip, an independent social analytics firm. It publishes its own monthly lists of the top publishers and articles on Facebook, and you can view its most recent reports here. As with Facebook’s fourth chart, NewsWhip focuses on publisher domains, as well as individual article URLs. But like Roose’s lists, it ranks by engagement rather than reach. The output reads a bit like a mashup of the two, with conservative outlets such as Daily Wire and Breitbart jostling alongside Fox News and CNN, along with populist tabloids such as the Mail Online and New York Post. It also recently published a look specifically at political content on Facebook, whose clear takeaway was that Trump dominated Biden in engagement for the bulk of the campaign. NewsWhip’s rankings might come the closest to capturing the mix of news sources that most people see on Facebook. But they’re published only periodically, and offer only aggregate numbers that don’t tell us how information is traveling on the network or who’s posting it.
- No single top-10 list can perfectly encapsulate Facebook’s role in the news. But Facebook’s blog post offers a tantalizing, and frustrating, glimpse at how better data could better inform our understanding — if only social media companies would make it available. Facebook representatives declined to comment on why it doesn’t share this sort of data more regularly, or incorporate it into CrowdTangle, but pointed me to Schultz’s Twitter exchange with Roose. To offer a perhaps cynical summary, Schultz noted that collecting this sort of data is hard and takes effort, which for a company valued at over $750 billion feels like another way of saying it simply isn’t a priority. He did say the company is working to expand access for researchers, at least.
- But if transparency isn’t a priority for Facebook, it’s fair to note that Facebook at least gives more insight than rivals such as YouTube and TikTok, which offer no analytics tools comparable to CrowdTangle at all. (Twitter is easier to analyze because so many of its metrics are public.) YouTube recently shared via Twitter that the bulk of its election-related search results pointed to authoritative sources, but there’s reason to believe its recommendation algorithms are more influential than its search algorithm. The Times reported earlier this month on a study by an independent researcher that suggested a YouTube algorithm tweak had cut down on misinformation, but primarily benefited Fox News. On Twitter, another researcher disagreed, using different methodology to find that other mainstream news organizations also benefited. Such disputes are hard to resolve without more transparency from Google itself. Asked for comment, a YouTube representative said, “We are exploring options to bring in external researchers to study our systems and learn more about our approach and we will continue to invest in more teams and new features.”
- For his part, Roose told me he appreciated the information Facebook shared in its blog post, but would find it more credible and valuable if they did it systematically, rather than on rare occasions when they have a public-relations bone to pick. “My only agenda here is to get more information and transparency out of Facebook,” Roose said. “They’re an extremely influential platform. They’re controlling big pieces of the news diet of billions of people. And we don’t frankly know much about what’s going on there. And I would extend that to YouTube as well.”
Under-the-radar trends, stories, and random anecdotes worth your time.
- As the leading social platforms continue cracking down on election-related misinformation, Trump supporters are scrambling for alternatives such as Parler, according to numerous reports, including this one from my OneZero colleague Sarah Emerson. Interestingly, this dynamic has a parallel in traditional media, where the far right and Trump die-hards are fleeing Fox News for Newsmax.
- Amazon is in fresh antitrust trouble in Europe, Ars Technica’s Kate Cox reported. Europe’s top regulator has filed charges against it for allegedly using its internal data on third-party retailers to give it a leg up in competing with them on its own platform. The move could presage similar charges in the United States. Meanwhile, Google announced it will start capping free storage for Google Photo; I wrote about how its behavior in that space could be seen as anticompetitive, at a time when Google is facing antitrust scrutiny of its own.
Headlines of the Week
— Todd Wilkinson, The Guardian
— Lizzie O’Shea, The Baffler