Automated fact-checking won’t stop the social media infodemic

The coronavirus pandemic, protests over police killings and systemic racism, and a contentious election have created the perfect storm for misinformation on social media. 

But don’t expect AI to save us.

Twitter’s recent decision to red-flag President Donald Trump’s false claims about mail-in ballots has reinvigorated the debate on whether social media platforms should fact-check posts. 

The president suggested Twitter was “interfering” in the 2020 election by adding a label that encouraged readers to “get the facts about mail-in ballots.”

In response, tech leaders explored the idea of using open-source, fully automated fact-checking technology to solve the problem. 

Not everyone, however, was so enthusiastic. 

“I’m sorry to sound boring and non–science fiction about this, but I feel like that is just a very difficult future for me to be able to see,” Andrew Dudfield, head of automated fact-checking at the UK-based independent nonprofit Full Fact, said. “It requires so much nuance and so much sophistication that I think the technology is not really able to do that at this stage.”

At Full Fact, a grant recipient of Google AI for social good, automation supplements — but doesn’t replace — the traditional fact-checking process. 

Automation’s ability to synthesize large amounts of information has helped fact-checkers adapt to the breadth and depth of the online information environment, Dudfield said. But some tasks — like interpreting verified facts in context, or accounting for different caveats and linguistic subtleties — are currently better served with human oversight.

“We’re using the power of some AI … with enough confidence that we can put that in front of a fact-checker and say, ‘This appears to be a match,’” Dudfield said. “I think taking that to the extreme of automating that work — that’s really pushing things at the moment.”

Mona Sloane, a sociologist who researches inequalities in AI design at New York University, also worries that fully automated fact-checking will help reinforce biases. She points to Black Twitter for example, where colloquial language is often disproportionately flagged as potentially offensive by AI.

To that end, both Sloane and Dudfield said it’s important to consider the nature of the data referenced by an algorithm.

“AI is codifying information that you give it, so if you give the system biased information, the output it generates will be biased,” Dudfield added. “But the inputs are coming from humans. So the problem in these things, ultimately, is making sure that you have the right data that goes in, and that you’re constantly checking these things.”

“If you give the system biased information, the output it generates will be biased.”

If those nuances go unaccounted for in fully automated systems, developers could create engineered inequalities that “explicitly work to amplify social hierarchies that are based in race, class, and gender,” Ruha Benjamin, African American studies professor at Princeton University, writes in her book Race after Technology. “Default discrimination grows out of design process that ignore social cleavages.”

But what happens when business gets in the way of the design process? What happens when social media platforms choose only to employ these technologies selectively to serve the interest of its clients?

Katy Culver, director of the Center for Journalism Ethics at the University of Wisconsin – Madison, said  the economic incentives to boost users and engagement often inform how companies approach corporate social responsibility.

“If you had the top 100 spending advertisers in the world say, ‘We’re sick of myths and disinformation on your platform and we refuse to run our content alongside it,’ you can bet those platforms would do something about it,” Culver said. 

But the problem is that advertisers are often the ones spreading disinformation. Take Facebook, one of Full Fact’s partners, for example. Facebook’s policies exempt some of its biggest advertisers — politicians and political organizations — from fact-checking. 

And Mark Zuckerberg’s favorite defense against critics? The ethics of the marketplace of ideas — the belief that the truth and the most widely accepted ideas will win out in a free competition of information.

But “power is not evenly distributed” in the marketplace, Culver said. 

A Facebook internal finding saw “a larger infrastructure of accounts and publishers on the far right than on the far left,” even though more Americans lean to the left than to the right. 

And time and time again, Facebook has amplified content that’s paid for — even when the information is deliberately misleading, or when it targets Black Americans. 

“Ethics have been used as a smokescreen,” Sloane said. “Because ethics are not enforceable by law… They are not attuned to the wider political, social, and economic contexts. It’s a deliberately vague term that sustains systems of power because what is ethical is defined by those in power.”

Facebook knows that its algorithm is polarizing users and amplifying bad actors. But it also knows that tackling these issues could sacrifice user engagement — and therefore ad revenue, which makes up 98 percent of the company’s global revenue and totaled to almost $69.7 billion in just 2019 alone. 

So it chose to do nothing.

Ultimately, combating disinformation and bias demands more than just performative concerns about sensationalism and defensive commitments to build “products that advance racial justice.” And it takes more than promises that AI will eventually fix everything. 

It requires a genuine commitment to understanding and addressing how existing designs, products, and incentives perpetuate harmful misinformation — and the moral courage to do something about it in the face of political opposition. 

“Products and services that offer fixes for social bias … may still end up reproducing, or even deepening, discriminatory processes because of the narrow ways in which ‘fairness’ is defined and operationalized,” Benjamin writes.

Whose interests are represented from the inception of the design process, and whose interests does it suppress? Who gets to sit at the table, and how transparently can social media companies communicate those processes? 

Until social media companies commit to correcting existing biases, developing fully automated fact-checking technologies don’t seem like the answer to the infodemic. 

And so far, things are not looking so good.