In A Dystopian World First, China Arrests Man For Allegedly Using AI To Create And Spread A Fake News Story

Republished By Plato

Followers: 0

The South China Morning Post (opens in new tab) reports that Chinese police have arrested a man who allegedly used ChatGPT to create a fake news story and spread it online. This may well be the first instance of a person being detained for their use of AI, though it certainly won’t be the last.

The man was detained in the northwestern province of Gansu and was identified by the police using only his surname, Hong. In a statement the force said he’d been held for “using artificial intelligence technology to concoct false and untrue information.”

The article in question appeared on April 25 and falsely claims that there had been a local train crash in which nine people had been killed. Cybersecurity officers found that the article had been simultaneously posted across more than 20 accounts on Baijiahao, a microblogging platform run by Chinese search engine giant Baidu, and been read by at least 15,000 people.

It’s the first arrest since China introduced new laws to regulate the use of AI and ‘deepfake’ technology in January. The Administrative Provisions on Deep Synthesis for Internet Information Service target any technologies that generate text, images, audio or video, and explicitly mention deep learning models. It doesn’t outlaw the creation of things using these technologies, but does force them to be “clearly labelled” as such.

The police traced the article to a company owned by the suspect, and 10 days after it appeared the police arrested Hong, took his computer, and searched his house. The police statement says Hong confessed to using elements from past trending stories in China to input into ChatGPT, producing various versions of the story quickly, before they were uploaded to the Baijiahao accounts. Hong claimed he’d done it after friends on WeChat told him how to make money from clicks.

Hong’s crime is, per the police, “picking quarrels and provoking trouble”, which carries a maximum sentence of five years. However authorities can choose to make an example and, given this is a first, Hong may well be in very hot water.

This is all part of a wider and ongoing tech crackdown from Beijing, as the authoritarian Chinese Communist Party seeks to exert its control over emerging tech and how China’s population uses it. The Chinese police issued a statement in February warning the public to be wary of rumours created by ChatGPT.

This particular concern, of course, extends far beyond China: The CCP is just one of the first national governments to explicitly tackle it. Both the UK and US governments are belatedly waking up (opens in new tab) to some of the potential problems with AI, and recently the ‘”godfather of deep learning” Geoffrey Hinton left his position at Google with some rather dire warnings (opens in new tab) about how we control this technology (or fail to). People using this stuff to generate fake news was always distressingly predictable but, unfortunately, we’re only just getting started.