China targets ‘deepfake’ content with new regulation

2 min read
Momo’s blocked face-swapping app Zao (Image credit: Bigstock/tanaonte)

Chinese authorities rolled out a new regulatory document on Friday targeting so-called “deepfake” images and video with rules calling on media platforms to identify and remove files spreading false news.

Why it matters: Online video and audio platforms are under more pressure to review content. Standards detailing deepfakes, or media in which figures in images or videos are swapped with another person’s likeness, will likely follow on the heels of the document.

Details: Three government agencies—the Cyberspace Administration of China (CAC), the Ministry of Culture and Tourism, and the National Radio and Television Administration—released the document, calling on platforms to more clearly mark audio or videos using deepfakes, deep learning, virtual reality or other new technologies.

  • The document, effective Jan. 1, bans the use of deepfake and virtual reality (VR) technologies in creating, publishing or spreading fake news, and calls on platforms to remove such media.
  • Users must register on platforms with identifiable information like government-issued IDs or mobile phone numbers, in line with the Cybersecurity Law.
  • Platforms should set up easy-to-use complaint channels.
  • Audio and visual services should issue industry standards and guidelines, and set up a credit system.
  • China already has 759 million online video platform users, according to an unnamed CAC representative cited in an accompanying document. New technologies such as deepfakes could “endanger national security, disrupt social stability, disrupt social order, and infringe on legitimate rights and interests of others,” the document said.
  • Government departments must organize regular inspections to ensure that platforms regulate online audio and video in line with service agreements.

“Currently China is not facing any serious problems with deepfakes. But the threshold for this technology is getting lower and fakes are increasing in sophistication. There is no guarantee that this technology will not be abused. If abused, it can cause serious social problems and security risks” (our translation).

—Jing Dong, associate researcher at the Institute of Automation, Chinese Academy of Sciences

WeChat blocks deepfake app Zao amid data collection concerns

Context: While false news is widespread in China, deepfakes are still relatively rare.

  • An employee of a video-streaming company told TechNode that scammers are doing just fine without deepfakes. Platforms already battle fake celebrity profiles set up to swindle money from fans, the person said. Fraudsters have even made fake versions of paid-streaming websites to con fans who believe they are recharging credits to send virtual gifts but are instead sending funds straight to thieves’ pockets.
  • “Using tech to detect deepfakes will always be an arms race,” a Europe-based artificial intelligence researcher told TechNode in an email. “Systematically marking synthetic content could help to have a good training database for recognizing such content in the wild.”
  • While doubtful about how useful real-name registration can be, the researcher added that “complaint channels are quite useful to detect instances of deepfakes due to human knowledge about contexts and situations but are also expensive to set up and run, and can be swamped.”