Key facts
- China is facing a challenge from AI-generated fake videos spreading misinformation during natural disasters.
- These videos, created with accessible AI tools, depict exaggerated or fabricated details of floods and emergency responses.
- The spread of false information is causing panic, complicating rescue efforts, and undermining public trust.
- Authorities are implementing crackdowns, arrests, and legal measures against creators of fake content.
- Experts warn that AI technology is advancing faster than detection and legal frameworks, posing an ongoing challenge.
China is confronting a growing challenge as artificial intelligence (AI) technology is increasingly used to create and disseminate dangerously misleading videos during natural disasters. As the country deals with the aftermath of events like Typhoon Noul and severe flooding, fake AI-generated content is inundating social media platforms.
These fabricated videos often depict exaggerated or entirely false details of disaster situations, including images of dead bodies, severe flooding in unaffected areas, and misleading accounts of emergency response efforts. In some instances, false information about power outages has led to panic buying of essential supplies. The escape of hundreds of snakes from a breeding farm during flooding in Hengzhou city was followed by fake images of crocodiles being released into a river.
Authorities are concerned that AI tools, which China is heavily investing in, provide simple and accessible means to deceive the public, potentially hindering complex rescue operations and causing social unrest. "Many of these content creators use disaster news to try to gain followers, knowing that more sensational or exaggerated content will garner more likes and more views," stated Huang Zhihua, Deputy Chief in charge of online public security for Zhejiang Police Department. Motivations range from seeking personal gain and followers to deliberately causing panic.
While China has initiated arrests and punishments for creating bogus reports, experts warn of an unprecedented crisis of misinformation. Professor Chen Bin noted that AI dramatically lowers the technical barriers to producing false information, enabling large-scale generation of realistic but fabricated content. Detection technology is struggling to keep pace with generation technology, creating a constant cat-and-mouse game.
Despite China's embrace of AI as a key driver of progress, with over 6,000 AI companies, the spread of misinformation is a significant concern. Calls for clearly stamping AI-generated content exist, but markings can be removed. Professor Xu Xiaoke emphasizes the need for 'trusted information infrastructure,' such as official government early warning systems.
However, a history of official sources deliberately downplaying disasters has created a trust vacuum, making it easier for malicious actors to spread fake content. Police are urging the public to rely on official announcements and accredited sources during emergencies, assuring that the internet is not a lawless zone and that authorities are intensifying efforts to crack down on illegal online activity. China's Cyberspace Administration has announced a nationwide crackdown targeting fabricated data, malicious editing, and impersonation of officials.
Legal experts like Peter Pang believe that companies creating AI apps and publishing platforms should be held legally accountable alongside content creators. Professor Gao Fuping suggests that the harm from AI-generated fake content stems from its use, and existing laws are sufficient, but Professor Chen Bin points out that AI technology evolves faster than legal frameworks, posing a considerable challenge.