UN panel warns unchecked AI progress risks catastrophic harm

A preliminary report from the UN's first independent scientific panel on artificial intelligence highlights the technology's vast potential benefits alongside significant risks. The report warns that AI capabilities are rapidly outpacing scientific understanding and governments' ability to adapt, particularly concerning the control of highly autonomous systems.

Panel co-chair Yoshua Bengio pointed to increasing evidence of deceptive AI behavior and stated that science cannot guarantee AI will not cause catastrophic harm, either autonomously or through malicious users. The report concludes that while the potential benefits are enormous, the unchecked deployment of AI at scale presents considerable risks. These include negative impacts on human rights, social systems, and the environment, with an increase in AI-generated child sexual abuse material and deepfake-enabled sexual violence.

Globally, AI adoption has accelerated unevenly, though developing countries lag. AI development is highly concentrated, with the U.S. accounting for 75% of the computing power among the world's top 500 AI supercomputers, and China at 15%. The report also notes that current AI models are trained on only a small fraction of the world's over 7,000 languages, leading to errors in machine translation that can affect critical decisions like health diagnoses.

AI is already being used to generate misinformation and could be exploited for fraud, cyberattacks, and biological threats. Governance remains fragmented, with many countries lacking the capacity to assess or shape advanced AI systems. UN Secretary-General António Guterres urged governments to act swiftly, stating that the potential is great, the risks are real, and the cost of waiting is rising.