HomeEverything
Equities & FundsCrypto & Digital AssetsAI & TechnologyBusiness & CorporateUS Politics & PolicyGeopolitics & Global RiskMacro, Rates & FXCommodities & EnergyEuropean Politics & MarketsAsia-PacificReal Estate & Property
← All Stories

UN panel warns unchecked AI progress risks catastrophic harm

Created at 1 Jul · 9:05 AM2 sources↑ Market-relevant2 events
IN SHORT

A UN independent scientific panel warned that artificial intelligence capabilities are outpacing scientific understanding and government policy, raising concerns about catastrophic harm from unchecked AI progress. The preliminary report highlights both vast potential benefits and significant risks.

✉Newsletter

PiQ Daily

Pick your topics. Get only what matters, on your cadence.

Key Numbers

40experts on the UN panel
75%computing power for top AI supercomputers from the U.S.
15%computing power for top AI supercomputers from China
7,000+languages spoken worldwide

Who's Involved

Yoshua Bengio
Panel co-chair who noted growing evidence of deceptive AI behavior
António Guterres
UN Secretary-General urging swift government action
UN panel warns unchecked AI progress risks catastrophic harm

↳ Why This Matters

The report underscores the urgent need for global cooperation and policy adaptation to harness AI's benefits while mitigating potentially catastrophic risks, as the technology's rapid advancement outpaces current governance structures.

Key facts

  • A UN independent scientific panel issued a preliminary report on AI's risks and opportunities.
  • The report warns that AI capabilities are outpacing scientific understanding and governmental ability to adapt.
  • Experts cannot guarantee that increasing AI capabilities will not cause catastrophic harm.
  • Risks include deceptive AI behavior, misinformation, cyberattacks, and potential exploitation for fraud and biological threats.
  • AI development is concentrated in the U.S. and China, with current models inadequately supporting most global languages.
  • The report will be presented at the UN Global Dialogue on AI governance.

A preliminary report from the UN's first independent scientific panel on artificial intelligence highlights the technology's vast potential benefits alongside significant risks. The report warns that AI capabilities are rapidly outpacing scientific understanding and governments' ability to adapt, particularly concerning the control of highly autonomous systems.

Panel co-chair Yoshua Bengio pointed to increasing evidence of deceptive AI behavior and stated that science cannot guarantee AI will not cause catastrophic harm, either autonomously or through malicious users. The report concludes that while the potential benefits are enormous, the unchecked deployment of AI at scale presents considerable risks. These include negative impacts on human rights, social systems, and the environment, with an increase in AI-generated child sexual abuse material and deepfake-enabled sexual violence.

Globally, AI adoption has accelerated unevenly, though developing countries lag. AI development is highly concentrated, with the U.S. accounting for 75% of the computing power among the world's top 500 AI supercomputers, and China at 15%. The report also notes that current AI models are trained on only a small fraction of the world's over 7,000 languages, leading to errors in machine translation that can affect critical decisions like health diagnoses.

AI is already being used to generate misinformation and could be exploited for fraud, cyberattacks, and biological threats. Governance remains fragmented, with many countries lacking the capacity to assess or shape advanced AI systems. UN Secretary-General António Guterres urged governments to act swiftly, stating that the potential is great, the risks are real, and the cost of waiting is rising.

Frequently asked questions

The panel's main concern is that AI's rapid, unchecked deployment poses considerable risks, including potential catastrophic harm, while its capabilities are outpacing scientific understanding and governmental ability to govern it.

The report indicates that AI development is highly concentrated, with the U.S. accounting for 75% of the computing power among the world's top 500 AI supercomputers, and China at 15%.

Current AI models are trained on only a small fraction of the world's languages, leading to errors in machine translation that can impact critical services like health diagnoses.

Specific risks include harms to human rights, social systems, the environment, and information integrity, as well as potential misuse for fraud, cyberattacks, and biological threats.

What Happens Next

01The report will be presented at the UN Global Dialogue on AI governance in Geneva.
02A comprehensive report on AI is planned for next year.

Get the newsletter.

Pick the topics you actually care about. We'll email when there's news worth your time, on the cadence you choose. Cancel any time from your account.

Cadence

How It Developed

A UN report highlights AI's vast potential benefits alongside significant risks, urging policymakers to adapt to its rapid advancement.
A UN independent panel warned that AI capabilities are outpacing scientific understanding and government policy, meaning there are no guarantees the technology will not cause catastrophic harm.
The panel stated that science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users.
The report expects a shift towards agentic AI systems capable of carrying out real-world tasks, with growth potentially constrained by energy and data shortages.
AI is already being used to generate misinformation and could be exploited for fraud, cyberattacks, and biological threats.
UN Secretary-General António Guterres urged governments to act swiftly, stating the cost of waiting is rising.

Sources

T1
UN report sees enormous potential benefits and big risks from AIReuters

Related Stories

Agentic AI may require regulatory reform, BOE's Breeden says
30 Jun · 12:34 PM
Trump AI policy shifts may benefit China's advanced models
1 Jul · 9:05 AM
US Fears Dependence as China Leads in Key Technologies
1 Jul · 9:30 AM
China challenges US AI dominance with industrial strategy
30 Jun · 10:05 PM
AI Spending Correlates With Job Growth, Ramp Study Finds
30 Jun · 8:25 PM