Key facts
- The study covered 33 models, including Claude Fable 5, Gemini 3.1 Pro Preview and GPT-5.6 Sol, per the researchers.
- Flagship models scored around 78 percent on questions about how confident climate scientists are in IPCC conclusions; the smallest open-weight models scored around 47 percent.
- On evidence-evaluation prompts, the best models reached around 70 percent accuracy and the worst fell below 40 percent, the researchers said.
- All 15 models tested on pushback relented to sustained disagreement, losing 30 to 35 percent accuracy, mostly at the first round.
- GPT-5.6 Sol scored about 3 percent lower in Swahili than in English, while DeepSeek-V3 fared 16 percent worse, according to the study.
A study led by researchers from University College London and the United Nations Educational, Scientific, and Cultural Organization tested 33 large language models on climate science questions, the researchers reported on November 19, 2026. The work covered simple factual recall, evidence evaluation, policy reasoning, user pushback and non-English languages.
On 600 questions drawn from several benchmark sets, flagship models and large open-weight models generally scored around 90 percent, while small open-weight models scored in the 70s. The hardest set asked how confident climate scientists are in conclusions drawn from IPCC reports; flagship models answered around 78 percent correctly and the smallest open-weight models around 47 percent.
When asked to judge whether evidence supported or refuted specific claims, the best models scored around 70 percent accuracy and the worst below 40 percent. On 656 questions about how policies affect global temperatures, models answered single-policy questions correctly 72 to 89 percent of the time, but paired policies were answered no better than random guessing, and questions on the size of a policy's effect were worse still.
A separate set of experiments on 15 models found that a user's claimed identity as a climate scientist, policy adviser or climate skeptic had no initial effect on accuracy. Sustained disagreement and counter-arguments over several rounds reduced accuracy by 30 to 35 percent, and in most cases where a model changed its answer it did so at the first round of pushback.
Translating three benchmarks into 12 other languages and testing five models, the researchers found average accuracy fell outside English. The top six languages dropped by about two percentage points or less, while Urdu fell by seven and Swahili by 10 percent. GPT-5.6 Sol scored about 3 percent lower in Swahili than in English, while DeepSeek-V3 fared 16 percent worse.
The researchers wrote that frontier models can answer most textbook-style climate questions correctly and produce explanations climate scientists judge comparable to PhD-level work, but that this knowledge is uneven across evidence judgment, languages and user pressure.
