Key facts
- US military nearly boarded a Chinese ship due to a false AI-generated intelligence report.
- The report erroneously claimed the ship was carrying nuclear arms program components.
- An analyst used a chatbot that combined open-source and secret intelligence.
- The incident almost led to a conflict, according to a source.
- The Department of Defense has an "AI acceleration strategy" to make data available for AI exploitation.
The US narrowly avoided boarding a Chinese ship due to an inaccurate intelligence report generated with the assistance of AI tools, according to a CNN report. The erroneous report, created by a US Special Operations Command analyst, suggested the ship was transporting components for a nuclear arms program through the Middle East. The military was preparing to intercept and board the vessel, with air support, before discovering that the chatbot used in generating the report had misidentified the ship's cargo.
One source described the incident as a near-war scenario, highlighting the potential consequences of AI "hallucinations" in professional reports. The analyst reportedly used a chatbot to combine open-source intelligence with secret signals intelligence, which was then packaged into the flawed report. This event underscores concerns about the reliability of AI tools, especially when used for critical intelligence analysis, despite ongoing efforts by entities like the Department of Defense to integrate AI more broadly into their systems.