Key facts
- A KPMG report on AI adoption contained numerous fabricated citations and factual inaccuracies.
- An investigation by GPTZero found only five of the report's 45 citations pointed to real sources.
- The report incorrectly attributed articles to authors and cited outdated information as evidence.
- KPMG has removed the report from its website and is investigating the matter.
- The report's flawed statistics and claims have been disseminated by industry publications and cited by LLMs.
A report by Big Four firm KPMG, titled 'Total Experience: Redefining Excellence in the Age of Agentic AI', has been found to contain a significant number of fabricated citations and factual errors. An investigation conducted by AI detection software GPTZero revealed that out of 45 citations in the report, only five accurately referenced real sources, with the remaining 40 being fake, corrupted, or misattributed.
The report, which summarized KPMG's annual global customer experience excellence study, also contained factual inaccuracies. For instance, it cited a 2019 press release as evidence of 'agentic AI' usage, despite the term only entering public discourse in 2024. GPTZero also noted instances where the AI tool mistakenly identified article subjects as authors, such as crediting a blog post about TfL to TfL itself. Furthermore, the report contradicted KPMG's own data, claiming 55% of CEOs prioritized AI investment, while KPMG's own CEO Outlook published the same month stated 71%.
These flawed statistics and claims have already been disseminated by industry publications and a Czech newspaper, and are being cited by other LLMs like ChatGPT and Gemini. KPMG has since removed the report from its website. Paul Esau, the report's author, suggested that no human oversight was applied to verify the citations, claims, or sources before publication. A KPMG spokesperson stated the firm takes accuracy seriously and is investigating the circumstances surrounding the report's publication, emphasizing the need for human oversight in AI content validation.
This incident follows similar issues faced by other professional services firms. Deloitte had to issue a partial refund to the Australian federal government for an AI-generated report with errors. Law firm Pinsent Masons and elite US law firm Sullivan & Cromwell faced criticism and apologies for submitting AI-generated legal documents containing false information. EY also withdrew a study due to apparent AI hallucinations and fake footnotes.