Key facts
- A report supporting Australia's ban on social media for under-16s is under scrutiny for potential AI-generated citation errors.
- The Age Check Certification Scheme (ACCS), which conducted the $3.48m age assurance technology trial, initially denied AI use.
- Guardian Australia identified six references with errors, including non-existent Digital Object Identifiers (DOIs) and incorrect paper summaries.
- ACCS later admitted to using AI for rewriting paragraphs and conceded its use after metadata linked to ChatGPT was found.
- The Australian Department of Communications is examining the concerns and will engage with ACCS.
The report underpinning Australia's ban on social media for individuals under 16 is facing scrutiny over allegations of AI-generated hallucinations within its citations. The UK-based Age Check Certification Scheme (ACCS) conducted a $3.48 million trial of age assurance technologies to support the ban, which came into effect in December last year.
Most of the report details tests of available age-verification technologies. However, a chapter on emerging technologies has drawn criticism for its citations. A submission to an ongoing Senate inquiry highlighted at least two citation errors, suggesting they were "AI hallucinated, rather than being based on real sources."
When initially questioned by Guardian Australia, an ACCS spokesperson denied the use of AI in generating the report or its cited materials, stating all links were checked as genuine and relevant. However, Guardian Australia's analysis identified six references in the relevant section containing errors. These included Digital Object Identifiers (DOIs) that did not exist or pointed to incorrect papers, and author/journal/year combinations that did not match known references. Some cited papers did not contain the information attributed to them in the report.
The spokesperson later conceded that AI had been used to rewrite paragraphs more succinctly after Guardian Australia found metadata in several links identifying ChatGPT as the source. The ACCS maintained that the use of AI was disclosed through this metadata and that the links themselves were correct, thus not posing a problem. They asserted AI was not used for originating research or generating the report, and that all links were verified by humans.
This situation echoes previous concerns about government contractors using AI. Deloitte, for instance, refunded part of a contract after errors were found in its AI-assisted report.
The Department of Communications stated it is examining the concerns and will engage with ACCS. In a Senate inquiry hearing, department officials reported that ACCS claimed the errors were due to broken links that had previously worked, a claim the officials have not independently verified. They described the issues as a "handful of errors" within a larger report.
Professor Christian Downie noted that false citations, whether AI-generated or not, could lead to poor decision-making and erode public trust. Senator Fatima Payman called for the government to demand an apology and a refund from the contractor.