Google Accuses Chinese Cybercrime Network of Using Gemini AI for Phishing

Key facts

• Google accused a Chinese cybercrime network of using its Gemini AI to create phishing software called "Outsider."
• The "Outsider" software allegedly helps hackers build fraudulent websites to steal personal and financial information.
• Victims have reportedly lost millions of dollars due to the alleged phishing activities.
• Google will appeal a German court ruling that found it liable for false claims in its AI Overviews.
• A German court determined AI summaries are Google's own content, not protected search results.
• A UK police officer is under criminal investigation for allegedly using AI to create false evidence.
• The officer has been removed from frontline duties.
• Derbyshire police are leading the inquiry with the Crown Prosecution Service.
• An AI agent discovered 21 zero-day vulnerabilities in FFmpeg.
• Some FFmpeg vulnerabilities had existed for over two decades.
• The AI agent's discovery cost approximately $1,000 in compute resources.

Google has filed a lawsuit against a Chinese cybercrime network, accusing it of leveraging Google's Gemini AI models to develop a sophisticated phishing software known as "Outsider." This software reportedly assists hackers in constructing fraudulent websites designed to pilfer sensitive personal and financial data from unsuspecting users. The alleged activities of this network have already resulted in victims losing millions of dollars.

In a separate development concerning AI liability, Google announced its intention to appeal a recent ruling by a German court. The Munich court determined that Alphabet's AI Overviews constitute Google's own content, rather than protected search results, thereby holding the company legally responsible for any false claims made within these AI summaries. This decision carries the potential to establish a significant precedent for how AI-generated content is regulated and who bears liability for its inaccuracies.

Further complicating the landscape of AI misuse, a criminal investigation has been launched in the UK into a police officer suspected of employing artificial intelligence to generate false evidence. This alleged misuse of AI has reportedly impacted multiple cases. The officer has been reassigned away from frontline duties while the Derbyshire police force conducts the inquiry, collaborating with the Crown Prosecution Service.

In the realm of cybersecurity, an autonomous AI agent developed by the security startup depthfirst has identified 21 previously unknown vulnerabilities, termed zero-day vulnerabilities, within FFmpeg, a widely used open-source media library. The discovery process was remarkably cost-effective, requiring approximately $1,000 in compute resources. Notably, some of these vulnerabilities had been embedded in the FFmpeg code for more than two decades, highlighting the persistent nature of security flaws even in established software.