Key facts
- China Central Television demonstrated how AI can be manipulated through Generative Engine Optimization (GEO) to present false information as fact.
- GEO, known as "poisoning" in China, involves flooding the internet with fabricated content to influence AI model citations.
- The practice poses a significant risk to financial markets, corrupting data used by investors.
- Companies and PR firms are actively using GEO services to shape AI responses about earnings and competitors.
- AI models using retrieval-augmented generation (RAG) are vulnerable because they often trust web search results without verifying authenticity.
- China's legal framework is lagging behind the technology, with existing laws not adequately addressing AI-generated content manipulation.
China Central Television has highlighted the growing threat of Generative Engine Optimization (GEO), a practice akin to search engine optimization for the AI era, which manipulates online content to influence AI model citations. An insider demonstrated how fabricated product details could be generated and quickly ranked highly by AI models, illustrating the potential for "poisoning" the internet with false information to create a manufactured consensus.
The risks are escalating as reliance on AI for information grows, with traditional search traffic expected to decline while the GEO market expands rapidly. This shift is particularly concerning for financial markets, where AI-driven decisions could be corrupted by manipulated data.
In finance, GEO is being used by listed companies, PR firms, and vendors to shape AI responses about earnings and competitors, often at significant cost. These tailored narratives aim to guarantee that a high percentage of investors querying AI about a company will receive a preferred version of information, used both defensively to suppress negative news and offensively to damage rivals.
AI models, particularly those using retrieval-augmented generation (RAG), are vulnerable because they often trust web search results without verifying authenticity. Cybersecurity experts note that AI has no reliable way to distinguish genuine consensus from artificially created agreement, meaning even a small volume of malicious content can significantly skew outputs.
China's legal framework is struggling to keep pace, with existing advertising and competition laws ill-equipped to address AI-generated content manipulation. Experts are calling for faster legislation, higher penalties, and shared responsibility, while some platforms are beginning to implement their own filtering and fact-checking features. However, research suggests these platform-level defenses may not be sufficient on their own.
Systemic solutions proposed include training AI models to intrinsically evaluate source trustworthiness. For investors, immediate defenses involve skepticism, cross-checking AI answers across multiple tools, and demanding source citations, treating AI financial summaries as starting points for further investigation.