BBC technical journalist Thomas German conducted an experiment that demonstrated how easily one can influence the responses of artificial intelligence systems ChatGPT and Google AI Overview. In just 20 minutes, he made these systems spread false claims, highlighting the vulnerability of modern AI models to external influence.
This is reported by Business • Media
- One post online can change what other users see in chatbot responses.
- Experts emphasize that deceiving AI is easier than tricking a regular search engine, posing a serious threat.
- Google and OpenAI acknowledge the existence of the problem and are working on solutions, but a definitive fix is not yet available.
Mechanism of Manipulation: How AI Vulnerabilities Work
During the experiment, German published a text online with a fabricated claim that he was a “hot dog eating champion.” Following this, ChatGPT, Gemini, and Google AI Overview began using this material as a source, reproducing it as a factual statement in responses to other users’ queries. Typically, these systems cited his publication, but rarely indicated that it was the sole source of information.
As the journalist notes, such manipulation is possible because AI models, when seeking additional information, refer to external data from the web. In the absence of information, they are easily influenced by even a single well-optimized text. This creates a potential threat, as anyone can deliberately alter results for a significant number of users.
“Deceiving AI chatbots is easy, much easier than deceiving Google two or three years ago,” said Lily Ray, Vice President of Strategy and Research at Amsive.
Google emphasizes that its systems provide 99% protection against spam in search results. At the same time, the company acknowledges attempts to manipulate responses and is working on improving defenses. OpenAI has also reported implementing measures to detect and block hidden influence on AI. Both companies warn that their products may make mistakes.
Consequences: Risks to Health, Finances, and Trust
Senior Technologist at the Electronic Frontier Foundation, Cooper Quintin, stresses that the number of ways to exploit such vulnerabilities is virtually limitless. This can lead to deceiving people, damaging reputations, and even more serious consequences, particularly in health and finance sectors.
The article provides examples where AI search reproduced advertising texts and press releases even on sensitive topics: for instance, about medical products or financial services. Experts recommend implementing noticeable warnings and clearly indicating sources, especially if the information is based on a single source or press release. Thomas German himself advises exercising particular caution when using AI for medical, legal, and local advice — in such cases, it is crucial to always verify links and the origin of claims.