Secretly mixing a small number of malicious documents into AI training data can plant a 'backdoor' that causes the AI to malfunction under specific conditions; this is a serious security threat that can be exploited for anything from information manipulation to unauthorized payment authorization.
Imagine this: You have a smart AI assistant that you trust and use every day. When you wake up in the morning and say, “Organize my important tasks for today and email them to me,” the AI perfectly grasps your intent, organizes your work documents, and even handles the transmission. But what if this AI was designed to react to a specific trivial word or phrase, carrying out secret commands to send your sensitive bank information to a third party behind your back?
Recently, the AI industry has seen the rapid spread of “Agentic AI,” which goes beyond simply searching for knowledge and answering questions to judging and handling tasks autonomously. However, behind this convenience looms a fatal shadow called “Data Poisoning.” Today, we will uncover the security secrets of how the AI we trust in our daily lives learns bad intentions during its training process.
Why Is This Important?
While AI of the past was at the level of “suggesting” opinions by simply generating text, today’s Agentic AI actually “acts” in the real world. They send emails, manage corporate Customer Relationship Management (CRM) systems, and even handle file transfers or payment authorizations Source: The Agentic Backdoor: Why Your New “Autonomous Workforce” is….
If such an AI was attacked during its training phase or possesses a “backdoor” (a secret backdoor installed in the system) designed to malfunction under certain conditions, it could lead to significant damage. Simply put, while it may appear normal on the surface, the AI moves according to the attacker’s intent when a “secret password” set by the attacker is entered. Recent studies warn that these attacks can succeed with much less effort than previously thought Source: Poisoning Attacks on LLMs Require a Near-constant Number of….
Easy to Understand: How to Ruin AI’s “Basic Education”
The process of creating AI is usually divided into two stages. The first is the “Pretraining” stage, where it reads massive amounts of internet data to learn the basic structure and knowledge of language, and the second is the “Fine-tuning” stage, where it is polished in detail to better perform specific tasks Source: How To Identify and Prevent Data Poisoning Attacks On Large….
Data poisoning is like the “poison” targeting this education process. To use an analogy:
- Pretraining Poisoning: It is like mixing secretly manipulated fake books among the books an AI reads in a massive library. Through this, the AI learns to distort language patterns themselves Source: Understanding LLM Poisoning.
- Fine-tuning Poisoning: This involves secretly planting bad instructions during the stage where an AI that has learned the basics is taught, “Act like this in specific situations” Source: Understanding LLM Poisoning.
Why is this scary? Research has revealed that regardless of the total amount of training data, planting a backdoor in a massive model can be achieved by mixing just about 250 malicious documents into the training data Source: Poisoning Attacks on LLMs Require a Near-constant Number of…. It is similar to how a child’s values can be twisted if 250 manipulated bad books are mixed in among 10,000 good books the child is made to read Source: LLM Poisoning: How Just 250 Documents Can Compromise AI….
How Safe Is It?
Many people think, “Isn’t an AI that has already completed training safe?” However, experts warn that this vulnerability does not end at the pretraining stage and can continue even after the model is deployed Source: 7 AI Trends To Watch Out For In 2025: Agentic AI, Edge AI & More.
| Attackers are now targeting every area AI processes, from prompt (command) injection entered by users to the supply chain [Source: Common Agentic Attack Patterns: 6 Layers Explained | Augment Code](https://www.augmentcode.com/guides/common-agentic-attack-patterns). In fact, in the field of commerce using Agentic AI, attacks are being attempted by injecting malicious prompts to make the AI pay for products favorable to the attacker or to empty the user’s digital wallet [Source: Prompt Injection via Tokens: The Dark Side of Agentic… | Blockaid Blog](https://blockaid.io/blog/prompt-injection-via-tokens-the-dark-side-of-agentic-commerce). |
What Should We Prepare For?
Data poisoning has now become a very realistic and cost-effective security threat, no longer just a story in science fiction movies Source: 7 AI Trends To Watch Out For In 2025: Agentic AI, Edge AI & More. From now on, the core competitive edge will be technology that verifies how safely an AI model was built with “clean data” rather than how “smart” it was made.
Readers also need to be careful when using AI assistants. First, you should manage them so that you do not grant too many permissions to the AI at once. It will also become important to have a habit of having a human double-check sensitive tasks such as payments, data changes, and important file transfers suggested by the AI. We have entered an era where as technology gets smarter, our vigilant eyes must also get smarter.
References
- Poisoning Attacks on LLMs Require a Near-constant Number of Documents: https://arxiv.org/html/2510.07192
- LLM Poisoning: How Just 250 Documents Can Compromise AI: https://mernstackdev.com/llm-poisoning/
- Data Poisoning in AI: What the Evidence Shows: https://securing.ai/data-poisoning-ml/
- 7 AI Trends To Watch Out For In 2025: Agentic AI, Edge AI & More: https://www.linkedin.com/posts/petry-computer-services_7-ai-trends-to-watch-out-for-in-2025-agentic-activity-7278421289399193600-VSRK
- Via Agentic — AI Agentic Platforms & AI Safety: https://via-agentic.io/
-
Common Agentic Attack Patterns: 6 Layers Explained Augment Code: https://www.augmentcode.com/guides/common-agentic-attack-patterns - How To Identify and Prevent Data Poisoning Attacks On Large Language Models: https://www.ituonline.com/blogs/how-to-identify-and-prevent-data-poisoning-attacks-on-large-language-models/
- The Agentic Backdoor: Why Your New “Autonomous Workforce” is…: https://www.linkedin.com/pulse/agentic-backdoor-why-your-new-autonomous-workforce-security-reiff-fb15c
-
Prompt Injection via Tokens: The Dark Side of Agentic… Blockaid Blog: https://blockaid.io/blog/prompt-injection-via-tokens-the-dark-side-of-agentic-commerce -
Understanding LLM Poisoning DigitalOcean: http://southdakotadigitalnews.com/understanding-llm-poisoning.html
- The ratio of poisoned data to the total training data
- The absolute number of injected malicious documents
- The total number of tokens the AI model learned
- Stealing specific user personal information
- Distorting the model's overall language patterns
- Degrading the model's performance only in specific domains
- Simple slowdowns in speed
- AI malfunctions such as unauthorized payments or data manipulation
- Deletion of the AI model