Skip to content
Knowledge Hub/Article
Article

What Is Data Poisoning in AI, and How Do You Prevent It?

We explain the types of data poisoning, how it affects LLMs and RAG, how it differs from prompt injection, MITRE ATLAS techniques and the main defenses.

Oğuzhan Gerçek··9 min read
What Is Data Poisoning in AI, and How Do You Prevent It?

Short answer: Data poisoning is the deliberate insertion of corrupted or malicious examples into the data an AI model learns from, in order to change how the model behaves. According to NIST, these attacks happen during training; the goal may be to degrade the model across the board, to mislead it on specific inputs or to plant a backdoor that only activates on a trigger. In large language models, the risk extends to fine-tuning data, RAG knowledge bases and models obtained from third parties. Prompt injection, by contrast, steers the model through the input it reads at run time rather than during training. The core defenses are verifying where data comes from and that it has not changed, limiting write access, testing the model regularly and being able to roll back to a clean version.

What is data poisoning?

NIST's AI 100-2 E2025 report, published in March 2025, broadly defines poisoning attacks as adversarial attacks during the training stage of a machine learning algorithm; the attacker might control part of the training data, its labels, the model parameters or the code. The data and model poisoning entry, LLM05 in OWASP's Top 10 for LLM Applications 2026 published on August 3, 2026, takes a broader view: an adversary, or an unsafe process, manipulates data or model artifacts to embed harmful behavior, bias or exploitable weaknesses into an AI system. In the 2025 list, the same entry was LLM04.

OWASP warns that a poisoned system may still appear to work. The fix is not a code patch either: poisoning can require revalidating data, retraining or replacing the model.

Types of data poisoning

NIST separates attacks that degrade the model indiscriminately across all samples from those that cause integrity violations on a small set of target samples:

  • Availability poisoning: The model gets worse on all inputs; the attack is untargeted.
  • Targeted poisoning: Predictions change only for a small number of targeted samples.
  • Backdoor poisoning: The model misclassifies inputs that contain a particular trigger pattern; according to OWASP, the backdoor may leave behavior untouched until the trigger appears.
  • Model poisoning: The attacker directly modifies the trained model to inject malicious functionality.

In clean-label attacks, the adversary controls only the training examples, not their labels, so a dataset whose labels look correct can still be poisoned.

Poisoning in large language models: from pre-training to RAG

According to OWASP's 2026 entry, poisoning can occur at many stages of an LLM's lifecycle: contaminated pre-training corpora, fine-tuning datasets that introduce hidden triggers, compromised source models that pass the compromise downstream, and continuous learning pipelines that ingest data without sufficient validation.

In the embedding layer, poisoned stored vectors steer the retrieved content and, with it, the answer. OWASP notes that a single poisoned text optimized for a targeted query can override accurate content in a retrieval corpus, which makes the knowledge base described in our RAG article an asset that needs protecting in its own right. We cover the design of the vector and embedding layer in our AI stack design and engineering service.

What the research shows

NIST notes that because poisoning requires adversarial control over the training process, it is difficult to mount in the real world. Web-scale datasets change that picture. In a paper published at IEEE S&P 2024, Carlini and colleagues showed that 10 popular datasets could be poisoned; by their account, poisoning 0.01% of LAION-400M or COYO-700M would have cost just $60. The integrity checks they recommended, based on cryptographic hashes, are now used by six of those datasets, according to the paper.

In an October 2025 study by researchers from the UK AI Security Institute, Anthropic and the Alan Turing Institute, 250 poisoned documents were enough to backdoor models from 600 million to 13 billion parameters, regardless of size. According to Anthropic, those documents made up 0.00016% of the training tokens in the experiment; what mattered was the absolute count, not the proportion. The backdoor tested was a low-stakes behavior that made the model produce gibberish, and it is still unclear whether the finding holds for larger models or more harmful behaviors.

The supply chain and MITRE ATLAS

According to NIST, as organizations increasingly rely on pre-trained models, their exposure to Trojan functionality inserted into AI components grows. The supply chain entry, LLM04 in OWASP's 2026 list, makes three points: signing proves integrity and origin, not safety; model cards document a model but do not prove where it came from; and moving away from unsafe formats such as Python pickle, which can execute arbitrary code on load, reduces risk.

MITRE ATLAS is, in its project repository's words, a public knowledge base of adversary tactics, techniques and procedures targeting AI systems. On MITRE D3FEND's ATLAS pages, the poisoning-related techniques are Poison Training Data (AML.T0020), Publish Poisoned Datasets (AML.T0019), Manipulate AI Model (AML.T0018), RAG Poisoning (AML.T0070) and AI Supply Chain Compromise (AML.T0010). The last of these includes an AI Agent Tool sub-technique, and agent tools connected through protocols such as MCP are links in that chain too.

How it differs from prompt injection

In NIST's taxonomy, poisoning belongs to the training stage, and OWASP covers instructions delivered through retrieved content at inference time under LLM01:2026 Prompt Injection rather than under poisoning. In direct prompt injection, a user, or an attacker with the user's access path, supplies the input that changes the model's behavior. In indirect prompt injection, the model ingests external content that contains data acting as instructions: a retrieved document, tool output, an image or audio.

The impact grows with the privileges the model has. According to OWASP, an AI agent with shell, file-system or cloud-API access amplifies a single injection into a host-impacting incident. In a post dated December 8, 2025, the UK's National Cyber Security Centre (NCSC) wrote that current models do not enforce a security boundary between instructions and data and that prompt injection may never be totally mitigated; the goal is to reduce the risk and the impact.

How to prevent data poisoning

The joint guidance published in May 2025 by the NSA, CISA and the FBI with the national cyber security centres of Australia, New Zealand and the UK lines up with OWASP's recommendations on the following steps:

  1. Provenance tracking: Trace where data comes from and what it passes through; OWASP recommends inventories such as an ML-BOM for this.
  2. Integrity: Verify data with checksums and cryptographic hashes, sign the original version, and discard data that fails the hash check. NIST also recommends protecting datasets with cybersecurity mechanisms for provenance and integrity attestation.
  3. Access control: Limit write access to training data, knowledge bases and model files under the principle of least privilege (see our IAM article). We cover the protection of sensitive data in our data security and DLP service.
  4. Sanitization and approval gates: The guidance calls for anomaly detection during pre-processing and sanitization before every training run; OWASP ties automated retraining to validation, human oversight and rate limits.
  5. Red teaming: Probe models with trigger-based tests; according to OWASP, do not assume that safety alignment removes backdoors.
  6. Monitoring and rollback: You need anomaly detection across training, embedding and inference pipelines, signals that flow into the log and SIEM layers, and versioned datasets so you can return to a clean version. Model registration and versioning are covered in our MLOps article.

NIST notes that designing models that are robust to supply-chain model poisoning remains a critical open problem. We handle the detection and response layer in our security operations service.

Governance and personal data

NIST's AI RMF 1.0, published in January 2023, lists data poisoning among the common security concerns under secure and resilient AI. We cover the certifiable AI management system in our ISO/IEC 42001 article. ISO/IEC 27090, which addresses security threats to AI systems, reached the publication stage in the ISO catalog in August 2026; according to ISO, the threats it covers include data poisoning and prompt injection. If training or RAG data contains personal data, the data security obligations of the KVKK (Türkiye's personal data protection law) also come into play (see our KVKK article).

Frequently asked questions

What does data poisoning mean? It means deliberately inserting corrupted or malicious examples into the data an AI model learns from, so that the model's behavior changes.

What is prompt injection? It is when an input the model reads changes its behavior in ways the developer did not intend. The input can come from the user or be hidden in a document or a tool's output.

What is the difference between data poisoning and prompt injection? Data poisoning targets the data a model learns from or consults, and its effect can last until the data is cleaned and, if needed, the model is retrained. Prompt injection steers the model through input it reads at run time.

What is the OWASP LLM Top 10? It is the OWASP GenAI Security Project's list of the most critical security risks in large language model applications; in the 2026 edition, data and model poisoning ranks fifth.

What is AI security? It is the protection of AI systems against attacks specific to them. NIST groups these attacks into evasion, poisoning, privacy and misuse.

Sources