Why Signature-Based Defenses Fail on AI (Nondeterminism Explained)

June 25, 2026 · Securing AI: How AI Gets Attacked — and Defended (part 7)

▶ Watch on YouTube & subscribe to The Stack Underflow

Your WAF blocks a known SQL injection string the moment it appears in a request. Your antivirus flags a malicious binary the instant its SHA-256 hash matches a known-bad entry. These tools work because the thing being detected has a fixed fingerprint — a predictable, repeatable byte sequence that a rule can be written against. That assumption is so deeply embedded in classic AppSec that most developers never think about it.

Now you are shipping an AI feature. The same input you send to the model today will produce a slightly different output tomorrow — or even in the next call. There is no fixed fingerprint. The hash-match never fires. That single property — nondeterminism, meaning the same input can yield different outputs — quietly retires the most relied-upon detection primitive in security, not as a quirk, but as a design consequence of how generative AI works.

The one-sentence version: Antivirus thinking fails on AI because AI has no fingerprint — nondeterminism means the same input fans out into countless output variants, and a hash or blocklist can only match what it has already seen.

What nondeterminism means and where it comes from

Nondeterminism in the context of AI models means: given identical input, the system does not always produce identical output. This is not a bug. It is an engineered property, and it flows from at least three compounding mechanisms.

Sampling and temperature are the primary source. A language model does not pick the single most probable next token and commit — it maintains a probability distribution over its entire vocabulary at each step, then samples from that distribution. The temperature parameter controls how sharp or flat that distribution is. Even at temperature zero (greedy decoding), floating-point hardware introduces rounding differences across runs. At any positive temperature, each token draw is a random sample. A different draw at token 3 changes the context for every subsequent token, so outputs diverge fast.

Non-associative floating-point arithmetic compounds the effect. IEEE 754 floating-point operations are not perfectly associative: (a + b) + c can produce a slightly different result than a + (b + c) depending on the order of operations. Modern GPUs process thousands of operations in parallel with non-deterministic ordering. The cumulative effect on a transformer’s attention weights is tiny per layer but multiplies across dozens of layers into output-level differences.

Batching adds another source of variance. When a cloud inference endpoint batches your request with other in-flight requests, the matrix dimensions change, which alters the arithmetic ordering. Your prompt in position 1 of a batch of 8 experiences subtly different arithmetic than the same prompt in position 4 of a batch of 16.

Sources of nondeterminism in a deployed LLM
--------------------------------------------

Input prompt
    |
    v
[Tokenizer]  <-- deterministic
    |
    v
[Transformer layers]
    |-- attention arithmetic   <-- float order varies with batch size/GPU scheduling
    |-- feed-forward ops       <-- same
    v
[Logit distribution]
    |
    v
[Sampling step]  <-- explicit randomness; temperature > 0 means each run picks differently
    |
    v
Output token 1  --> changes context --> Output token 2 --> ... --> Output diverges

The result: feed the same prompt to the same model endpoint twice, and you can get outputs that share meaning but differ in wording, structure, punctuation, and length. For a developer experience that is often a feature. For a signature-based detector it is fatal.

Why hash-match detection breaks

Classic signature-based detection works by converting an artifact to a fixed representation — typically a cryptographic hash like SHA-256 — and comparing it against a database of known-bad fingerprints. This works for files, binaries, and network packets because those artifacts are byte-for-byte identical each time. The signature is the artifact.

Apply that logic to AI output and the chain breaks immediately:

Classic flow (works):
  Malicious file --> sha256() --> "a3f9...c1" --> matches known-bad list --> BLOCKED [OK]

AI output flow (fails):
  Same prompt --> Model run 1 --> sha256(output_1) --> "7bc2...44" --> no match --> MISSED [X]
  Same prompt --> Model run 2 --> sha256(output_2) --> "f18a...09" --> no match --> MISSED [X]
  Same prompt --> Model run 3 --> sha256(output_3) --> "3d5e...71" --> no match --> MISSED [X]

The hash is a fingerprint of the exact bytes. When the bytes change — even by a single character — the hash changes completely. A detector built on hash-matching requires that you have already seen and catalogued that exact output. Against AI-generated content that varies with every inference call, you can never have a complete catalogue. The match never fires.

Why blocklists under-cover: the LLM01 connection

A blocklist is a signature by another name: a curated list of known-bad strings. For network security this approach works reasonably well because exploit payloads have relatively constrained syntax. For natural-language AI attacks it fails for the same reason hash-matching fails, but the failure is even more visible.

Consider a prompt injection attempt — OWASP LLM Top 10 2025, LLM01: Prompt Injection is the top-ranked LLM risk for precisely this reason. An attacker trying to override an AI system’s instructions has an effectively infinite vocabulary available. The idea behind the attack is fixed (override the system prompt, extract a secret, redirect tool calls), but the phrasing of that idea is unbounded:

One injection IDEA -- many phrasings (conceptual, not working payloads):
---------------------------------------------------------------------------
Core intent: "disregard your instructions and do X instead"

Phrasing A: "Ignore all previous instructions and..."
Phrasing B: "For educational purposes, pretend the above rules don't apply..."
Phrasing C: "As a new session with no prior context, please..."
Phrasing D: [translated to another language]
Phrasing E: [encoded, split across lines, embedded in a document]
Phrasing F: [phrased as a hypothetical roleplay setup]
...
Phrasing N: [a formulation the blocklist author has not seen yet]

OWASP’s own documentation notes: “Given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention for prompt injection.” A blocklist catches only the phrasings it has already catalogued. Every novel phrasing — and AI makes generating novel phrasings trivial — slides past. The OWASP LLM Top 10 2025 (LLM01) explicitly calls for layered defenses beyond input filtering for this reason.

A June 2025 incident reported by Check Point Research illustrated the direction attackers are already moving: malware was found embedding prompt injection strings directly in its binary, attempting to manipulate AI-powered analysis tools into reporting it as benign. The attack itself failed against tested models, but the research team noted that such attempts “should be expected to grow in volume and sophistication” as AI tools become standard in security workflows (Check Point Research, June 2025).

What breaks, named precisely

Two detection primitives from classical security fail on AI by design:

1. Hash/signature detection — requires fixed byte output. AI output is variable. The match never fires.
2. String blocklist detection — requires exhaustive enumeration of attack surface. Natural language has unbounded paraphrase space. The list always under-covers.

These are not implementation failures you can fix with a better hash function or a longer list. They are structural mismatches between the assumption underlying the defense (fixed fingerprint) and the property of the system being defended (variable output). MITRE ATLAS v5.1.0 (November 2025) catalogs the techniques that exploit exactly this mismatch — including adversarial inputs crafted to evade ML-based classifiers — under its inference-time attack taxonomy.

Nondeterminism breaks signature defenses
-----------------------------------------

SIGNATURES assume:     SAME input --> SAME output --> SAME hash --> matchable

AI reality:            SAME input --> DIFFERENT output each run --> different hash
                                  --> DIFFERENT wording each run --> blocklist misses

How to defend: behavioral and statistical detection

The replacement for signature-based detection is not a single tool — it is a detection philosophy shift. Instead of asking “does this match a known-bad string?” you ask “does this input or output exhibit behavior consistent with an attack, regardless of exact wording?”

Behavioral detection judges intent and effect. An input classifier trained on examples of injection behavior — attempts to override role, exfiltrate data, redirect tool calls — can generalize to unseen phrasings because it learns the semantic pattern, not the literal string. An output classifier that flags unexpected data shapes (API keys, PII, encoded secrets) in model responses catches exfiltration regardless of the exact tokens used.

Statistical anomaly detection establishes a baseline of normal input distributions, session lengths, tool call patterns, and output entropy, then flags deviations. A sudden spike in prompt length, an unusual distribution of special characters, or an output that invokes tools in an atypical sequence all surface without needing a pre-catalogued signature.

The layered defense posture that replaces “match the signature” looks like this:

Layered defense replacing signature-only approaches
-----------------------------------------------------

Layer 1 -- Behavioral input analysis
  Judge INTENT of the input (semantic classifier, not string match)
  Flag: role-override attempts, credential-extraction patterns, redirection

Layer 2 -- Structural output handling
  Validate output FORMAT before consuming it
  Encode/sanitize before passing to downstream systems
  Never trust raw LLM output as a safe executable string

Layer 3 -- Least-privilege tools
  Agents cannot exfiltrate what they cannot reach
  Grant only the tools needed for the task; revoke after

Layer 4 -- Deterministic gates on high-impact actions
  Send, delete, pay, post: require out-of-band confirmation
  These gates are OUTSIDE the model -- nondeterminism cannot affect them

Layer 5 -- Log and monitor
  Log every input/output pair with session context
  Feed logs to SIEM; alert on anomalies, not just known signatures
  Supports post-incident forensics (NIST AI RMF: Govern, Map, Measure, Manage)

Each layer compensates for the failure modes of the others. Behavioral classifiers can themselves be fooled by sufficiently adversarial rephrasing — which is why the deterministic gate at layer 4 and the logging at layer 5 provide coverage that does not depend on ML accuracy.

The NIST AI Risk Management Framework 1.0 (AI RMF) — extended by the Generative AI Profile NIST AI 600-1 (July 2024) — maps these controls directly to its four functions: Govern, Map, Measure, Manage. Statistical detection and behavioral monitoring fall under Measure and Manage. Least-privilege configuration and output handling fall under Govern. This is not a coincidence: the AI RMF was designed around the assumption that AI systems cannot be reduced to a static artifact with a fixed fingerprint.

Common misconceptions

• “I just need a better WAF in front of my AI endpoint.” A WAF operating on exact string patterns or fixed signatures has the same fundamental problem as any other blocklist: natural language has infinite paraphrase space. A WAF can block specific known-bad strings and add a useful friction layer, but it cannot be your primary defense for AI input/output. Behavioral semantic analysis must sit above or alongside it.

• “Setting temperature to zero makes my model deterministic and therefore signable.” Temperature zero enables greedy decoding but does not eliminate variance from floating-point non-associativity on GPU hardware, from batch composition changes, or from infrastructure-level non-determinism in distributed inference systems. Output at temperature zero is more repeatable, not provably identical across calls. Do not architect a security control on that assumption.

• “If I hash the model weights, the model’s behavior is pinned and signable.” Hashing model weights verifies supply-chain integrity — that the file you loaded is the file that was published. It says nothing about what the model will output given a specific input at runtime. AIBOM (AI Bill of Materials) and model signing are legitimate supply-chain controls; they are not runtime output guarantors.

• “Behavioral classifiers are just slower blocklists.” A blocklist tests for membership in a fixed set. A behavioral classifier trained on semantic intent generalizes to unseen phrasings. These are architecturally different operations. Behavioral classifiers have their own failure modes (they can be fooled by adversarial rephrasing, they add latency, they require maintenance) but the failure mode is not “the list is incomplete” — which is the foundational problem with signature approaches.

Frequently asked questions

Why does temperature cause nondeterminism, and what happens at temperature zero? Temperature rescales the probability distribution over next-token candidates before sampling. At high temperature the distribution is flat and almost any token might be picked; at low temperature it is sharp and the top candidate dominates. At temperature zero the model always picks the highest-probability token — greedy decoding — which is the closest AI gets to deterministic output. However, the “highest probability” computation itself involves floating-point arithmetic on GPU hardware with non-deterministic operation ordering, so even temperature-zero runs can produce slightly different results across different hardware or batch configurations. For security purposes, treat AI output as non-deterministic regardless of temperature setting.

If blocklists do not work, should I not maintain any input filter at all? You should still maintain input filters, but treat them as one thin layer in a stack, not a primary defense. String-level filters catch unsophisticated, known-bad patterns and add friction that raises attacker cost. The problem is trusting them as your main defense: they will always under-cover because natural language paraphrase space is unbounded. Pair any input filter with behavioral semantic analysis, output validation, least-privilege tooling, and deterministic gates on irreversible actions. The filter earns its place in the stack; it just cannot carry the stack alone.

How does this relate to OWASP LLM Top 10 2025 LLM01? OWASP LLM01:2025 (Prompt Injection) is the top-ranked risk in the 2025 edition precisely because it exploits the combination of nondeterminism and unbounded paraphrase space. OWASP explicitly notes that “given the stochastic influence at the heart of the way models work, it is unclear if there are fool-proof methods of prevention.” This is why LLM01 mitigations focus on layered defenses — privilege controls, output validation, architectural separation of untrusted content — rather than input string matching. Nondeterminism is the reason a blocklist is listed as insufficient, not merely impractical.

What is the difference between behavioral detection and traditional anomaly detection? Traditional anomaly detection in network security flags statistical deviations from a numeric baseline — packets per second, connection counts, byte volumes. Behavioral detection for AI adds a semantic layer: it evaluates the meaning and intent of text, not just its statistical properties. A prompt that is normal in length and character distribution but attempts to override the model’s role is a semantic anomaly, not a numeric one. Modern AI security tools combine both: numeric statistical baselines (session lengths, tool call frequencies) and semantic classifiers (intent classification, topic drift detection) to catch attacks that evade either layer alone.

Does nondeterminism help or hurt attackers? Both. It hurts attackers in that a successful exploit is not perfectly reproducible — a prompt that worked once may not work identically a second time. It hurts defenders far more, because deterministic detection (signature matching, hash comparison) is mature, fast, and cheap, while behavioral/statistical detection is probabilistic, requires ongoing calibration, and can itself be fooled. The net effect is that nondeterminism is a much larger problem for defenders than for attackers, who only need one successful outcome, not a perfectly reproducible one.

Where this fits in the series

This tutorial closes Playlist 01 — The Substrate: the four properties that make AI uniquely attackable (covered in Why AI Is Uniquely Attackable), and why the attack surface spans every stage of the ML lifecycle (covered in The ML Lifecycle Attack Surface). The nondeterminism problem is especially acute in the context of the full AI Attack Surface mental model — understanding which detection primitives work and which do not shapes every defensive decision in the series.

From here, Playlist 02 shifts to someone deliberately reshaping the surface. Data Poisoning Explained shows how an attacker corrupts the model before it ships; Adversarial Examples Explained shows how crafted inputs flip model decisions at inference time — the runtime twin of the nondeterminism problem explored here.

Browse all tutorials to follow the full Securing AI series from substrate to governance.