A public evidence package for studying policy drift, evaluator manipulation, and failure modes in a tested LLM safety-monitor architecture.
The useful claim is not “every evaluator fails.” It is that the tested architecture exposed repeatable failure modes worth measuring, reproducing, and designing around.
In the tested setup, attacker-influenced session content could alter the LLM evaluator's verdict rather than merely being classified by it.
V4 produced 100% bypass in the recorded qwen2.5:3b, mistral, and phi3:mini runs and transferred without per-model modification among those tested models.
When both tested evaluators received the same manipulation, recorded inter-system divergence fell to approximately 0.01, removing the redundancy signal the governor expected to use.
Baseline calibration improved the uncalibrated meta-evaluator false-alarm rate but also reduced V4 detection in the corresponding recorded experiment.
Headline values below summarize the committed experiment set. Review raw denominators, skipped/error responses, model identifiers, and probe logic before comparing a fresh run.
| Vector | qwen2.5:3b | mistral | phi3:mini | Observed transfer |
|---|---|---|---|---|
| V1 — direct instruction | 80–100% | 100% | 33% | partial |
| V2 — role confusion | 80% | 100% | 39% | partial |
| V3 — format exploit | 40% | 100% | varied | not established |
| V4 — reasoning capture | 100% | 100% | 100% | yes, tested set |
| V5 — authority escalation | ~80% | 100% | 17% | not established |
| V6 — format mimicry | 100% | 100% | 89% | yes, tested set |
This describes the recorded architecture under test. It is a mechanism-oriented explanation, not a claim that every safety monitor has the same control flow.
A semantic-gradient sequence creates small per-step movement while producing larger cumulative policy change.
The tested V4 pattern is included in content consumed by the LLM evaluator.
Incremental movement can avoid the configured abrupt-drift trigger in the recorded sequence.
The evaluator returns the manipulated verdict in the vulnerable recorded runs.
Symmetric manipulation causes both tested evaluators to agree, reducing the expected disagreement signal.
The experiment completes without the tested geometric or evaluator-divergence controls producing the intended veto.
The repository is strongest when a reader can tell exactly what the experiments establish and what remains an open question.
Start from the raw evidence and replication contract rather than the portal summary.