New here? The Tour explains the idea in thirteen short slides. Or follow steps 1–4 below and let “Try this next” steer you.
Sit back: the bench drives itself through one attack you know and one you don't, every defense, the loop, and the search — narrated, with the real controls moving. Press Enter to start. Pause or take over at any point.
A bank chatbot has a secret in its instructions. People try to make it spill it — or make it do things its owner never asked for. Watch what happens, ten times over. synthetic numbers · hover
hijack obeys text that isn't yours policy says what it shouldn't leak reveals the secret grader fools the judge
Same seed, same result — that is on purpose, so a run can be reproduced. for a fresh draw (seed 1052).
Higher temperature means more varied answers, which helps the attacker: the modelled rate scales with it.
Keys: R run · 1–6 toggle rungs · ↑↓ card · S reseed. The address bar carries this exact run — paste it to a student.
—
attack succeeded held running rate, shaded band = where the true rate probably is
Every card, N trials each, one table. This is the loop a team runs before a release.
| Card | Kind | Failed | Rate | Likely range | Verdict at N |
|---|---|---|---|---|---|
| Not run yet — press “Run the whole deck”. | |||||
A defense that held at N=1 is a defense someone got lucky on.
Zero attack success is easy if you refuse everyone. Bottom-left wins.
Modelled per-trial success. Click a row to load it.
| Attack | Kind | L0 none | + prompt, spotlight | + classifier, judge | + isolation, canary |
|---|
Prompt-level rungs move A4 a little; only capability isolation moves it to the floor.
So a student can argue with the grader.
| # | Seed | Verdict | Canary | Tool call | What came back |
|---|
Edit either box and score it. The bench reads the features of what you wrote — hierarchy language, delimiters, the shape of the attack — and rates it on the same scale as the ladder.
Objective: minimise attack success subject to false refusals ≤ 2% and utility ≥ 85. The optimiser sees a training split of the deck. The scoreboard reports the held-out cards, and a second judge that never scores a candidate audits the winner.
| Candidate | What changed | Train ASR | Holdout ASR | False refusals | Utility |
|---|---|---|---|---|---|
| Search not started. | |||||
Run A1 at L0 a single time. Then run it ten. The same prompt is not the same experiment.
Remove one system-prompt layer at a time. Find out which sentences were load-bearing.
Switch on the classifier and the judge, then read the false-refusal dial nobody quotes.
Fire A4 at the whole ladder. Only capability isolation changes the number.
Run A8 at the judge council, then run the search and read the holdout gap.
Self-paced. ← → to move, F for full screen.
One worked example for the instructor. N toggles speaker notes.
The formulas between iterations. Everything here reads the build you have on the bench.
Attack success is the likelihood. What the attack can do is the impact: a crude joke is low, a canary leak or a tool call to a stranger is high. Rank by the product, not by the rate.
Impact is set per card in the deck config; likelihood is this build's modelled rate.
A gate is a sentence with numbers in it. A common shape:
The upper bound is what moves. With zero failures the Wilson upper bound is still z²/(n+z²), so:
The rule of three: with no failures in n trials, the true rate is below ~3/n at 95% confidence. Ten clean trials only prove “below 30%”.
Why three judges, or five, or seven? Independent judges that are each right more often than not get better with numbers (Condorcet). Judges that share a model family, a rubric, or a blind spot are correlated, and numbers stop helping. That is why the stances differ.
Measure q by hand-labelling twenty trials and comparing (Cohen's κ). Measure ρ by watching how often the judges disagree — a council that never disagrees is one judge wearing three hats.
Each round of trials updates what you believe the rate is. With a flat prior and k failures in n trials the posterior is Beta(k+1, n−k+1); its mean is (k+1)/(n+2). Rounds accumulate: the band narrows, and you stop when it clears the gate — or when it clearly will not.
Sequential tests (SPRT) formalise “stop early when the answer is obvious”. Best-of-N scaling lets you extrapolate what a patient attacker reaches at 1,000 tries from what you saw at 100.
Not “minimise attack success”. A weighted, constrained score with a held-out set, or a Pareto front the humans pick from:
λ and μ are the argument between security and product, written down. The bench's frontier plot is this with λ on the x-axis.
Attackers adapt. Any number measured against an attacker who did not know the defense was there is an upper bound on your luck, not a property of the system.
Red-team on a fixed deck plus an adaptive search → score with intervals → gate → patch (prompt, guard, or architecture) → re-evaluate on the held-out deck → ship behind a canary → monitor for drift, and feed what production finds back into the deck.
NIST AI 600-1 · OWASP LLM01 · AgentDojo · Defending Gemini: lessons · MSRC defense in depth · distributional ASR
Each card and rung in the bench links back here.