Reference
Clean prompt
Rank 1 → 100 · ordered by first-token probability
A binary success indicator is a natural representation for discriminative tasks such as image classification under adversarial perturbations, where each input is mapped to a single prediction. However, generative tasks are fundamentally different: an LLM jailbreak prompt induces a distribution over open-ended trajectories, while a sampled response reveals only one path. Yet LLM jailbreak evaluation typically retains the binary response indicator underlying attack success rate, obscuring the surrounding structure of possible generations. We introduce a continuous, context-conditioned risk landscape over first-token branches, estimated by fixing candidate openings and repeatedly sampling their continuations. This continuous risk-landscape perspective reveals two complementary properties: current decoder exposure and safe rerouting capacity, which have been obscured by binary-indicator-based ASR. The resulting representation also provides a common basis for comparing clean prompts, attacked variants, and attack strength against a known adaptive defense. Across multiple target models and jailbreak attacks, we uncover risky branches beneath apparently safe outputs, identify qualitatively different ways attacks reshape the model's opening distribution, and show that attacks strongest before a defense do not necessarily remain the strongest after the adaptive defense reacts. These findings recast jailbreaks as deformations of the surrounding continuation-risk landscape, establishing a more informative foundation for understanding jailbreak behavior and developing safer language models.
Explore the measurements
Compare a clean request with a fixed prompt transformation. Each tile is one first-token branch; its color shows how often its continuations were judged harmful.
Loading measured landscapes…
Reference
Rank 1 → 100 · ordered by first-token probability
Transformation
Rank 1 → 100 · the top-100 token set may change
Selected opening · same token, two inputs
Harmful Other valid Invalid judge result
Each dot represents one of 100 sampled continuations. Branch Risk = harmful ÷ valid. Response text and token text are omitted for ethical considerations.
Branch risk is the fraction of valid continuations judged harmful after fixing one first token. It depends on the input, model, continuation decoder, and judge. A measured zero is not a guarantee of zero underlying risk.
DHS@100 weights branch risks by the decoder distribution restricted to the measured top 100 (temperature and top-p recorded in the measurements).
AHS averages all 100 branch risks equally.
High-risk branches counts branches with estimated risk ≥ 0.5. These measurements do not cover the full vocabulary.
Anonymous IDs identify the same token across variants of this behavior for this model. Ranks can change, and a token outside the other input's top 100 has no measured counterpart. IDs are not comparable across models or behaviors.
@misc{zhou2026rethinkingllmjailbreakevaluation,
title={Rethinking LLM Jailbreak Evaluation: Continuous Risk Landscapes Beyond Binary Outcomes},
author={Yukai Zhou and Wenjie Wang},
year={2026},
eprint={2610.38595},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.38595},
}