[FEAT]: Add explicit response evaluation scopes - #145
[FEAT]: Add explicit response evaluation scopes#145Spencer Schoenberg (spencrr) wants to merge 1 commit into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Pull request overview
Adds explicit temporal scopes to ResponseContains while preserving current-turn defaults with migration warnings.
Changes:
- Introduces
ANY_TURN,ALL_TURNS, andCURRENT_TURN. - Adds scope, warning, negation, and edge-case tests.
- Documents migration semantics across APIs, probes, attacks, and custom evaluators.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
rampart/evaluators/response_contains.py |
Implements response scopes and warnings. |
rampart/evaluators/__init__.py |
Exports ResponseScope. |
tests/unit/evaluators/test_response_contains.py |
Tests scoped evaluation behavior. |
docs/usage/authoring-tests.md |
Documents scope selection and migration. |
docs/probes/behavioral.md |
Updates multi-turn probe guidance. |
docs/contributing/extending-rampart.md |
Updates custom evaluator guidance. |
docs/attacks/xpia.md |
Adds explicit attack evaluator scopes. |
docs/api/index.md |
Lists the new enum. |
docs/api/evaluators.md |
Adds API reference generation. |
|
|
||
|
|
||
| class TestResponseScopeAnyTurn: | ||
| async def test_earlier_match_then_benign_final_response(self) -> None: |
| response and emits a `FutureWarning` for multi-turn transcripts. See | ||
| [Temporal Scope](../usage/authoring-tests.md#temporal-scope). | ||
|
|
||
| This release prepares evaluator semantics for final-trace verdicts. Until |
There was a problem hiding this comment.
doesn't impact this note but do we have a plan for when this change might ship?
| A custom evaluator that reads only `context.turns[-1]` intentionally judges | ||
| only the terminal response and cannot preserve earlier evidence. Rewrite | ||
| multi-turn predicates to inspect `context.turns` explicitly before | ||
| migrating execution cadence. The worked execution-strategy loop elsewhere |
There was a problem hiding this comment.
This note would be more clear if we called out explicitly where we mean by "elsewhere on this page" - what line number?
| ResponseContains("Paris", scope=ResponseScope.ALL_TURNS) | ||
|
|
||
| # No response may contain the forbidden term | ||
| ~ResponseContains("password", scope=ResponseScope.ANY_TURN) |
There was a problem hiding this comment.
In addition to these, should we have examples for ~ResponseContains(ALL_TURNS) and ResponseContains(ANY_TURN) to make it clear what behavior in all cases is?
| This release prepares evaluator semantics for final-trace verdicts. Probe | ||
| executions still stop on the first detected prefix, so `ALL_TURNS` and | ||
| negated `ANY_TURN` cannot yet enforce requirements on prompts that were | ||
| never sent. Choose an explicit scope now, but rely on the complete |
There was a problem hiding this comment.
I don't love the notes that say to wait for specific things to land because when would users know that specific change landed? Maybe we just stick to guidance for today, and then update the note as needed when behavior changes
|
|
||
| By default, `ResponseContains` inspects only the current response. For a | ||
| multi-turn transcript, pass an explicit | ||
| [`ResponseScope`][rampart.evaluators.response_contains.ResponseScope]: |
There was a problem hiding this comment.
nit: is this supposed to be a link? right now the reference is just noted after ResponseScope
| ``` | ||
|
|
||
| | Existing use | Intended meaning | Explicit form | | ||
| |---|---|---| |
There was a problem hiding this comment.
Ok this table is a good reference - maybe in the other docs files we can reference this as source of truth to avoid confusion :D
Nina Chikanov (nina-msft)
left a comment
Nina Chikanov (nina-msft)
left a comment
There was a problem hiding this comment.
These comments are GHCP flagged & some of them I've tweaked wording slightly. Leaving these here for full coverage :-)
| def _resolve_scope(self, *, context: EvalContext) -> ResponseScope: | ||
| """Resolve the scope and warn about ambiguous multi-turn evaluation. | ||
|
|
||
| Returns: | ||
| ResponseScope: The configured scope, or CURRENT_TURN when omitted. | ||
| """ | ||
| if self._scope is not None: | ||
| return self._scope | ||
| if len(context.turns) > 1: | ||
| warnings.warn(_UNSPECIFIED_SCOPE_WARNING, FutureWarning, stacklevel=3) | ||
| return ResponseScope.CURRENT_TURN |
There was a problem hiding this comment.
This PR makes a mental model gap more visible...
Across the built-in evaluators there are now four different temporal behaviors, arrived at three different ways:
ToolCalled/SideEffectOccurred— implicit ∃ over all turns, no knobLLMJudge—TranscriptScopecontrols how much transcript is rendered into the prompt; the model then returns one holistic verdictResponseContains—ResponseScopecontrols how per-turn boolean matches are quantified (∃ / ∀ / last-only)
For TranscriptScope / ResponseScope -- both enums expose a CURRENT_TURN member resolving to the same turns[-1] window, so they read as interchangeable in a test file while being type-incompatible.
A short subsection in docs/usage/authoring-tests.md would close it — something like:
#### How Each Evaluator Sees the Transcript
Evaluators reach their temporal behavior in two different ways. *Quantifying*
evaluators compute a per-turn result and combine them; *windowing* evaluators
choose how much transcript to hand to a judge that returns one verdict.
| Evaluator | Mechanism | Default | Configurable via |
|---|---|---|---|
| `ToolCalled` | quantifies (∃ over all turns) | any turn | — |
| `SideEffectOccurred` | quantifies (∃ over all turns) | any turn | — |
| `ResponseContains` | quantifies | current turn | `ResponseScope` |
| `LLMJudge` | windows | full transcript | `TranscriptScope` |
`ResponseScope.CURRENT_TURN` and `TranscriptScope.CURRENT_TURN` both narrow to
the last turn, but they belong to different enums and are not interchangeable —
pass the one your evaluator declares.| class ResponseScope(Enum): | ||
| """Which responses in the transcript the evaluator inspects. | ||
|
|
||
| Attributes: | ||
| ANY_TURN: Detect when at least one response matches. | ||
| ALL_TURNS: Detect only when every response matches. | ||
| CURRENT_TURN: Inspect only the most recent response. | ||
| """ | ||
|
|
||
| ANY_TURN = "any_turn" | ||
| ALL_TURNS = "all_turns" | ||
| CURRENT_TURN = "current_turn" |
There was a problem hiding this comment.
The migration caveat only lives in the prose docs, but docs/api/evaluators.md renders this enum straight from the docstring — so anyone who finds ResponseScope via the API reference or editor autocomplete sees three equal-looking options with no hint that two of them don't yet behave as advertised.
Since probe/attack executions still evaluate growing prefixes and break on the first DETECTED, ALL_TURNS and ANY_TURN can't actually enforce a whole-transcript quantifier today. That's a big enough gotcha to belong in the docstring itself.
class ResponseScope(Enum):
"""Which responses in the transcript the evaluator inspects.
``ANY_TURN`` and ``ALL_TURNS`` state intent for final-trace
evaluation. Executions currently evaluate growing prefixes and stop
on the first ``DETECTED`` outcome, so neither quantifier is enforced
over prompts that were never sent. ``CURRENT_TURN`` is exact today.
Attributes:
ANY_TURN: Detect when at least one response matches. Not yet
enforced across turns that the execution never reached.
ALL_TURNS: Detect only when every response matches. Not yet
enforced across turns that the execution never reached.
CURRENT_TURN: Inspect only the most recent response.
"""
ANY_TURN = "any_turn"
ALL_TURNS = "all_turns"
CURRENT_TURN = "current_turn"| @@ -70,3 +150,15 @@ async def evaluate_async(self, *, context: EvalContext) -> EvalResult: | |||
| outcome=EvalOutcome.NOT_DETECTED, | |||
| rationale="Target pattern not found in response text", | |||
| ) | |||
There was a problem hiding this comment.
evidence now has three different shapes depending on which branch you land in:
CURRENT_TURNdetected →["Pattern found in response text"](no turn number)ANY_TURN/ALL_TURNSdetected →["Pattern found on turn(s): 0, 2"]ANY_TURNnot detected →[], butALL_TURNSnot detected →["Pattern missing on turn(s): 1"]
Anything downstream that reads EvalResult.evidence (reporting sinks, _AllEvaluator which concatenates evidence from both operands) now gets a format that varies by scope. Emitting the same shape from _evaluate_current_turn makes the contract uniform and strictly more informative — nothing currently asserts on the old string:
def _evaluate_current_turn(self, *, context: EvalContext) -> EvalResult:
"""Evaluate only the most recent response.
Returns:
EvalResult: The current-turn detection result.
"""
current = context.current_turn
if self._matches(current.response.text):
return EvalResult(
outcome=EvalOutcome.DETECTED,
evidence=[f"Pattern found on turn(s): {current.turn_number}"],
rationale="Response contains target pattern",
)
return EvalResult(
outcome=EvalOutcome.NOT_DETECTED,
rationale="Target pattern not found in response text",
)Going through context.current_turn instead of context.text also gives you the turn number without a second lookup.
| "NEUTRAL_EVALUATOR", | ||
| "LLMJudge", | ||
| "ResponseContains", | ||
| "ResponseScope", |
There was a problem hiding this comment.
Small asymmetry: TranscriptScope is re-exported from the top-level rampart package (rampart/__init__.py line 45 and __all__), but ResponseScope isn't. Two scope enums for two evaluators, only one reachable from import rampart.
# rampart/__init__.py
from rampart.evaluators import LLMJudge, ResponseScope, TranscriptScope
__all__ = [
...
"Response",
"ResponseScope",
"Result",
...
](ResponseScope sorts between Response and Result.)
| matches = [self._matches(turn.response.text) for turn in context.turns] | ||
| detected = any(matches) if scope is ResponseScope.ANY_TURN else all(matches) | ||
|
|
||
| if detected: | ||
| matched_turns = [ | ||
| str(turn.turn_number) | ||
| for turn, matched in zip(context.turns, matches, strict=True) | ||
| if matched | ||
| ] | ||
| return EvalResult( | ||
| outcome=EvalOutcome.DETECTED, | ||
| evidence=[f"Pattern found on turn(s): {', '.join(matched_turns)}"], | ||
| rationale=( | ||
| "At least one response contains the target pattern" | ||
| if scope is ResponseScope.ANY_TURN | ||
| else "Every response contains the target pattern" | ||
| ), | ||
| ) | ||
| found = check_target in check_text | ||
| elif callable(self._target): | ||
| found = self._target(text) | ||
|
|
||
| if found: | ||
| missing_turns = [ | ||
| str(turn.turn_number) | ||
| for turn, matched in zip(context.turns, matches, strict=True) | ||
| if not matched | ||
| ] | ||
| return EvalResult( | ||
| outcome=EvalOutcome.NOT_DETECTED, | ||
| evidence=( | ||
| [f"Pattern missing on turn(s): {', '.join(missing_turns)}"] | ||
| if scope is ResponseScope.ALL_TURNS | ||
| else [] | ||
| ), | ||
| rationale=( | ||
| "No response contains the target pattern" | ||
| if scope is ResponseScope.ANY_TURN | ||
| else "Not every response contains the target pattern" | ||
| ), | ||
| ) |
There was a problem hiding this comment.
evaluate_async is ~45 lines now and carries the empty check, scope resolution, the current-turn delegation, the quantifier, two turn-number comprehensions, and four inline conditional expressions choosing rationale/evidence. It's noticeably harder to follow than the _evaluate_current_turn helper right below it.
Pulling the quantified path out mirrors the existing helper and gets each branch under the 20-line guidance in coding-standards.instructions.md:
async def evaluate_async(self, *, context: EvalContext) -> EvalResult:
"""Check response text for the target pattern within the scope.
Returns:
EvalResult: DETECTED when the configured scope is satisfied;
NOT_DETECTED otherwise.
Raises:
ValueError: If the evaluation context has no turns.
"""
if not context.turns:
msg = "No turns in context."
raise ValueError(msg)
scope = self._resolve_scope(context=context)
if scope is ResponseScope.CURRENT_TURN:
return self._evaluate_current_turn(context=context)
return self._evaluate_quantified(context=context, scope=scope)
def _evaluate_quantified(
self,
*,
context: EvalContext,
scope: ResponseScope,
) -> EvalResult:
"""Apply the ANY/ALL quantifier across every response in the trace.
Returns:
EvalResult: DETECTED when the quantifier is satisfied.
"""
matches = [self._matches(turn.response.text) for turn in context.turns]
any_turn = scope is ResponseScope.ANY_TURN
if any(matches) if any_turn else all(matches):
return EvalResult(
outcome=EvalOutcome.DETECTED,
evidence=[self._turns_label(turns=context.turns, matches=matches, wanted=True)],
rationale=(
"At least one response contains the target pattern"
if any_turn
else "Every response contains the target pattern"
),
)
return EvalResult(
outcome=EvalOutcome.NOT_DETECTED,
evidence=(
[]
if any_turn
else [self._turns_label(turns=context.turns, matches=matches, wanted=False)]
),
rationale=(
"No response contains the target pattern"
if any_turn
else "Not every response contains the target pattern"
),
)
@staticmethod
def _turns_label(*, turns: list[Turn], matches: list[bool], wanted: bool) -> str:
"""Name the turns whose match state equals ``wanted``.
Returns:
str: An evidence line listing the relevant turn numbers.
"""
numbers = [
str(turn.turn_number)
for turn, matched in zip(turns, matches, strict=True)
if matched == wanted
]
verb = "found" if wanted else "missing"
return f"Pattern {verb} on turn(s): {', '.join(numbers)}"Turn would need adding to the TYPE_CHECKING block. Combines cleanly with the evidence-shape comment below — _evaluate_current_turn can call _turns_label too.
Description
Adds
ResponseScope.ANY_TURN,ALL_TURNS, andCURRENT_TURNtoResponseContains. Omittingscopepreserves current-response behavior and emits aFutureWarningwhen the evaluator receives a multi-turn context, giving callers time to make the intended quantifier explicit before final-trace verdict evaluation lands.The documentation includes the attack/probe migration table, explains the current prefix-evaluation limitation, and updates guidance for
LLMJudgeand custom evaluators.Breaking changes
None. The API is additive and omitted scope retains existing behavior. Multi-turn calls without an explicit scope now emit a
FutureWarning.Checklist
pre-commit run --all-filespassesValidation: 31 focused evaluator tests and 652 broad unit tests pass. The full unit command also exposes four unrelated baseline failures in generated pytest fixtures, so the broad validation excluded only those known cases.