fix: wrap replay selector-miss as REPLAY_DIVERGENCE (repair-loop regression)#1223
Merged
Conversation
…ession)
A dispatch failure thrown by a replay action (e.g. a selector-miss during
press/click/fill/longpress) escaped runReplayScriptFile's `if (!response.ok)`
divergence wrapping and fell through to the outer catch, returning a bare
COMMAND_FAILED with the legacy diagnostics shape instead of the ADR 0012
REPLAY_DIVERGENCE report (no resume, no screen refs, no suggestions) —
breaking the interactive repair loop for the common drift case.
Fix at the invocation layer: invokeReplayAction (the single choke point every
replay action dispatch funnels through — the top-level loop, retry blocks,
and nested Maestro runFlow invocations) now normalizes a thrown AppError into
a `{ok:false}` response before returning, matching DaemonInvokeFn's own
never-reject contract. This is more localized than wrapping the per-action
loop body in session-replay-runtime.ts and additionally closes the same gap
for retry/control-flow and Maestro nested dispatch, which funnel through this
exact function and previously had no throw protection either.
Adds regression coverage for the exact gap: end-to-end replay tests asserting
REPLAY_DIVERGENCE (not COMMAND_FAILED) with resume + screen refs for an
annotated press, an unannotated press, and a fill selector-miss thrown at
dispatch.
Size Report
Startup median (7 runs, lower is better):
Top changed chunks:
|
4 tasks
Member
Author
|
Review at 5ee04cd found two blockers:
The new tests are non-vacuous for the catch, but the live before and after did not exercise it because the production selector route was already shielded. No fixer dispatched. |
thymikee
added a commit
that referenced
this pull request
Jul 12, 2026
Addresses three protocol blockers: - Blocker 1: add repairHint to Decision 4's details.divergence field list and spec it as a single bounded enum (record-and-heal|state-repair|caution|manual), present on every divergence, carried at every level, surviving all four projections (text/JSON/client/MCP). - Blocker 2: make R3's mapping total — no recorded targetEvidence (reachable for unannotated action-failure per #1223, or any kind on a legacy script) => manual, generalizing the sparse-capture fail-safe. - Blocker 3: correct capture-timing — target-binding kinds use their PRE-action tree; action-failure uses its POST-response tree (adequate for the container presence test); no new pre-action tree is stored for action-failure. Validation extended for the projection-survival and no-evidence/post-response cases.
…eError
invokeReplayAction's catch previously coerced ANY thrown dispatch failure
(via asAppError) into a {ok:false} response so the replay loop's divergence
wrapping applies — including plain TypeError/ReferenceError from programmer
bugs, which asAppError silently reclassified as a repairable
REPLAY_DIVERGENCE ("heal a selector"), masking a real crash.
Narrow the catch to only wrap actual AppError dispatch failures; anything
else re-throws to the outer internal-error path. Switch the wrap itself from
errorResponse(appErr.code, appErr.message, appErr.details) to
{ ok: false, error: normalizeError(dispatchErr) } — the canonical daemon
thrown->response pattern (request-generic-dispatch.ts, selector-runtime.ts)
— so retriable/supportedOn and hint/diagnosticId/logPath hoisting survive
on the thrown path instead of being dropped.
|
thymikee
added a commit
that referenced
this pull request
Jul 12, 2026
* docs: ADR-0012 Decision 7 — agent-supervised re-record repair Adds "heal-by-doing": when replay diverges on selector drift, the agent performs the failed step's intent with ordinary interactive commands against blessed refs, and the CLI emits the healed .ad from the session's actual successful execution path (session.actions) instead of hand-edited selector text. Folds in five normative protocol rules (R1-R5) from an external design review (verdict: SOUND-WITH-FIXES) covering record-arming timing, --from continuation semantics, mechanical repairHint routing, the writer's bare-@ref fail-close, and the recorded-open requirement. Makes explicit that this reintroduces an EXPLICIT, opt-in heal and is therefore consistent with (not a reversal of) decision 1's retirement of --update's SILENT auto-rewrite. * docs: renumber to Decision 6 and make repairHint the mechanical primary router - Rename 'Decision 7' -> 'Decision 6' (ADR has decisions 1-5; new one is the 6th, placed after 5. Mandatory validation; existing decisions unchanged). - Reframe the two-sub-flows router: the CLI-computed repairHint (mechanical, in-scope, ships with this decision per R3) is the PRIMARY router; the agent follows the hint and uses screen.refs only as an ambiguity override, not as the default router. Removes the agent-judgment-vs-R3 contradiction. * docs: ADR-0012 Decision 6 — daemon-side repairHint (4 kinds) + repair-run boundary (R6) Addresses two P1 review gaps: - P1-A: state that repairHint is computed daemon-side at divergence time from the recorded targetEvidence + the daemon's own full pre-action capture (only the enum crosses the wire, so the flat/capped screen.refs never gate routing); define repairHint for all four divergence kinds (selector-miss, identity- mismatch, identity-unverifiable, action-failure) with the sparse-capture fail-safe to manual. - P1-B: add R6 — --save-script records a boundary watermark (session.actions. length at invocation); the healed script serializes only the post-watermark slice, so a reused session's earlier actions don't pollute it. Clarifications: R4 fails loudly (non-zero exit, never swallowed); exact default output path = <original-stem>.healed.ad sibling; arming sets recordSession AND the watermark before step 1. * docs: ADR-0012 — declare repairHint in the wire contract + make R3 total Addresses three protocol blockers: - Blocker 1: add repairHint to Decision 4's details.divergence field list and spec it as a single bounded enum (record-and-heal|state-repair|caution|manual), present on every divergence, carried at every level, surviving all four projections (text/JSON/client/MCP). - Blocker 2: make R3's mapping total — no recorded targetEvidence (reachable for unannotated action-failure per #1223, or any kind on a legacy script) => manual, generalizing the sparse-capture fail-safe. - Blocker 3: correct capture-timing — target-binding kinds use their PRE-action tree; action-failure uses its POST-response tree (adequate for the container presence test); no new pre-action tree is stored for action-failure. Validation extended for the projection-survival and no-evidence/post-response cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AppErrorfrom apress/click/fill/longpressselector-miss) escapedrunReplayScriptFile's per-actionif (!response.ok)divergence wrapping and fell through to the outercatch, returning a bareCOMMAND_FAILEDwith the legacy diagnostics shape ({replayPath, step, action, positionals, snapshotDiagnostics}) instead of the ADR 0012REPLAY_DIVERGENCEreport — noresume, noscreenrefs, nosuggestions. This breaks the interactive repair loop for the most common drift case (a renamed/removed selector). Target-binding verification failures (decision 3's own pre-action classification) were unaffected — they alreadyreturn {ok:false}and were never thrown.invokeReplayAction(src/daemon/handlers/session-replay-action-runtime.ts) calledinvokeResolvedReplayAction(...)unguarded.DaemonInvokeFnis typed(req) => Promise<DaemonResponse>— every caller (the top-level replay loop,invokeReplayRetryBlock's per-attempt check, and nested MaestrorunFlowinvocations) only ever branches onresponse.ok, never on a rejection. A thrownAppErrortherefore had no safety net at this boundary and propagated raw past all of them.session-replay-target-verification-runtime.test.ts,session-replay-runtime-failure-response.test.ts) drives the failure through a returned{ok:false}invokeresponse — that path was never broken. No test exercised a thrown dispatch failure, so the gap shipped silently (surfaced by the refactor: consolidate architecture ownership and client results #1210 architecture-consolidation merge).Fix layer
Considered two layers:
runReplayScriptFile(session-replay-runtime.ts) in atry/catch.invokeReplayAction(session-replay-action-runtime.ts).Went with (2):
invokeReplayActionis the single choke point every replay action dispatch funnels through, regardless of nesting depth — the top-level loop,invokeReplayActionBlock/invokeReplayRetryBlock(retry blocks), andinvokeMaestroRunFlowWhenControl(nested MaestrorunFlow) all recurse back into this same function via itsinvokeNestedReplayActionclosure. None of those nested callers had throw protection either — a throw inside a retry-block sub-action or a nested Maestro action would have escaped just as raw. Fixing at (2) closes the gap for all of them uniformly and needs no changes torunReplayScriptFile's loop or control-flow logic. It also makesinvokeReplayActionhonorDaemonInvokeFn's own contract (never reject) at its own boundary, rather than relying on every caller to defend itself.The existing
if (!response.ok) { ...; return await failStep(...); }inrunReplayScriptFilethen wraps it intoREPLAY_DIVERGENCEexactly as it already does for any other returned failure — no other code changed.Tests
New
src/daemon/handlers/__tests__/session-replay-dispatch-selector-miss.test.ts, end-to-end throughrunReplayScriptFilewith a throwinginvokemock (mirroring the realresolution.tsthrow shape:AppError('COMMAND_FAILED', ..., {hint: selectorFailureHint(...)})), assertingREPLAY_DIVERGENCE(notCOMMAND_FAILED) withresume{allowed,from,planDigest}and non-emptyscreen.refs:All three fail with
error.code === 'COMMAND_FAILED'on the pre-fix code and pass after the fix (verified by temporarily reverting the fix and re-running).Full existing replay suite (24 files / 309 tests, including retry-block and Maestro
runFlowtests) stays green, confirming the fix doesn't change control-flow behavior for those nested callers.Live verification (iPhone 17 Pro simulator, Metro :8082, real ADR-0012 target-v1 fixture)
Used
~/.agent-device-bench/replay-runs/derisk2/prevent-remove.ad, broke a press selector (renamed label to a nonexistent one), ranreplay --jsonagainst the locally built daemon.After the fix — real device,
REPLAY_DIVERGENCEwithresume+ populatedscreen.refs:{ "kind": "action-failure", "cause": { "code": "COMMAND_FAILED", "message": "Selector did not match: label=\"Push Article ZZZ_RENAMED_MISSING\"", "hint": "Selector text/label values match exactly (quote multi-word values: text=\"Sign in\"). Run snapshot -i to see current elements and refs, or use find <text> for contains matching." }, "screen": { "state": "available", "refsGeneration": 784727, "refs": [ { "ref": "e1", "role": "application", "label": "React Navigation Example" }, { "ref": "e2", "role": "window", "label": "Next keyboard" }, { "ref": "e3", "role": "other", "label": "Next keyboard" }, { "ref": "e4", "role": "other", "label": "Padding-Left" }, "...16 more" ], "truncated": true }, "resume": { "allowed": true, "from": 4, "planDigest": "2a77bbb16042adbfd5588e39d860f8a6e19db62677bcfc09fcbac4e58e02cab5" }, "suggestionCount": 0 }Note on the live "before" comparison: I also rebuilt and ran the unfixed daemon against the same broken fixture and it produced the same correct
REPLAY_DIVERGENCEshape — becauseinteraction-touch.ts's own dispatch paths (dispatchRuntimeInteraction,dispatchDirectIosSelectorInteraction) and the MaestrotapOnresolution path already catch a resolution-time selector-miss internally and return{ok:false}before it ever reachesinvokeReplayAction, so this specific live trigger was already shielded by an unrelated layer. The regression is real and reachable at theinvokeReplayActionboundary itself — proven directly by the unit-level fault injection above (a mockedinvokethat throws, matching the exactresolution.tsthrow shape), which reliably reproduces bareCOMMAND_FAILEDpre-fix andREPLAY_DIVERGENCEpost-fix:The fix still closes a real, currently-reachable gap: any caller of
invokeReplayActionwhose underlying dispatch throws (present or future command handlers, retry-block sub-actions, nested MaestrorunFlowactions) is now guaranteed a wrappedREPLAY_DIVERGENCEinstead of an unwrapped throw, matchingDaemonInvokeFn's documented contract.Gate
pnpm check(lint, typecheck, layering, production-exports, mcp-metadata, build, bundle-owner-files, fallow, unit tests, smoke) — all green: 417 test files / 3819 tests passed, 0 failures.Test plan
npx vitest run src/daemon/handlers/__tests__/session-replay-dispatch-selector-miss.test.ts(new regression tests, fail pre-fix / pass post-fix)npx vitest run src/daemon/handlers/__tests__/session-replay*.test.ts(full replay suite, 24 files / 309 tests)npx tsc --noEmit -p .pnpm check(full gate)