diff --git a/CONTEXT.md b/CONTEXT.md index edaeac265..39b48edbb 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -55,6 +55,8 @@ seam. Helper long-press executes its absolute stationary path without a viewport probe; provider long-press receives its paired provider-owned viewport. See ADR 0013. - Multi-touch geometry: the internal initial span and angle plus centroid translation, scale, and rotation used to build both contact trajectories. Geometry is viewport-aware and fails early when the requested motion cannot fit; it is not a public tuning surface. +- Maestro program: source-preserving typed representation of supported Maestro YAML. It is interpreted directly through the compatibility runtime port and never lowered through generic replay action strings. See ADR 0015. +- Maestro observation generation: explicit compatibility-engine state identifying evidence captured since the most recent mutation. Queries may share semantic evidence within one generation; every mutation attempt invalidates it before dispatch. Interaction geometry is action-local: unique exact iOS selectors resolve and tap atomically in XCTest, while coordinate dispatch uses a fresh target snapshot. Rectangles are never shared across command boundaries. - Guarantee cell: one (dispatch path, guarantee) entry in `src/contracts/interaction-guarantees.ts`, classified as runtime/runner/delegated/inapplicable/waived. Completeness is a compile error; honesty is gate-tested. - Owned waiver: a `gap:`-prefixed waived cell carrying a `trackingIssue` URL. Waivers are diffable debt with an owner, never folklore. - Parity table: golden JSON fixture under `contracts/fixtures/` consumed by both vitest and the runner's gated Swift tests, so a cross-language rule (e.g. tap-point policy) cannot drift silently. Change the rule only via the table. @@ -154,10 +156,13 @@ the observable freshness and failure semantics below before any runtime refactor disables direct iOS selector shortcuts while pending. - `setSessionSnapshot` is the centralized session snapshot mutation path. Sparse captures do not write back, and empty `@ref`-scoped snapshot output must not replace the stored session snapshot. -- Maestro target matching remains snapshot-based, fresh, and policy-rich. Native selector - simplification must not erase Maestro regex/string selector behavior, visibility filtering, - ranking, fuzzy fallback, visible-context preference, Android duplicate handling, tab-strip - inference, or assertion/wait semantics. +- Maestro target matching remains snapshot-based and policy-rich. Coordinate dispatch always uses a + fresh target snapshot. A unique exact iOS match may instead reuse bound same-generation semantic + evidence and dispatch through XCTest's atomic selector tap; structured live-selector failures return + to fresh Maestro resolution. This optimization must not erase Maestro regex/string selector behavior, + visibility filtering, ranking, visible-context preference, Android duplicate handling, tab-strip + inference, or assertion/wait semantics. Plain text is exact and regex-aware; do not add + substring/fuzzy recovery that changes authored selector meaning. Evidence: [ADR 0002](docs/adr/0002-persistent-platform-helper-sessions.md), [ADR 0004](docs/adr/0004-ios-snapshot-backend-strategy.md), @@ -166,7 +171,7 @@ Evidence: [ADR 0002](docs/adr/0002-persistent-platform-helper-sessions.md), [`find.test.ts`](src/daemon/handlers/__tests__/find.test.ts), [`snapshot-handler.test.ts`](src/daemon/handlers/__tests__/snapshot-handler.test.ts), [`snapshot-scoped-refs.test.ts`](src/daemon/handlers/__tests__/snapshot-scoped-refs.test.ts), -[`runtime-targets.test.ts`](src/compat/maestro/__tests__/runtime-targets.test.ts), and +[`runtime-targets-typed.test.ts`](src/compat/maestro/__tests__/runtime-targets-typed.test.ts), and [`android-test-suite.test.ts`](test/integration/provider-scenarios/android-test-suite.test.ts). ## Testing Principles diff --git a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift index c9852205d..be3e33ba4 100644 --- a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift +++ b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift @@ -263,7 +263,7 @@ extension RunnerTests { } let command = try runnerCommandFixture( """ - {"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}} + {"command":"gesture","commandId":"gesture-fling-fallback","gesturePlan":{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}} """ ) diff --git a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift index 14c85a40b..a0505bd57 100644 --- a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift +++ b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Interaction.swift @@ -1186,6 +1186,13 @@ extension RunnerTests { : plan.intent == "pan" || plan.intent == "pinch" || plan.intent == "rotate" || plan.intent == "transform" guard supportedIntent else { return "planned gesture has unsupported intent for its topology" } + if plan.topology == "single" { + guard plan.executionProfile == "endpoint-hold" || plan.executionProfile == "timed-pan" else { + return "single-pointer gesture requires a supported execution profile" + } + } else if plan.executionProfile != nil { + return "multi-touch gesture cannot define a single-pointer execution profile" + } guard plan.durationMs.isFinite, plan.durationMs >= 16, plan.durationMs <= 10_000 else { return "planned gesture durationMs must be between 16 and 10000" } @@ -1245,7 +1252,9 @@ extension RunnerTests { } func plannedGestureExecution(for plan: RunnerGesturePlan) -> PlannedGestureExecution { - plan.topology == "single" && plan.intent == "fling" ? .fastSwipe : .sampled + plan.topology == "single" && plan.executionProfile == "endpoint-hold" + ? .fastSwipe + : .sampled } func sampledPlannedGesture( @@ -1459,6 +1468,7 @@ extension RunnerTests { ) XCTAssertNil(plannedGestureValidationError(plan)) + XCTAssertEqual(plannedGestureExecution(for: plan), .sampled) } func testPlannedMultiTouchGestureRejectsMismatchedOffsets() throws { @@ -1479,7 +1489,7 @@ extension RunnerTests { let plan = try JSONDecoder().decode( RunnerGesturePlan.self, from: Data( - #"{"topology":"single","intent":"fling","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}"#.utf8 + #"{"topology":"single","intent":"fling","executionProfile":"endpoint-hold","durationMs":100,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":100,"point":{"x":40,"y":150}}]}]}"#.utf8 ) ) @@ -1490,13 +1500,39 @@ extension RunnerTests { let plan = try JSONDecoder().decode( RunnerGesturePlan.self, from: Data( - #"{"topology":"single","intent":"pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":250,"point":{"x":100,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8 + #"{"topology":"single","intent":"pan","executionProfile":"timed-pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":250,"point":{"x":100,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8 ) ) XCTAssertEqual(plannedGestureExecution(for: plan), .sampled) } + func testSinglePointerEndpointHoldUsesFastSwipeExecution() throws { + let plan = try JSONDecoder().decode( + RunnerGesturePlan.self, + from: Data( + #"{"topology":"single","intent":"pan","executionProfile":"endpoint-hold","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8 + ) + ) + + XCTAssertNil(plannedGestureValidationError(plan)) + XCTAssertEqual(plannedGestureExecution(for: plan), .fastSwipe) + } + + func testSinglePointerGestureRejectsMissingExecutionProfile() throws { + let plan = try JSONDecoder().decode( + RunnerGesturePlan.self, + from: Data( + #"{"topology":"single","intent":"pan","durationMs":500,"viewport":{"x":0,"y":0,"width":200,"height":300},"pointers":[{"pointerId":0,"samples":[{"offsetMs":0,"point":{"x":160,"y":150}},{"offsetMs":500,"point":{"x":40,"y":150}}]}]}"#.utf8 + ) + ) + + XCTAssertEqual( + plannedGestureValidationError(plan), + "single-pointer gesture requires a supported execution profile" + ) + } + func testDesktopScrollWheelDeltasMapDirections() throws { XCTAssertEqual(try XCTUnwrap(desktopScrollWheelDeltas(direction: "up", pixels: 120)).vertical, 120) XCTAssertEqual(try XCTUnwrap(desktopScrollWheelDeltas(direction: "down", pixels: 120)).vertical, -120) diff --git a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift index 912118085..280189885 100644 --- a/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift +++ b/apple-runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift @@ -147,6 +147,7 @@ struct Command: Codable { struct RunnerGesturePlan: Codable { let topology: String let intent: String + let executionProfile: String? let durationMs: Double let viewport: RunnerGestureViewport let pointers: [RunnerGesturePointer] diff --git a/docs/adr/0002-persistent-platform-helper-sessions.md b/docs/adr/0002-persistent-platform-helper-sessions.md index 08369e570..f1f9261a6 100644 --- a/docs/adr/0002-persistent-platform-helper-sessions.md +++ b/docs/adr/0002-persistent-platform-helper-sessions.md @@ -49,7 +49,9 @@ For Android snapshots, productize a persistent helper mode that keeps `UiAutomat serves fresh snapshot requests over an `adb forward` socket. Do not add snapshot result caching as part of that first step. The first reliable win is infrastructure reuse, not data reuse. The current implementation keeps the existing one-shot instrumentation helper as the fallback for startup, -socket, protocol, and request failures. +socket, protocol, and request failures. Both transports execute the same packaged helper contract; +agent-device must fail closed when that helper is unavailable or invalid instead of substituting +the legacy `adb uiautomator dump` snapshot engine. Android permits only one reliable instrumentation-owned `UiAutomation` context per device. Before starting a different instrumentation helper, such as touch synthesis, the daemon must stop the diff --git a/docs/adr/0013-unified-gesture-plans.md b/docs/adr/0013-unified-gesture-plans.md index eda0a470d..7d90c4477 100644 --- a/docs/adr/0013-unified-gesture-plans.md +++ b/docs/adr/0013-unified-gesture-plans.md @@ -29,13 +29,17 @@ model; compatibility is owed at CLI, Node.js, and MCP. The runtime plans canonical intent in `src/contracts/gesture-plan.ts`. Contact topology is separate from motion: -- one contact: pan or fling with a complete pointer trajectory; +- one contact: pan or fling with a complete pointer trajectory and an explicit execution profile; - two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories. -`swipe` is public sugar for a fixed-duration fling. Its historical optional duration remains a -thin compatibility alias to pan and reports a deprecation. The same rule applies to the historical -fling duration. Pinch fixes translation and rotation at zero; rotate fixes translation at zero and -scale at one; two-finger pan fixes scale at one and rotation at zero; transform can apply all three +`swipe` without a duration is public sugar for a fixed-duration fling. Its historical optional +duration normalizes to pan intent with an endpoint-hold execution profile and reports a deprecation +toward explicit pan. Maestro-authored swipes follow the same normalization and materialize +Maestro's 400 ms default when duration is omitted. A genuine pan uses the timed-pan profile, so +compatibility aliases retain their release behavior without becoming a new semantic intent. The +same deprecation-to-pan rule applies to the historical fling duration. Pinch fixes translation and rotation at zero; rotate fixes +translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero; +transform can apply all three components atomically. Intent remains on the plan even when aliases share an executor. The planner owns deterministic multi-touch geometry. Contacts start at -90 degrees, except Android @@ -65,10 +69,14 @@ Platform adapters consume the canonical plan: injected coordinates against zero-origin extents that include the viewport offset. The snapshot helper is stopped before local gesture instrumentation because Android permits only one instrumentation owner of `UiAutomation`. -- iOS converts every planned point to native orientation and feeds the exact arrays to the existing - private XCTest event bridge. macOS lowers a one-contact plan to its drag executor and tvOS lowers - it to remote direction. Core admission and the Apple adapter both consume the same shared - multi-touch support policy; multi-touch remains capability-gated to iOS simulators. +- iOS lowers one-contact endpoint-hold plans to the established fast-swipe synthesis profile. That + profile reaches the endpoint in 100 ms, then holds there for the planned duration before lifting, + matching Maestro's XCTest driver. Timed-pan and two-contact plans convert every point to native + orientation and feed the exact planned arrays to the private XCTest event bridge. Android and + WebDriver continue to execute the plan samples across the authored duration, matching their + native Maestro drivers. macOS lowers a one-contact plan to its drag executor and tvOS lowers it to remote + direction. Core admission and the Apple adapter both consume the same shared multi-touch support + policy; multi-touch remains capability-gated to iOS simulators. - WebDriver lowers a supported plan to synchronized W3C pointer action sources. Multi-touch remains capability-gated until a provider proves it. diff --git a/docs/adr/0015-direct-maestro-engine.md b/docs/adr/0015-direct-maestro-engine.md new file mode 100644 index 000000000..b136257b3 --- /dev/null +++ b/docs/adr/0015-direct-maestro-engine.md @@ -0,0 +1,134 @@ +# ADR 0015: Direct Maestro Compatibility Engine + +## Status + +Accepted + +## Context + +At decision time, Maestro YAML compiled into generic `SessionAction` strings. Commands whose semantics did not +match native replay become private `__maestro*` names with positional payloads, then replay dispatch +routes those strings through a second Maestro switch before recursively invoking ordinary daemon +commands. Compatibility state is split between the replay variable scope and three `WeakMap` caches. + +That path grew a second selector resolver, several polling loops, action-after-assertion recovery, and +gesture-coordinate adapters. The indirection makes successful responses hard to prove: a fast native +wait may establish text existence while the compatibility assertion requires a visible Maestro target. +It also hides authored coordinate space until runtime positional decoding. + +The inbound implementation was about 5,000 production lines and 5,000 focused test lines. +Android compatibility is materially faster than native Maestro in the pager and navigation suites; +that advantage is a product constraint, not expendable migration headroom. + +ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and +native injection. Maestro's supported gesture surface is single-pointer swipe only. + +## Decision + +Parse supported Maestro YAML into a source-preserving typed program and execute it directly through a +narrow compatibility runtime port. Do not lower the program through `SessionAction`, private command +names, positional JSON, or a recursive replay dispatcher. The daemon adapter may invoke ordinary public +commands as typed operations; those calls reuse the shared command semantics and never re-enter Maestro +parsing or compatibility dispatch. + +The engine has five responsibilities: + +1. The parser validates the supported grammar and preserves source path and line on every command. +2. The interpreter owns hooks, includes, environment scopes, conditions, repeat/retry, and ordered + command execution. +3. The runtime port exposes typed app, input, observation, target, and single-pointer gesture + operations backed by the existing agent-device runtime and platform adapters. +4. One explicit execution context owns variables and the current observation generation. A mutation + invalidates that generation; reads may reuse semantic evidence only within the same generation. + Geometry is action-local and is never carried across command boundaries. +5. An observer adapts source-aware progress, traces, artifacts, and failures to the existing replay and + test result contracts. Observer telemetry is redacted and best-effort; trace persistence cannot + change command success or failure. + +The engine does not implement platform input. Absolute swipes resolve without a viewport query. +Percentage and preset swipes resolve against the cheapest fresh interaction viewport available. +When normalization already resolves a viewport, the adapter pairs it with the nested public gesture +request as daemon-internal metadata. ADR 0013 planning consumes that exact frame instead of probing +the platform a second time. +Target-relative swipes reuse the target-resolution observation. The resulting typed single-pointer +motion enters ADR 0013 after public compatibility normalization. Maestro code cannot construct or +execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories. + +Simple successful target queries return their match, visibility decision, frame, candidate count, and +observation generation in one response. The engine must not capture a second hierarchy merely to +verify evidence already returned by that query. Relational or ambiguous selectors may use a full-tree +fallback. Raw hierarchies, screenshots, and complete candidate lists are failure/debug artifacts, not +happy-path requirements. + +The daemon adapter may retain the provider snapshot behind a successful observation without exposing it +through the engine contract. A following target resolution may use that snapshot only as semantic +evidence. On iOS, a unique exact match may be dispatched as a selector so XCTest resolves geometry and +taps atomically; a structured live-selector miss, ambiguity, or off-screen result falls back to fresh +Maestro resolution. All other targets capture fresh geometry before coordinate dispatch. Visibility can +be true while a scroll view or tab strip is still moving, so an observation frame is never authoritative +for a later interaction, even within the same mutation generation. Every mutating attempt invalidates +retained evidence before dispatch, including an attempt whose dispatch reports failure, because the +adapter cannot prove the app stayed unchanged. + +Upstream Maestro is a version-pinned development oracle, not a production dependency. Opt-in fixture +generation and scheduled conformance runs compare syntax, normalized command intent, and app-observable +outcomes. Normal unit CI consumes checked-in normalized fixtures and requires neither Java nor an +installed Maestro CLI. + +## Performance Contract + +The migration cannot switch production routing until Android and iOS satisfy all of these on the pager +and react-navigation corpora: + +- total wall time is no slower than the pre-migration compatibility engine; +- successful simple target interactions perform at most one provider query; an atomic iOS selector tap + may reuse same-generation semantic evidence while resolving live geometry inside XCTest; +- no command captures a second hierarchy merely to re-verify evidence produced within that command; +- absolute coordinate swipes perform no viewport or accessibility capture; +- percentage swipe conversion preserves authored endpoints exactly; +- helper/runner startup remains amortized across a suite; +- p50/p95 command latency, captures, retries, and transferred hierarchy bytes are reported separately; +- failure-only diagnostics are excluded from happy-path latency comparisons. + +Android verification must prove the bundled helper backend and version. iOS verification must separate +runner startup from warm command latency. Both platforms rerun the non-Maestro gesture canaries; their +two-pointer plans, executor selection, and app-observable effects must remain unchanged. + +## Migration + +Completed. Production Maestro YAML now uses the typed program, immutable replay plan, and direct +runtime port exclusively. The `SessionAction` lowering path, private `__maestro*` commands, +`replayControl`, hidden compatibility caches, positional decoders, and their fallback routing were +deleted in the same change. Generic `.ad` replay remains independent. + +1. Add the typed program, runtime port, direct interpreter, and normalized upstream fixtures without + changing production routing. +2. Differentially compare the old lowering path and direct engine at the typed operation boundary. +3. Move lifecycle, input, screenshot, and keyboard commands. +4. Move target queries and assertions, deleting unverified fast-path success and assertion-triggered + action replay. +5. Move single-pointer swipes through ADR 0013's normalized input boundary. +6. Move hooks, includes, conditions, repeat/retry, and trusted `runScript`. +7. Switch `--maestro` atomically, then delete private Maestro commands, positional decoding, hidden + caches, and obsolete converter/runtime tests. + +The old and new engines may coexist only in tests during migration. Shipping two production engines or +a runtime fallback between them is rejected because it doubles semantic and performance ownership. + +## Consequences + +- Maestro remains a supported subset with explicit failures; this refactor does not expand parity. +- Source provenance and runtime values stay typed through execution. +- Compatibility policy remains local while device behavior stays in shared runtimes and backends. +- Cross-platform correctness may require richer provider query evidence, but not additional round trips. +- ADR 0013 can be rewritten internally without changing Maestro as long as its normalized + single-pointer boundary and executor guarantees remain available. + +## Alternatives Considered + +- Embed upstream Orchestra: rejected because Java startup, package weight, driver ownership, and + platform coverage would erase performance and backend advantages. +- Build a shared replay VM first: rejected until native `.ad` needs structured runtime control flow; + one caller does not justify a broader abstraction. +- Keep compiling to typed `SessionAction` variants: rejected because it retains the replay trampoline + and prevents the compatibility engine from owning observation generations directly. diff --git a/docs/adr/README.md b/docs/adr/README.md index 8d1170330..1caeab1dc 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -16,6 +16,7 @@ | [0012 Interactive Replay](0012-interactive-replay.md) | replay healing/`--update`, diagnostic resolution disclosure, bounded `.ad` target-binding evidence, bounded divergence wire/error handling, plan-bound replay-only `--from` semantics, and agent-supervised re-record repair ("heal-by-doing") | | [0013 Unified Gesture Plans](0013-unified-gesture-plans.md) | gesture API/routing, contact topology, multi-touch geometry, native pointer injection, two-finger pan | | [0014 Session Ref-Frame Lifetime](0014-session-ref-frame-lifetime.md) | ref authorization epochs, complete/partial issuance, pre-side-effect expiration, replay/batch compatibility, and cross-platform stale-mutation policy | +| [0015 Direct Maestro Compatibility Engine](0015-direct-maestro-engine.md) | Maestro YAML parsing/execution, compatibility observation policy, conformance, performance gates, gesture integration | ADRs record *why*; the registries and gates they describe are the living source of truth — when prose and a registry disagree, the registry wins and the ADR needs a follow-up. diff --git a/docs/maestro-compat-debt-map.md b/docs/maestro-compat-debt-map.md index d063cef97..194fe015d 100644 --- a/docs/maestro-compat-debt-map.md +++ b/docs/maestro-compat-debt-map.md @@ -1,22 +1,36 @@ -# Maestro Compatibility Debt Map +# Maestro Compatibility Architecture -This map summarizes the Maestro compatibility surface after the lane 6 audit. LOC values are approximate and come from `wc -l` on the owning files; rows that share a file use an estimated slice. The current compat implementation is about 4.1k LOC under `src/compat/maestro`, with about 2.1k LOC of focused compat unit tests plus daemon/provider integration guards. +This map describes the direct compatibility engine after ADR 0015. Maestro YAML has one production +route: typed parse, immutable plan compilation, typed execution, and a runtime port backed by public +agent-device commands. There is no `SessionAction` lowering, private command namespace, runtime +fallback, or second compatibility engine. -| Area | Owning files | Approx LOC/code size | Custom handling summary | Native overlap and dependencies | Reliability or faster convergence opportunity | Risk | Test guard status | Recommendation | Suggested PR lane | -| --- | --- | ---: | --- | --- | --- | --- | --- | --- | --- | -| Parser and command mapping | `src/compat/maestro/replay-flow.ts`, `command-mapper.ts`, `support.ts`, `types.ts`, plus parse-side slices of `interactions.ts`, `device-actions.ts`, `flow-control.ts`, `run-script.ts` | ~1.5k LOC | Parses multi-document YAML, splits config/commands, resolves env, tracks rough top-level line numbers, flattens hooks, maps commands to `SessionAction`, rejects unsupported commands/fields loudly, emits `replayControl` for flow blocks/retry, and emits `__maestro*` runtime commands for remaining semantics that native replay does not model. | Depends on `yaml`, replay `SessionAction`, `AppError`, and native commands such as `open`, `click`, `fill`, `type`, `wait`, `is`, `scroll`, `swipe`, `keyboard`, and `screenshot`. | Common replay parsing helpers could own env precedence, source-path handling, and line diagnostics. Native command builders could reduce stringly `SessionAction` construction for commands that already match `.ad`. | Line mapping is intentionally approximate for nested lists; parse-time expansion can hide runtime structure; unsupported syntax must keep failing loudly. | Strong: `replay-flow.test.ts` covers supported subset, env, hooks, runFlow, repeat, retry, launch reset, unsupported fields, fixture parsing; `replay-input.test.ts` covers backend routing/env precedence. | Keep Maestro YAML grammar local. Converge shared replay env/source/line utilities and typed action construction where native commands already exist. | Lane 7: parser contract hardening | -| Runtime target matching | `src/compat/maestro/runtime-targets.ts` | 926 LOC | Implements Maestro-specific selector resolution over raw snapshots: id/label/text/value equality-or-regex matching, visible text fuzzy fallback, `childOf`, `index`, visible-only filtering, React Native overlay blocking, foreground duplicate preference, Android rectless hidden navigation handling, tab-strip slot inference, localized breadcrumb selection, and tap-target ancestor promotion. | Uses native selector parsing/matching (`parseSelectorChain`, `matchesSelector`), native visible predicates, snapshot text normalization, and React Native overlay detection. | Extract a reusable snapshot target resolver with policy hooks for ranking, visibility, overlay filtering, and promotion. Native `click`/`wait` could then opt into the generic parts without inheriting Maestro ranking. | Highest debt concentration. Heuristics encode real platform quirks and can regress seemingly unrelated flows, especially Android duplicate/hidden nodes and broad containers. | Strong: `runtime-targets.test.ts` is large and focused; PR #620 adds provider-level Android fresh-snapshot guard through Maestro replay. | Split generic snapshot traversal/filtering from Maestro ranking policy. Keep fuzzy/regex/read-order compatibility local. | Lane 7: target resolver extraction | -| Input and focus | `replay-flow.ts` input coalescing, `interactions.ts` input/erase/paste/pressKey slices, `runtime.ts`, daemon Maestro fallback in `interaction-touch.ts` | ~350 LOC plus shared daemon flags | Coalesces `tapOn` + `inputText` + `pressKey Enter` into `wait`/`fill`/enter for likely text-entry selectors, leaves focused-field input as `type`, maps `eraseText` to backspaces, falls back from keyboard enter to newline typing, and allows Maestro non-hittable coordinate fallback for tap selectors. | Uses native `fill`, `type`, `keyboard`, `click`, replay variable substitution, and daemon touch fallback flags. | A native "focus then fill" replay primitive and focused-field clear/erase operation would remove most parser heuristics while improving `.ad` flows too. | Text-entry detection is name-based; focused-field commands depend on existing device focus; non-hittable coordinate fallback is intentionally compatibility-only and can mask poor selectors. | Moderate: parser coalescing tests in `replay-flow.test.ts`; daemon fallback covered in `interaction.test.ts`; no broad provider guard for focused erase/paste. | Converge on native focus/fill/clear semantics. Keep Maestro's optional/non-hittable quirks as compat flags. | Lane 8: native input primitive | -| Assertions and waits | `runtime-assertions.ts`, wait/assert slices of `interactions.ts`, `runtime-flow.ts` visible condition path | ~450 LOC | Polls raw snapshots for `assertVisible`, adds a terminal grace capture, treats `assertNotVisible` as stable hidden after repeated samples or timeout, computes animation stability from snapshot signatures, maps `extendedWaitUntil`, and waits briefly for `runFlow.when.visible` while keeping `notVisible` point-in-time. | Uses native `snapshot`, `wait`, `is`, visible predicates, reference frames, replay blocks, and timeout helpers. | A shared waiter/poller utility with explicit grace, stable-hidden, raw-snapshot, and timeout policies would let native `wait` and Maestro share mechanics without sharing defaults. | Timeout and stability semantics are subtle; raw full snapshots are expensive; making `notVisible` wait would change cleanup-flow behavior. | Good: `runtime-assertions.test.ts` covers deadline/hidden edge cases; `runtime-flow.test.ts` covers visible wait and immediate notVisible. | Extract generic polling/stability helpers; keep Maestro timing constants and condition semantics local. | Lane 8: waiter convergence | -| Scroll, swipe, and geometry | `points.ts`, `runtime-geometry.ts`, geometry/scroll/swipe slices of `interactions.ts` and `runtime-interactions.ts` | ~500 LOC | Parses absolute and percentage points, converts authored percentage swipes through the shared core frame-bounds helper, caches reference frames in replay scope, delegates directional screen swipes to core gesture presets, biases tap points for large text containers and bottom tabs, loops `scrollUntilVisible` with `wait`/`find` probes, and maps `swipe.label`. | Uses native `click`, `gesture`, `scroll`, `swipe`, `find`, `wait`, touch reference frames, and raw snapshots. | Target-derived swipes and frame caching could move into native gesture planning when non-Maestro callers need them. | Stale or missing raw snapshot frames break percent gestures; target-derived swipe geometry and tap bias remain compatibility heuristics. | Good: `runtime-geometry.test.ts`, `runtime-interactions.test.ts`; the provider guard asserts fresh snapshots and exact authored Android percentage coordinates. | Keep authored coordinates lossless and route semantic gestures through core planners. Extract target geometry or frame caching only with a second native caller. | Lane 8 or 9: gesture planner | -| Flow control, `runFlow`, retry, and hooks | `flow-control.ts`, `runtime-flow.ts`, hook flattening in `replay-flow.ts` | ~700 LOC | Handles `onFlowStart`/`onFlowComplete`, file and inline `runFlow`, per-block env, static platform gates, limited `when.true` boolean/platform expressions, runtime visible/notVisible gates through `SessionAction.replayControl`, parse-expanded `repeat.times`, and runtime `retry` via replay retry blocks. | Depends on replay block runtime (`invokeReplayActionBlock`, `invokeReplayRetryBlock`), replay vars/env, typed `SessionReplayControl`, and native snapshot visibility resolution. | Flow block execution is now mostly converged. A native replay block AST would only be worth it if native `.ad` syntax also needs conditional blocks, deterministic repeats, or faithful nested line reporting. | `repeat.times` materializes actions with a guardrail; nested line numbers are lossy; expression support is intentionally tiny; visible conditions are snapshot-dependent; compat flow controls are intentionally in-memory (a `retry`/`runFlow.when` block is one entry in the replay-resume executable plan, never individually addressable by `replay --from`, and ADR 0012 migration step 5's resume preflight rejects skipping into or resuming at one). | Strong: `replay-flow.test.ts` covers hooks/runFlow/env/repeat/retry/platform gates/expressions; `runtime-flow.test.ts` covers runtime condition behavior; daemon replay tests guard execution and resume's control-flow rejection; PR #620 covers provider retry/runFlow path. | Keep `replayControl` as the shared runtime boundary. Do not add a native block AST unless native replay gains its own block syntax; keep Maestro expression grammar and unsupported `repeat.while` local until native runtime has loop semantics. | Lane 7 complete / revisit only with native block syntax | -| `runScript` | `run-script.ts`, `runtime.ts` runScript branch | 229 LOC plus router branch | Executes trusted flow-local JavaScript with `node:vm`, exposes env values, `output`, `json`, and synchronous-looking `http.post` through a timeout-bounded child Node process; exports output as `output.` replay variables. | Uses replay variable scope, `runCmdSync`, and `AppError`; there is no native `.ad` command equivalent. | Shared env/output variable plumbing could converge, but script execution should not become native without a separate security model and product decision. | High security and determinism risk: `node:vm` is not a sandbox, scripts can make network requests, async support is intentionally narrow, and output key rules are compatibility-specific. | Partial: parser/order/env/path behavior covered in `replay-flow.test.ts`; docs describe trust/security limits; no focused execution tests for `http.post`, `json`, or output validation. | Keep compat-local. Add focused execution tests before expanding helpers; do not expose as native command until sandboxing and trust model are explicit. | Lane 9: runScript guard tests | -| Suite discovery and test artifacts | `src/daemon/handlers/session-test-discovery.ts`, `session-test-suite.ts`, `session-test-artifacts.ts`, replay grammar/backend plumbing | ~120 Maestro-specific LOC in daemon/test path | When `--maestro`/backend is selected, discovers `.ad`, `.yaml`, and `.yml`, allows untyped YAML through platform filtering, runs through native replay test suite, and preserves original Maestro flow filenames in artifacts. | Native test suite runner, replay backend selection, session lifecycle, artifact materialization, and CLI `--maestro` flag. | Mostly converged already. The remaining improvement is making replay backend extension/filter policy data-driven so future backends avoid daemon conditionals. | Low. Main risk is accidentally including non-Maestro YAML when backend is set, or changing platform-filter behavior for untyped YAML. | Good: `session-test-discovery.test.ts`, `session-test-suite.test.ts`, `session-test-artifacts.test.ts`; PR #620 provider suite runs a Maestro YAML test. | Keep small daemon hook for now; extract backend discovery policy only if another replay backend appears. | Lane 6 complete / no follow-up unless backend count grows | -| Docs and support matrix | `support-matrix.ts`, `website/docs/docs/replay-e2e.md`, CLI flag/help tests, issue tracker references | 42 LOC source matrix plus docs | Maintains supported/unsupported capability lists, formats CLI help copy, links tracker/new issue URLs, and keeps replay docs synced with the source matrix. | CLI flag definitions/help rendering and website docs. | Generate or embed the support list from one source in docs/help to remove manual prose drift. | Medium user-facing risk: stale docs can imply unsupported parity or hide known gaps. | Good: `support-matrix.test.ts` asserts CLI help and docs stay synced with the shared matrix. | Keep `support-matrix.ts` as source of truth; update it with every behavior change and mirror only explanatory context in docs. | Lane 6 complete / ongoing docs hygiene | +| Area | Owners | Shared boundary | Remaining risk | +| --- | --- | --- | --- | +| Program parsing | `program-ir*.ts`, `program-loader.ts` | Source-preserving typed IR | The supported Maestro subset must reject unknown syntax explicitly. Keep parser modules focused as the grammar grows. | +| Plan and resume | `replay-plan*.ts`, `replay-plan-digest.ts` | Immutable expanded plan and digest | Runtime control steps are opaque resume boundaries. Includes, static conditions, and environment inputs must remain digest-bound. | +| Execution | `engine*.ts`, `runtime-port*.ts` | Typed commands, observation generations, runtime port | Mutation must invalidate observation evidence. Scoped and output variables must not leak across flow boundaries. | +| Daemon binding | `daemon-runtime-port*.ts` | Typed calls to public daemon commands | Reuse same-generation observation snapshots only as semantic evidence for atomic iOS selector dispatch, never as coordinate geometry. Otherwise preserve one fresh target snapshot, structured target retries, and platform-independent command semantics. Never restore private compatibility dispatch or re-enter Maestro parsing. | +| Target policy | `runtime-target*.ts` | Shared snapshot model with Maestro ranking policy | Exact-or-regex selector behavior, visibility, duplicate ranking, ancestor promotion, and platform quirks are compatibility policy. Fuzzy substring fallback is intentionally absent. | +| Gestures | `runtime-port-geometry*.ts` | ADR 0013 normalized single-pointer input | Preserve authored absolute/percentage endpoints exactly. Maestro does not own multi-touch planning or injection. | +| Failure and resume reporting | `session-replay-maestro-*.ts` | ADR 0012 `REPLAY_DIVERGENCE` wire contract | Source paths, step ordinals, artifacts, scrubbed variables, screen digest, and typed resume preflight must survive nested control failures. | +| Trusted scripts | `run-script-*.ts` | Typed environment/output maps | `node:vm` is not a security sandbox. Keep execution flow-local and do not expand trust without a separate product decision. | +| Suite integration | `session-test-*.ts`, replay backend selection | Existing test lifecycle and artifacts | YAML discovery must remain backend-scoped; `.ad` continues through generic replay even inside a Maestro suite. | -## Convergence Priority +## Deliberately Removed -1. Target matching is the largest and riskiest debt. Extract reusable snapshot traversal/filtering first, then keep Maestro ranking rules as a policy. -2. Replay control-flow is partly converged through `SessionAction.replayControl`; avoid a broader native block AST until native replay has its own block syntax or needs faithful nested line reporting. -3. Input/focus and wait semantics are good candidates for native primitives because they improve `.ad` replay as well as Maestro compatibility. -4. `runScript` should remain compatibility-local unless a separate secure native script story is designed. +- Generic `SessionAction` conversion and `replayControl` wrappers. +- Private `__maestro*` command routing and positional JSON decoding. +- WeakMap compatibility state and action-after-assertion replay. +- `wait`/`find` fallback chains after coordinate input. +- Selector-name heuristics for text-entry coalescing. +- Fuzzy substring matching that broadened plain Maestro text selectors. +- Cached gesture frames; percentage gestures use fresh shared viewport evidence. + +## Convergence Rules + +1. Compatibility aliases normalize into shared public commands; they do not clone platform input. +2. Single-pointer target and viewport plans stay distinct from ADR 0013 multi-touch plans. +3. Retry policy is typed and budgeted. Infrastructure failures propagate unless a structured error is explicitly retriable. +4. Performance comparisons count provider queries, captures, retries, hierarchy bytes, and warm latency, not only daemon round trips. +5. New parity behavior needs a typed-IR fixture, an engine/runtime-port test, and live Android+iOS evidence when device behavior is involved. diff --git a/package.json b/package.json index 0dbfacb46..4cce16271 100644 --- a/package.json +++ b/package.json @@ -103,6 +103,7 @@ "build:all": "pnpm build:node && pnpm build:xcuitest", "ad": "node bin/agent-device.mjs", "bench:help-conformance": "node scripts/help-conformance-bench.mjs", + "maestro:conformance": "node --experimental-strip-types scripts/maestro-conformance.ts", "size": "node scripts/size-report.mjs", "size:markdown": "node scripts/size-report.mjs --json .tmp/size-report.json --markdown .tmp/size-report.md", "perf": "node --experimental-strip-types scripts/perf/run.ts", diff --git a/scripts/maestro-conformance-fixtures/README.md b/scripts/maestro-conformance-fixtures/README.md new file mode 100644 index 000000000..6ea6bd2fb --- /dev/null +++ b/scripts/maestro-conformance-fixtures/README.md @@ -0,0 +1,23 @@ +# Maestro Conformance Fixtures + +The harness compares the supported `agent-device` Maestro flow model with a +small, checked-in capture of the Maestro 2.5.1 command model. The upstream tag, +commit, source paths, and SHA-256 values live in +`upstream-maestro-2.5.1.json`. + +The default check is deterministic and does not need Java or Maestro: + +```sh +node --experimental-strip-types scripts/maestro-conformance.ts +``` + +The raw capture is normalized into `normalized-maestro-2.5.1.json`. Regenerate +that file only after reviewing an intentional upstream capture update: + +```sh +node --experimental-strip-types scripts/maestro-conformance.ts --regenerate +``` + +The capture includes fixture source locations so the comparison also protects +`runFlow` include provenance. Timestamps and internal action names are removed +before comparison. diff --git a/scripts/maestro-conformance-fixtures/launch-defaults.yaml b/scripts/maestro-conformance-fixtures/launch-defaults.yaml new file mode 100644 index 000000000..0bf925311 --- /dev/null +++ b/scripts/maestro-conformance-fixtures/launch-defaults.yaml @@ -0,0 +1,3 @@ +appId: com.example.pager +--- +- launchApp diff --git a/scripts/maestro-conformance-fixtures/normalized-maestro-2.5.1.json b/scripts/maestro-conformance-fixtures/normalized-maestro-2.5.1.json new file mode 100644 index 000000000..710900214 --- /dev/null +++ b/scripts/maestro-conformance-fixtures/normalized-maestro-2.5.1.json @@ -0,0 +1,173 @@ +{ + "schemaVersion": 1, + "upstream": { + "project": "mobile-dev-inc/Maestro", + "version": "2.5.1", + "tag": "v2.5.1", + "commit": "a4c7c95f5ba1884858f7e35efa6b8e0165db9448", + "sourceUrl": "https://github.com/mobile-dev-inc/Maestro/tree/v2.5.1", + "artifacts": [ + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlSwipe.kt", + "role": "YAML swipe classification and percentage endpoint parsing", + "sha256": "1ad0e1c8d80edabc1b5de9390bc64d29117c1d5eff2016faef9f8cf6b9d728e3" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlFluentCommand.kt", + "role": "YAML command to Maestro command-model conversion", + "sha256": "6ab05c3932040e20dffa0a683ce0eecb9241e0c9b7bcc0188b697c196797c01d" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlLaunchApp.kt", + "role": "launchApp scalar/map decoding", + "sha256": "2e656607e44acb51969afcafe6d75316cf2586f6e0614696f08cab3719f6b615" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlRunFlow.kt", + "role": "runFlow file/inline command shape", + "sha256": "a1c50cbd18fd329e9ca91751f324b1caf07aff33616cc4de7ddecaabd3fda66e" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlElementSelector.kt", + "role": "selector YAML fields", + "sha256": "907b0c8298d2cdc0a369937f1cc0f1787dfe195a82de6f8b2c46f28d0571446b" + }, + { + "path": "maestro-orchestra-models/src/main/java/maestro/orchestra/Commands.kt", + "role": "LaunchAppCommand, SwipeCommand, and RunFlowCommand model defaults", + "sha256": "64c98d657b649e90e12c0246bfe18465bf2691233f11884ca975f955afbd3836" + }, + { + "path": "maestro-orchestra-models/src/main/java/maestro/orchestra/ElementSelector.kt", + "role": "normalized selector model", + "sha256": "b8e97a99806d992cb02d9edc33f258af7404e33dad9e9ce4786bda86d81171a9" + } + ] + }, + "cases": [ + { + "id": "pager-percentage-swipe", + "flow": "pager-percentage-swipe.yaml", + "expected": [ + { + "kind": "swipe", + "mode": "relative", + "start": [ + 90, + 50 + ], + "end": [ + 10, + 50 + ], + "durationMs": 400, + "source": { + "path": "pager-percentage-swipe.yaml", + "line": 3 + } + } + ] + }, + { + "id": "launch-defaults", + "flow": "launch-defaults.yaml", + "expected": [ + { + "kind": "launchApp", + "appId": "com.example.pager", + "stopApp": true, + "source": { + "path": "launch-defaults.yaml", + "line": 3 + } + } + ] + }, + { + "id": "selectors", + "flow": "selectors.yaml", + "expected": [ + { + "kind": "tapOn", + "selector": { + "text": "Open details" + }, + "source": { + "path": "selectors.yaml", + "line": 3 + } + }, + { + "kind": "tapOn", + "selector": { + "id": "pager-next", + "index": 1, + "childOf": { + "id": "pager" + } + }, + "source": { + "path": "selectors.yaml", + "line": 4 + } + }, + { + "kind": "tapOn", + "selector": { + "text": "Ready", + "enabled": false + }, + "source": { + "path": "selectors.yaml", + "line": 9 + } + } + ] + }, + { + "id": "runflow-include-provenance", + "flow": "runflow-main.yaml", + "expected": [ + { + "kind": "tapOn", + "selector": { + "text": "Before" + }, + "source": { + "path": "runflow-main.yaml", + "line": 3 + } + }, + { + "kind": "launchApp", + "appId": "com.example.include", + "stopApp": true, + "source": { + "path": "runflow-child.yaml", + "line": 3 + } + }, + { + "kind": "tapOn", + "selector": { + "id": "included-button" + }, + "source": { + "path": "runflow-child.yaml", + "line": 4 + } + }, + { + "kind": "tapOn", + "selector": { + "text": "After" + }, + "source": { + "path": "runflow-main.yaml", + "line": 6 + } + } + ] + } + ] +} diff --git a/scripts/maestro-conformance-fixtures/pager-percentage-swipe.yaml b/scripts/maestro-conformance-fixtures/pager-percentage-swipe.yaml new file mode 100644 index 000000000..d5f89d32e --- /dev/null +++ b/scripts/maestro-conformance-fixtures/pager-percentage-swipe.yaml @@ -0,0 +1,7 @@ +appId: com.example.pager +--- +- swipe: + from: + id: pager-view + start: 90%, 50% + end: 10%, 50% diff --git a/scripts/maestro-conformance-fixtures/runflow-child.yaml b/scripts/maestro-conformance-fixtures/runflow-child.yaml new file mode 100644 index 000000000..6060da132 --- /dev/null +++ b/scripts/maestro-conformance-fixtures/runflow-child.yaml @@ -0,0 +1,5 @@ +appId: com.example.include +--- +- launchApp +- tapOn: + id: included-button diff --git a/scripts/maestro-conformance-fixtures/runflow-main.yaml b/scripts/maestro-conformance-fixtures/runflow-main.yaml new file mode 100644 index 000000000..1a57ec86c --- /dev/null +++ b/scripts/maestro-conformance-fixtures/runflow-main.yaml @@ -0,0 +1,6 @@ +appId: com.example.include +--- +- tapOn: Before +- runFlow: + file: runflow-child.yaml +- tapOn: After diff --git a/scripts/maestro-conformance-fixtures/selectors.yaml b/scripts/maestro-conformance-fixtures/selectors.yaml new file mode 100644 index 000000000..ad647cf7c --- /dev/null +++ b/scripts/maestro-conformance-fixtures/selectors.yaml @@ -0,0 +1,11 @@ +appId: com.example.pager +--- +- tapOn: Open details +- tapOn: + id: pager-next + index: 1 + childOf: + id: pager +- tapOn: + text: Ready + enabled: false diff --git a/scripts/maestro-conformance-fixtures/upstream-maestro-2.5.1.json b/scripts/maestro-conformance-fixtures/upstream-maestro-2.5.1.json new file mode 100644 index 000000000..19df4dbbc --- /dev/null +++ b/scripts/maestro-conformance-fixtures/upstream-maestro-2.5.1.json @@ -0,0 +1,174 @@ +{ + "schemaVersion": 1, + "upstream": { + "project": "mobile-dev-inc/Maestro", + "version": "2.5.1", + "tag": "v2.5.1", + "commit": "a4c7c95f5ba1884858f7e35efa6b8e0165db9448", + "sourceUrl": "https://github.com/mobile-dev-inc/Maestro/tree/v2.5.1", + "artifacts": [ + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlSwipe.kt", + "role": "YAML swipe classification and percentage endpoint parsing", + "sha256": "1ad0e1c8d80edabc1b5de9390bc64d29117c1d5eff2016faef9f8cf6b9d728e3" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlFluentCommand.kt", + "role": "YAML command to Maestro command-model conversion", + "sha256": "6ab05c3932040e20dffa0a683ce0eecb9241e0c9b7bcc0188b697c196797c01d" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlLaunchApp.kt", + "role": "launchApp scalar/map decoding", + "sha256": "2e656607e44acb51969afcafe6d75316cf2586f6e0614696f08cab3719f6b615" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlRunFlow.kt", + "role": "runFlow file/inline command shape", + "sha256": "a1c50cbd18fd329e9ca91751f324b1caf07aff33616cc4de7ddecaabd3fda66e" + }, + { + "path": "maestro-orchestra/src/main/java/maestro/orchestra/yaml/YamlElementSelector.kt", + "role": "selector YAML fields", + "sha256": "907b0c8298d2cdc0a369937f1cc0f1787dfe195a82de6f8b2c46f28d0571446b" + }, + { + "path": "maestro-orchestra-models/src/main/java/maestro/orchestra/Commands.kt", + "role": "LaunchAppCommand, SwipeCommand, and RunFlowCommand model defaults", + "sha256": "64c98d657b649e90e12c0246bfe18465bf2691233f11884ca975f955afbd3836" + }, + { + "path": "maestro-orchestra-models/src/main/java/maestro/orchestra/ElementSelector.kt", + "role": "normalized selector model", + "sha256": "b8e97a99806d992cb02d9edc33f258af7404e33dad9e9ce4786bda86d81171a9" + } + ] + }, + "cases": [ + { + "id": "pager-percentage-swipe", + "flow": "pager-percentage-swipe.yaml", + "commands": [ + { + "type": "SwipeCommand", + "startRelative": "90%, 50%", + "endRelative": "10%, 50%", + "duration": 400, + "source": { + "path": "pager-percentage-swipe.yaml", + "line": 3 + } + } + ] + }, + { + "id": "launch-defaults", + "flow": "launch-defaults.yaml", + "commands": [ + { + "type": "LaunchAppCommand", + "appId": "com.example.pager", + "source": { + "path": "launch-defaults.yaml", + "line": 3 + } + } + ] + }, + { + "id": "selectors", + "flow": "selectors.yaml", + "commands": [ + { + "type": "TapOnElementCommand", + "selector": { + "textRegex": "Open details" + }, + "source": { + "path": "selectors.yaml", + "line": 3 + } + }, + { + "type": "TapOnElementCommand", + "selector": { + "idRegex": "pager-next", + "index": 1, + "childOf": { + "idRegex": "pager" + } + }, + "source": { + "path": "selectors.yaml", + "line": 4 + } + }, + { + "type": "TapOnElementCommand", + "selector": { + "textRegex": "Ready", + "enabled": false + }, + "source": { + "path": "selectors.yaml", + "line": 9 + } + } + ] + }, + { + "id": "runflow-include-provenance", + "flow": "runflow-main.yaml", + "commands": [ + { + "type": "TapOnElementCommand", + "selector": { + "textRegex": "Before" + }, + "source": { + "path": "runflow-main.yaml", + "line": 3 + } + }, + { + "type": "RunFlowCommand", + "sourceDescription": "runflow-child.yaml", + "source": { + "path": "runflow-main.yaml", + "line": 4 + }, + "commands": [ + { + "type": "LaunchAppCommand", + "appId": "com.example.include", + "source": { + "path": "runflow-child.yaml", + "line": 3 + } + }, + { + "type": "TapOnElementCommand", + "selector": { + "idRegex": "included-button" + }, + "source": { + "path": "runflow-child.yaml", + "line": 4 + } + } + ] + }, + { + "type": "TapOnElementCommand", + "selector": { + "textRegex": "After" + }, + "source": { + "path": "runflow-main.yaml", + "line": 6 + } + } + ] + } + ] +} diff --git a/scripts/maestro-conformance-model.ts b/scripts/maestro-conformance-model.ts new file mode 100644 index 000000000..13ca6f681 --- /dev/null +++ b/scripts/maestro-conformance-model.ts @@ -0,0 +1,347 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { parseMaestroProgram } from '../src/compat/maestro/program-ir-parser.ts'; +import type { + MaestroCommand, + MaestroProgram, + MaestroSelector, +} from '../src/compat/maestro/program-ir.ts'; +import { normalizeUpstreamSelector } from './maestro-conformance-selectors.ts'; +import type { + NormalizedAction, + NormalizedCase, + NormalizedFixture, + NormalizedSelector, + NormalizedSource, + RawCase, + RawCommand, + RawFixture, + UpstreamSource, +} from './maestro-conformance-types.ts'; +import { readOptionalString, readRequiredRecord } from './maestro-conformance-values.ts'; + +const DEFAULT_SWIPE_DURATION_MS = 400; + +export function normalizeUpstreamFixture( + fixture: RawFixture, + fixtureDirectory: string, +): NormalizedFixture { + return { + schemaVersion: 1, + upstream: fixture.upstream, + cases: fixture.cases.map((entry) => ({ + id: entry.id, + flow: entry.flow, + expected: entry.commands.flatMap((command) => + normalizeUpstreamCommand(command, fixtureDirectory, { + path: entry.flow, + line: 1, + }), + ), + })), + }; +} + +export function normalizeAgentCase(fixture: RawCase, fixtureDirectory: string): NormalizedCase { + const flowPath = resolveFixturePath(fixtureDirectory, fixture.flow); + const program = parseMaestroProgram(fs.readFileSync(flowPath, 'utf8'), { sourcePath: flowPath }); + return { + id: fixture.id, + flow: fixture.flow, + expected: normalizeAgentProgram(program, fixtureDirectory), + }; +} + +function normalizeAgentProgram( + program: MaestroProgram, + fixtureDirectory: string, +): NormalizedAction[] { + return [ + ...(program.config.onFlowStart ?? []), + ...program.commands, + ...(program.config.onFlowComplete ?? []), + ].flatMap((command) => normalizeAgentCommand(command, program, fixtureDirectory)); +} + +function normalizeAgentCommand( + command: MaestroCommand, + program: MaestroProgram, + fixtureDirectory: string, +): NormalizedAction[] { + const source = normalizeSource(command.source, fixtureDirectory); + switch (command.kind) { + case 'runFlow': + return normalizeAgentRunFlow(command, program, fixtureDirectory); + case 'launchApp': { + const appId = command.appId ?? program.config.appId; + if (!appId) throw new Error('launchApp conformance fixture requires appId.'); + return [{ kind: 'launchApp', appId, stopApp: command.stopApp !== false, source }]; + } + case 'swipe': + return [normalizeAgentSwipe(command.gesture, source)]; + case 'tapOn': + return [normalizeAgentTap(command, source)]; + case 'assertVisible': + case 'assertNotVisible': + return [ + { + kind: command.kind, + selector: normalizeTypedSelector(command.target), + timeoutMs: 17_000, + source, + }, + ]; + default: + throw new Error(`Unsupported typed command in conformance fixture: ${command.kind}`); + } +} + +function normalizeAgentRunFlow( + command: Extract, + program: MaestroProgram, + fixtureDirectory: string, +): NormalizedAction[] { + if (command.include.kind === 'commands') { + return command.include.commands.flatMap((nested) => + normalizeAgentCommand(nested, program, fixtureDirectory), + ); + } + const parentPath = command.source.path ?? program.source.path; + if (!parentPath) throw new Error('File runFlow requires source path provenance.'); + const includePath = path.resolve(path.dirname(parentPath), command.include.path); + const included = parseMaestroProgram(fs.readFileSync(includePath, 'utf8'), { + sourcePath: includePath, + }); + return normalizeAgentProgram(included, fixtureDirectory); +} + +function normalizeAgentTap( + command: Extract, + source: NormalizedSource, +): NormalizedAction { + if (command.target.space !== 'target') { + throw new Error('tapOn conformance fixtures require selector targets.'); + } + return { + kind: 'tapOn', + selector: { + ...normalizeTypedSelector(command.target.selector), + ...(command.index === undefined ? {} : { index: command.index }), + ...(command.childOf === undefined + ? {} + : { childOf: normalizeTypedSelector(command.childOf) }), + }, + source, + }; +} + +function normalizeAgentSwipe( + gesture: Extract['gesture'], + source: NormalizedSource, +): NormalizedAction { + const durationMs = gesture.duration ?? DEFAULT_SWIPE_DURATION_MS; + if (gesture.kind === 'screen') { + return { kind: 'swipe', mode: 'direction', direction: gesture.direction, durationMs, source }; + } + if (gesture.kind === 'target') { + throw new Error('Target-relative swipe is not part of the conformance fixture set.'); + } + return { + kind: 'swipe', + mode: gesture.start.space === 'percent' ? 'relative' : 'absolute', + start: [gesture.start.x, gesture.start.y], + end: [gesture.end.x, gesture.end.y], + durationMs, + source, + }; +} + +function normalizeTypedSelector(selector: MaestroSelector): NormalizedSelector { + const text = selector.text ?? selector.label; + return { + ...(selector.id === undefined ? {} : { id: selector.id }), + ...(text === undefined ? {} : { text }), + ...(selector.enabled === undefined ? {} : { enabled: selector.enabled }), + ...(selector.selected === undefined ? {} : { selected: selector.selected }), + }; +} + +function normalizeUpstreamCommand( + command: RawCommand, + fixtureDirectory: string, + fallbackSource: UpstreamSource, +): NormalizedAction[] { + const source = normalizeSource(command.source ?? fallbackSource, fixtureDirectory); + + switch (command.type) { + case 'RunFlowCommand': { + if (!command.commands) throw new Error('RunFlowCommand artifact is missing commands.'); + return command.commands.flatMap((nested) => + normalizeUpstreamCommand(nested, fixtureDirectory, source), + ); + } + case 'LaunchAppCommand': + return [ + { + kind: 'launchApp', + appId: requiredString(command, 'appId'), + stopApp: command.stopApp !== false, + source, + }, + ]; + case 'SwipeCommand': + return [normalizeUpstreamSwipe(command, source)]; + case 'TapOnElementCommand': + return [ + { + kind: 'tapOn', + selector: normalizeUpstreamSelector(requiredRecord(command, 'selector')), + source, + }, + ]; + case 'AssertConditionCommand': + return [normalizeUpstreamAssertion(command, source)]; + default: + throw new Error(`Unsupported upstream command artifact: ${command.type}`); + } +} + +function normalizeUpstreamSwipe(command: RawCommand, source: NormalizedSource): NormalizedAction { + const durationMs = integerOrDefault(command.duration, DEFAULT_SWIPE_DURATION_MS); + const startRelative = readOptionalString(command, 'startRelative'); + const endRelative = readOptionalString(command, 'endRelative'); + if (startRelative !== undefined || endRelative !== undefined) { + if (startRelative === undefined || endRelative === undefined) { + throw new Error('SwipeCommand artifact must include both relative endpoints.'); + } + return { + kind: 'swipe', + mode: 'relative', + start: parsePoint(startRelative, '%'), + end: parsePoint(endRelative, '%'), + durationMs, + source, + }; + } + + const direction = readOptionalString(command, 'direction'); + if (direction !== undefined) { + return { + kind: 'swipe', + mode: 'direction', + direction: direction.toLowerCase(), + durationMs, + source, + }; + } + + const startPoint = optionalPoint(command, 'startPoint'); + const endPoint = optionalPoint(command, 'endPoint'); + if (startPoint && endPoint) { + return { + kind: 'swipe', + mode: 'absolute', + start: startPoint, + end: endPoint, + durationMs, + source, + }; + } + throw new Error('SwipeCommand artifact has no supported gesture shape.'); +} + +function normalizeUpstreamAssertion( + command: RawCommand, + source: NormalizedSource, +): NormalizedAction { + const condition = requiredRecord(command, 'condition'); + const visible = condition.visible; + const notVisible = condition.notVisible; + if (visible !== undefined && notVisible === undefined) { + return { + kind: 'assertVisible', + selector: normalizeUpstreamSelector(requiredRecordValue(visible, 'condition.visible')), + timeoutMs: integerOrDefault(command.timeout, 17000), + source, + }; + } + if (notVisible !== undefined && visible === undefined) { + return { + kind: 'assertNotVisible', + selector: normalizeUpstreamSelector(requiredRecordValue(notVisible, 'condition.notVisible')), + timeoutMs: integerOrDefault(command.timeout, 17000), + source, + }; + } + throw new Error('AssertConditionCommand artifact must contain one condition.'); +} + +function optionalPoint(record: Record, key: string): [number, number] | undefined { + const value = record[key]; + if (value === undefined) return undefined; + if (Array.isArray(value) && value.length === 2) { + return [numberValue(value[0], `${key}[0]`), numberValue(value[1], `${key}[1]`)] as [ + number, + number, + ]; + } + if (typeof value === 'string') return parsePoint(value, ''); + throw new Error(`Unsupported ${key} point artifact.`); +} + +function parsePoint(value: string, suffix: string): [number, number] { + const escapedSuffix = suffix === '%' ? '%' : ''; + const match = value.match( + new RegExp( + `^\\s*(\\d+(?:\\.\\d+)?)${escapedSuffix}\\s*,\\s*(\\d+(?:\\.\\d+)?)${escapedSuffix}\\s*$`, + ), + ); + if (!match) throw new Error(`Invalid ${suffix ? 'relative ' : ''}point: ${value}`); + return [Number(match[1]), Number(match[2])]; +} + +function integerOrDefault(value: unknown, fallback: number): number { + if (value === undefined || value === null || value === '') return fallback; + const number = typeof value === 'number' ? value : Number(value); + if (!Number.isInteger(number) || number < 0) + throw new Error(`Expected non-negative integer, got ${value}`); + return number; +} + +function numberValue(value: unknown, name: string): number { + const number = typeof value === 'number' ? value : Number(value); + if (!Number.isFinite(number)) throw new Error(`Invalid ${name}: ${String(value)}`); + return number; +} + +function requiredString(record: Record, key: string): string { + const value = readOptionalString(record, key); + if (value === undefined || value.length === 0) throw new Error(`${key} is required.`); + return value; +} + +function requiredRecord(record: Record, key: string): Record { + return requiredRecordValue(record[key], key); +} + +function requiredRecordValue(value: unknown, name: string): Record { + return readRequiredRecord(value, name); +} + +function resolveFixturePath(fixtureDirectory: string, relativePath: string): string { + const resolved = path.resolve(fixtureDirectory, relativePath); + const relative = path.relative(fixtureDirectory, resolved); + if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) { + throw new Error(`Fixture path escapes fixture directory: ${relativePath}`); + } + return resolved; +} + +function normalizeSource(source: UpstreamSource, fixtureDirectory: string): NormalizedSource { + const resolved = path.resolve(fixtureDirectory, source.path); + const relative = path.relative(fixtureDirectory, resolved); + if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) { + throw new Error(`Source path escapes fixture directory: ${source.path}`); + } + return { path: relative.split(path.sep).join('/'), line: source.line }; +} diff --git a/scripts/maestro-conformance-selectors.ts b/scripts/maestro-conformance-selectors.ts new file mode 100644 index 000000000..f373d62ff --- /dev/null +++ b/scripts/maestro-conformance-selectors.ts @@ -0,0 +1,38 @@ +import type { NormalizedSelector } from './maestro-conformance-types.ts'; +import { readOptionalString, readRequiredRecord } from './maestro-conformance-values.ts'; + +export function normalizeUpstreamSelector(selector: Record): NormalizedSelector { + const result: NormalizedSelector = {}; + const text = readOptionalString(selector, 'textRegex') ?? readOptionalString(selector, 'text'); + const id = readOptionalString(selector, 'idRegex') ?? readOptionalString(selector, 'id'); + if (text !== undefined) result.text = text; + if (id !== undefined) result.id = id; + if (selector.index !== undefined) result.index = numberValue(selector.index, 'selector index'); + if (selector.childOf !== undefined) { + result.childOf = normalizeUpstreamSelector( + readRequiredRecord(selector.childOf, 'selector.childOf'), + ); + } + readSelectorState(selector, result, 'enabled'); + readSelectorState(selector, result, 'selected'); + if (Object.keys(result).length === 0) throw new Error('Selector artifact is empty.'); + return result; +} + +function readSelectorState( + record: Record, + result: NormalizedSelector, + key: 'enabled' | 'selected', +): void { + const value = record[key]; + if (value === undefined) return; + if (typeof value !== 'boolean') throw new Error(`Selector ${key} must be boolean.`); + result[key] = value; +} + +function numberValue(value: unknown, name: string): number { + const number = typeof value === 'number' ? value : Number(value); + if (!Number.isInteger(number) || number < 0) + throw new Error(`${name} must be a non-negative integer.`); + return number; +} diff --git a/scripts/maestro-conformance-types.ts b/scripts/maestro-conformance-types.ts new file mode 100644 index 000000000..e99e0e721 --- /dev/null +++ b/scripts/maestro-conformance-types.ts @@ -0,0 +1,84 @@ +export type UpstreamArtifact = { + path: string; + role: string; + sha256: string; +}; + +export type UpstreamPin = { + project: string; + version: string; + tag: string; + commit: string; + sourceUrl: string; + artifacts: UpstreamArtifact[]; +}; + +export type UpstreamSource = { + path: string; + line: number; +}; + +export type RawCommand = { + type: string; + source?: UpstreamSource; + commands?: RawCommand[]; + [key: string]: unknown; +}; + +export type RawCase = { + id: string; + flow: string; + commands: RawCommand[]; +}; + +export type RawFixture = { + schemaVersion: 1; + upstream: UpstreamPin; + cases: RawCase[]; +}; + +export type NormalizedSource = UpstreamSource; + +export type NormalizedSelector = { + id?: string; + text?: string; + index?: number; + childOf?: NormalizedSelector; + enabled?: boolean; + selected?: boolean; +}; + +export type NormalizedAction = + | { + kind: 'launchApp'; + appId: string; + stopApp: boolean; + source: NormalizedSource; + } + | { + kind: 'swipe'; + mode: 'relative' | 'absolute' | 'direction'; + start?: [number, number]; + end?: [number, number]; + direction?: string; + durationMs: number; + source: NormalizedSource; + } + | { + kind: 'tapOn' | 'assertVisible' | 'assertNotVisible'; + selector: NormalizedSelector; + timeoutMs?: number; + source: NormalizedSource; + }; + +export type NormalizedCase = { + id: string; + flow: string; + expected: NormalizedAction[]; +}; + +export type NormalizedFixture = { + schemaVersion: 1; + upstream: UpstreamPin; + cases: NormalizedCase[]; +}; diff --git a/scripts/maestro-conformance-values.ts b/scripts/maestro-conformance-values.ts new file mode 100644 index 000000000..8e68b1555 --- /dev/null +++ b/scripts/maestro-conformance-values.ts @@ -0,0 +1,16 @@ +export function readRequiredRecord(value: unknown, name: string): Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new Error(`${name} must be an object.`); + } + return value as Record; +} + +export function readOptionalString( + record: Record, + key: string, +): string | undefined { + const value = record[key]; + if (value === undefined || value === null) return undefined; + if (typeof value !== 'string') throw new Error(`${key} must be a string.`); + return value; +} diff --git a/scripts/maestro-conformance.test.ts b/scripts/maestro-conformance.test.ts new file mode 100644 index 000000000..7f862a2fe --- /dev/null +++ b/scripts/maestro-conformance.test.ts @@ -0,0 +1,91 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { test } from 'node:test'; +import { + MAESTRO_CONFORMANCE_FIXTURE_DIRECTORY, + MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE, + checkConformance, + regenerateConformance, +} from './maestro-conformance.ts'; + +test('checked-in Maestro 2.5.1 fixtures match the agent-device model', () => { + const result = checkConformance(); + + assert.equal(result.upstream.version, '2.5.1'); + assert.equal(result.upstream.commit, 'a4c7c95f5ba1884858f7e35efa6b8e0165db9448'); + assert.deepEqual( + result.cases.map((entry) => entry.id), + ['pager-percentage-swipe', 'launch-defaults', 'selectors', 'runflow-include-provenance'], + ); +}); + +test('normalization covers percentage swipes, launch defaults, selectors, and includes', () => { + const result = checkConformance(); + const pager = result.cases.find((entry) => entry.id === 'pager-percentage-swipe'); + const launch = result.cases.find((entry) => entry.id === 'launch-defaults'); + const selectors = result.cases.find((entry) => entry.id === 'selectors'); + const include = result.cases.find((entry) => entry.id === 'runflow-include-provenance'); + + assert.deepEqual(pager?.expected[0], { + kind: 'swipe', + mode: 'relative', + start: [90, 50], + end: [10, 50], + durationMs: 400, + source: { path: 'pager-percentage-swipe.yaml', line: 3 }, + }); + assert.deepEqual(launch?.expected[0], { + kind: 'launchApp', + appId: 'com.example.pager', + stopApp: true, + source: { path: 'launch-defaults.yaml', line: 3 }, + }); + assert.deepEqual( + selectors?.expected.map((entry) => entry), + [ + { + kind: 'tapOn', + selector: { text: 'Open details' }, + source: { path: 'selectors.yaml', line: 3 }, + }, + { + kind: 'tapOn', + selector: { id: 'pager-next', index: 1, childOf: { id: 'pager' } }, + source: { path: 'selectors.yaml', line: 4 }, + }, + { + kind: 'tapOn', + selector: { text: 'Ready', enabled: false }, + source: { path: 'selectors.yaml', line: 9 }, + }, + ], + ); + assert.deepEqual( + include?.expected.map((entry) => entry.source), + [ + { path: 'runflow-main.yaml', line: 3 }, + { path: 'runflow-child.yaml', line: 3 }, + { path: 'runflow-child.yaml', line: 4 }, + { path: 'runflow-main.yaml', line: 6 }, + ], + ); +}); + +test('regeneration is explicit and produces a checkable normalized fixture', () => { + const temporaryDirectory = fs.mkdtempSync(path.join(os.tmpdir(), 'maestro-conformance-')); + try { + fs.cpSync(MAESTRO_CONFORMANCE_FIXTURE_DIRECTORY, temporaryDirectory, { recursive: true }); + fs.writeFileSync( + path.join(temporaryDirectory, MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE), + '{"schemaVersion":1,"cases":[]}\n', + ); + + const regenerated = regenerateConformance({ fixtureDirectory: temporaryDirectory }); + assert.equal(regenerated.cases.length, 4); + assert.doesNotThrow(() => checkConformance({ fixtureDirectory: temporaryDirectory })); + } finally { + fs.rmSync(temporaryDirectory, { recursive: true, force: true }); + } +}); diff --git a/scripts/maestro-conformance.ts b/scripts/maestro-conformance.ts new file mode 100644 index 000000000..456da096c --- /dev/null +++ b/scripts/maestro-conformance.ts @@ -0,0 +1,255 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { normalizeAgentCase, normalizeUpstreamFixture } from './maestro-conformance-model.ts'; +import type { + NormalizedCase, + NormalizedFixture, + RawCase, + RawCommand, + RawFixture, + UpstreamArtifact, + UpstreamPin, + UpstreamSource, +} from './maestro-conformance-types.ts'; +import { readRequiredRecord } from './maestro-conformance-values.ts'; + +export const MAESTRO_CONFORMANCE_FIXTURE_DIRECTORY = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + 'maestro-conformance-fixtures', +); +export const MAESTRO_CONFORMANCE_RAW_FIXTURE = 'upstream-maestro-2.5.1.json'; +export const MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE = 'normalized-maestro-2.5.1.json'; +const PINNED_UPSTREAM = { + project: 'mobile-dev-inc/Maestro', + version: '2.5.1', + tag: 'v2.5.1', + commit: 'a4c7c95f5ba1884858f7e35efa6b8e0165db9448', +} as const; + +export type ConformanceCheckResult = { + upstream: UpstreamPin; + cases: NormalizedCase[]; +}; + +export function checkConformance( + options: { + fixtureDirectory?: string; + } = {}, +): ConformanceCheckResult { + const fixtureDirectory = options.fixtureDirectory ?? MAESTRO_CONFORMANCE_FIXTURE_DIRECTORY; + const raw = readRawFixture(path.join(fixtureDirectory, MAESTRO_CONFORMANCE_RAW_FIXTURE)); + const normalized = readNormalizedFixture( + path.join(fixtureDirectory, MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE), + ); + assertPinnedUpstream(raw.upstream); + compareJson(normalized.upstream, raw.upstream, 'normalized fixture upstream pin'); + + const regenerated = normalizeUpstreamFixture(raw, fixtureDirectory); + compareJson(regenerated.cases, normalized.cases, 'normalized upstream cases'); + + const actual = raw.cases.map((entry) => normalizeAgentCase(entry, fixtureDirectory)); + compareJson(actual, normalized.cases, 'agent-device flow model'); + return { upstream: raw.upstream, cases: actual }; +} + +export function regenerateConformance( + options: { + fixtureDirectory?: string; + } = {}, +): NormalizedFixture { + const fixtureDirectory = options.fixtureDirectory ?? MAESTRO_CONFORMANCE_FIXTURE_DIRECTORY; + const raw = readRawFixture(path.join(fixtureDirectory, MAESTRO_CONFORMANCE_RAW_FIXTURE)); + assertPinnedUpstream(raw.upstream); + const normalized = normalizeUpstreamFixture(raw, fixtureDirectory); + fs.writeFileSync( + path.join(fixtureDirectory, MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE), + `${JSON.stringify(normalized, null, 2)}\n`, + ); + return normalized; +} + +function readRawFixture(filePath: string): RawFixture { + const value = readJson(filePath); + const record = requiredRecord(value, filePath); + if (record.schemaVersion !== 1) throw new Error(`${filePath} has an unsupported schemaVersion.`); + const cases = requiredArray(record.cases, `${filePath}.cases`).map((entry, index) => + readRawCase(entry, `${filePath}.cases[${index}]`), + ); + return { + schemaVersion: 1, + upstream: readUpstreamPin(record.upstream, `${filePath}.upstream`), + cases, + }; +} + +function readNormalizedFixture(filePath: string): NormalizedFixture { + const value = readJson(filePath); + const record = requiredRecord(value, filePath); + if (record.schemaVersion !== 1) throw new Error(`${filePath} has an unsupported schemaVersion.`); + return { + schemaVersion: 1, + upstream: readUpstreamPin(record.upstream, `${filePath}.upstream`), + cases: requiredArray(record.cases, `${filePath}.cases`).map((entry, index) => { + const caseRecord = requiredRecord(entry, `${filePath}.cases[${index}]`); + return { + id: requiredString(caseRecord.id, `${filePath}.cases[${index}].id`), + flow: requiredString(caseRecord.flow, `${filePath}.cases[${index}].flow`), + expected: requiredArray( + caseRecord.expected, + `${filePath}.cases[${index}].expected`, + ) as NormalizedCase['expected'], + }; + }), + }; +} + +function readRawCase(value: unknown, name: string): RawCase { + const record = requiredRecord(value, name); + return { + id: requiredString(record.id, `${name}.id`), + flow: requiredString(record.flow, `${name}.flow`), + commands: requiredArray(record.commands, `${name}.commands`).map((entry, index) => + readRawCommand(entry, `${name}.commands[${index}]`), + ), + }; +} + +function readRawCommand(value: unknown, name: string): RawCommand { + const record = requiredRecord(value, name); + const command: RawCommand = { + ...record, + type: requiredString(record.type, `${name}.type`), + }; + if (record.source !== undefined) command.source = readSource(record.source, `${name}.source`); + if (record.commands !== undefined) { + command.commands = requiredArray(record.commands, `${name}.commands`).map((entry, index) => + readRawCommand(entry, `${name}.commands[${index}]`), + ); + } + return command; +} + +function readUpstreamPin(value: unknown, name: string): UpstreamPin { + const record = requiredRecord(value, name); + const artifacts = requiredArray(record.artifacts, `${name}.artifacts`).map((entry, index) => { + const artifact = requiredRecord(entry, `${name}.artifacts[${index}]`); + return { + path: requiredString(artifact.path, `${name}.artifacts[${index}].path`), + role: requiredString(artifact.role, `${name}.artifacts[${index}].role`), + sha256: requiredString(artifact.sha256, `${name}.artifacts[${index}].sha256`), + } satisfies UpstreamArtifact; + }); + return { + project: requiredString(record.project, `${name}.project`), + version: requiredString(record.version, `${name}.version`), + tag: requiredString(record.tag, `${name}.tag`), + commit: requiredString(record.commit, `${name}.commit`), + sourceUrl: requiredString(record.sourceUrl, `${name}.sourceUrl`), + artifacts, + }; +} + +function readSource(value: unknown, name: string): UpstreamSource { + const record = requiredRecord(value, name); + const line = record.line; + if (typeof line !== 'number' || !Number.isInteger(line) || line < 1) { + throw new Error(`${name}.line must be a positive integer.`); + } + return { + path: requiredString(record.path, `${name}.path`), + line, + }; +} + +function readJson(filePath: string): unknown { + return JSON.parse(fs.readFileSync(filePath, 'utf8')) as unknown; +} + +function requiredRecord(value: unknown, name: string): Record { + return readRequiredRecord(value, name); +} + +function requiredArray(value: unknown, name: string): unknown[] { + if (!Array.isArray(value)) throw new Error(`${name} must be an array.`); + return value; +} + +function requiredString(value: unknown, name: string): string { + if (typeof value !== 'string' || value.length === 0) throw new Error(`${name} must be a string.`); + return value; +} + +function assertPinnedUpstream(upstream: UpstreamPin): void { + for (const key of ['project', 'version', 'tag', 'commit'] as const) { + if (upstream[key] !== PINNED_UPSTREAM[key]) { + throw new Error( + `Upstream fixture must pin ${key}=${PINNED_UPSTREAM[key]}; found ${upstream[key]}.`, + ); + } + } +} + +function compareJson(actual: unknown, expected: unknown, name: string): void { + const actualJson = JSON.stringify(actual, null, 2); + const expectedJson = JSON.stringify(expected, null, 2); + if (actualJson === expectedJson) return; + throw new Error( + `${name} mismatch. Run node --experimental-strip-types scripts/maestro-conformance.ts --regenerate only after reviewing the upstream capture.\nExpected:\n${expectedJson}\nActual:\n${actualJson}`, + ); +} + +function parseMode(args: readonly string[]): 'check' | 'regenerate' | 'help' { + let mode: 'check' | 'regenerate' | 'help' = 'check'; + for (const arg of args) { + if (arg === '--help' || arg === '-h') { + mode = 'help'; + continue; + } + if (arg === '--check' || arg === '--regenerate') { + const next = arg === '--check' ? 'check' : 'regenerate'; + if (mode !== 'check' && mode !== next) throw new Error('Choose only one conformance mode.'); + mode = next; + continue; + } + throw new Error(`Unknown argument: ${arg}`); + } + return mode; +} + +function printHelp(): void { + console.log( + 'Usage: node --experimental-strip-types scripts/maestro-conformance.ts [--check|--regenerate]', + ); + console.log(''); + console.log( + ' --check Compare the parser against checked-in normalized fixtures (default).', + ); + console.log(' --regenerate Rebuild the normalized fixture from the pinned upstream capture.'); +} + +function main(args: readonly string[]): void { + try { + const mode = parseMode(args); + if (mode === 'help') { + printHelp(); + return; + } + if (mode === 'regenerate') { + regenerateConformance(); + console.log(`Regenerated ${MAESTRO_CONFORMANCE_NORMALIZED_FIXTURE}.`); + return; + } + const result = checkConformance(); + console.log( + `Maestro ${result.upstream.version} conformance: ${result.cases.length} cases passed.`, + ); + } catch (error) { + console.error(error instanceof Error ? error.message : String(error)); + process.exitCode = 1; + } +} + +if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + main(process.argv.slice(2)); +} diff --git a/src/__tests__/cli-network.test.ts b/src/__tests__/cli-network.test.ts index f77a41e09..c43576d52 100644 --- a/src/__tests__/cli-network.test.ts +++ b/src/__tests__/cli-network.test.ts @@ -250,14 +250,14 @@ test('test command --verbose omits step telemetry for passing tests without debu type: 'replay_action_start', step: 1, line: 3, - command: '__maestroTapOn', + command: 'tapOn', positionals: ['text="Log in"'], }, { type: 'replay_action_stop', step: 1, line: 3, - command: '__maestroTapOn', + command: 'tapOn', ok: true, durationMs: 250, }, @@ -265,14 +265,14 @@ test('test command --verbose omits step telemetry for passing tests without debu type: 'replay_action_start', step: 2, line: 4, - command: '__maestroAssertVisible', + command: 'assertVisible', positionals: ['text="Home"'], }, { type: 'replay_action_stop', step: 2, line: 4, - command: '__maestroAssertVisible', + command: 'assertVisible', ok: true, durationMs: 75, }, @@ -333,14 +333,14 @@ test('test command --verbose includes step telemetry in completed progress outpu type: 'replay_action_start', step: 1, line: 3, - command: '__maestroTapOn', + command: 'tapOn', positionals: ['text="Log in"'], }, { type: 'replay_action_stop', step: 1, line: 3, - command: '__maestroTapOn', + command: 'tapOn', ok: true, durationMs: 250, }, @@ -436,14 +436,14 @@ test('test command --verbose omits nested passing step telemetry', async () => { type: 'replay_action_start', step: 2.001, line: 4, - command: '__maestroAssertVisible', + command: 'assertVisible', positionals: ['label="Chat" || text="Chat" || id="Chat"', '60000'], }, { type: 'replay_action_stop', step: 2.001, line: 4, - command: '__maestroAssertVisible', + command: 'assertVisible', ok: true, durationMs: 2580, }, @@ -603,14 +603,14 @@ test('test command --debug prints failed attempt step window when timing trace e type: 'replay_action_start', step: 2, line: 4, - command: '__maestroTapOn', + command: 'tapOn', positionals: ['text="Pay"'], }, { type: 'replay_action_stop', step: 2, line: 4, - command: '__maestroTapOn', + command: 'tapOn', ok: true, durationMs: 200, }, @@ -618,14 +618,14 @@ test('test command --debug prints failed attempt step window when timing trace e type: 'replay_action_start', step: 3, line: 5, - command: '__maestroAssertVisible', + command: 'assertVisible', positionals: ['text="Receipt"', '3000'], }, { type: 'replay_action_stop', step: 3, line: 5, - command: '__maestroAssertVisible', + command: 'assertVisible', ok: false, durationMs: 1500, errorCode: 'ASSERTION_FAILED', diff --git a/src/__tests__/test-utils/android-snapshot-helper.ts b/src/__tests__/test-utils/android-snapshot-helper.ts new file mode 100644 index 000000000..8bb036dfb --- /dev/null +++ b/src/__tests__/test-utils/android-snapshot-helper.ts @@ -0,0 +1,75 @@ +import type { AndroidAdbExecutor } from '../../platforms/android/adb-executor.ts'; +import type { AndroidSnapshotHelperArtifact } from '../../platforms/android/snapshot-helper-types.ts'; +import { fileURLToPath } from 'node:url'; + +const SNAPSHOT_HELPER_PACKAGE = 'com.callstack.agentdevice.snapshothelper'; +const SNAPSHOT_HELPER_FIXTURE_APK_PATH = fileURLToPath( + new URL('./fixtures/android-helper-apk.fixture', import.meta.url), +); +const SNAPSHOT_HELPER_FIXTURE_APK_SHA256 = + 'a5f6a2fba1163bba2f13026bd3a192f52ba2816524b7cfa83c6b7ca568f6710a'; + +export const ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT: AndroidSnapshotHelperArtifact = { + apkPath: SNAPSHOT_HELPER_FIXTURE_APK_PATH, + manifest: { + name: 'android-snapshot-helper', + version: '0.13.3', + apkUrl: null, + sha256: SNAPSHOT_HELPER_FIXTURE_APK_SHA256, + packageName: SNAPSHOT_HELPER_PACKAGE, + versionCode: 13003, + instrumentationRunner: `${SNAPSHOT_HELPER_PACKAGE}/.SnapshotInstrumentation`, + minSdk: 23, + targetSdk: 36, + outputFormat: 'uiautomator-xml', + statusProtocol: 'android-snapshot-helper-v1', + installArgs: ['install', '-r', '-t'], + }, +}; + +export function createAndroidSnapshotHelperExecutor(options: { + readonly exec: AndroidAdbExecutor; + readonly captureXml: () => string | Promise; +}): AndroidAdbExecutor { + return async (args, execOptions) => { + if (isAndroidSnapshotHelperVersionProbe(args)) { + return { + exitCode: 0, + stdout: `package:${SNAPSHOT_HELPER_PACKAGE} versionCode:999999`, + stderr: '', + }; + } + if (isAndroidSnapshotHelperCapture(args)) { + return { + exitCode: 0, + stdout: androidSnapshotHelperOutput(await options.captureXml()), + stderr: '', + }; + } + return await options.exec(args, execOptions); + }; +} + +export function isAndroidSnapshotHelperCapture(args: readonly string[]): boolean { + return args[0] === 'shell' && args[1] === 'am' && args[2] === 'instrument'; +} + +export function androidSnapshotHelperOutput(xml: string): string { + return [ + 'INSTRUMENTATION_STATUS: agentDeviceProtocol=android-snapshot-helper-v1', + 'INSTRUMENTATION_STATUS: helperApiVersion=1', + 'INSTRUMENTATION_STATUS: outputFormat=uiautomator-xml', + 'INSTRUMENTATION_STATUS: chunkIndex=0', + 'INSTRUMENTATION_STATUS: chunkCount=1', + `INSTRUMENTATION_STATUS: payloadBase64=${Buffer.from(xml, 'utf8').toString('base64')}`, + 'INSTRUMENTATION_STATUS_CODE: 1', + 'INSTRUMENTATION_RESULT: agentDeviceProtocol=android-snapshot-helper-v1', + 'INSTRUMENTATION_RESULT: helperApiVersion=1', + 'INSTRUMENTATION_RESULT: ok=true', + 'INSTRUMENTATION_CODE: 0', + ].join('\n'); +} + +function isAndroidSnapshotHelperVersionProbe(args: readonly string[]): boolean { + return args.includes('--show-versioncode') && args.includes(SNAPSHOT_HELPER_PACKAGE); +} diff --git a/src/platforms/android/__tests__/fixtures/helper-apk.fixture b/src/__tests__/test-utils/fixtures/android-helper-apk.fixture similarity index 100% rename from src/platforms/android/__tests__/fixtures/helper-apk.fixture rename to src/__tests__/test-utils/fixtures/android-helper-apk.fixture diff --git a/src/__tests__/test-utils/index.ts b/src/__tests__/test-utils/index.ts index 56fd18592..8858352fc 100644 --- a/src/__tests__/test-utils/index.ts +++ b/src/__tests__/test-utils/index.ts @@ -20,6 +20,13 @@ export { export { makeSnapshotState } from './snapshot-builders.ts'; +export { + ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, + androidSnapshotHelperOutput, + createAndroidSnapshotHelperExecutor, + isAndroidSnapshotHelperCapture, +} from './android-snapshot-helper.ts'; + export { makeSessionStore } from './store-factory.ts'; export { withNoColor } from './color.ts'; diff --git a/src/client/client-types.ts b/src/client/client-types.ts index 803b88162..dc3fa3f2e 100644 --- a/src/client/client-types.ts +++ b/src/client/client-types.ts @@ -816,6 +816,7 @@ export type FindOptions = export type ReplayRunOptions = AgentDeviceRequestOverrides & AgentDeviceSelectionOptions & { path: string; + runtime?: SessionRuntimeHints; /** * @deprecated ADR 0012 migration step 6: `--update` no longer rewrites * the script. Accepted for backward compatibility; every divergence @@ -848,6 +849,7 @@ export type ReplayRunOptions = AgentDeviceRequestOverrides & export type ReplayTestOptions = AgentDeviceRequestOverrides & AgentDeviceSelectionOptions & { paths: string[]; + runtime?: SessionRuntimeHints; update?: boolean; /** @deprecated Use backend: 'maestro'. */ maestro?: boolean; diff --git a/src/commands/capture/runtime/snapshot.test.ts b/src/commands/capture/runtime/snapshot.test.ts index db3587dd7..f5222f61f 100644 --- a/src/commands/capture/runtime/snapshot.test.ts +++ b/src/commands/capture/runtime/snapshot.test.ts @@ -149,24 +149,6 @@ test('runtime snapshot emits filtered Android guidance from backend analysis', a ]); }); -test('runtime snapshot warns when Android helper falls back to stock UIAutomator', async () => { - const device = createSnapshotOnlyDevice({ - nodes: [{ ref: 'e1', index: 0, depth: 0, type: 'Window', label: 'Home' }], - truncated: false, - backend: 'android', - androidSnapshot: { - backend: 'uiautomator-dump', - fallbackReason: 'helper artifact missing', - }, - }); - - const result = await device.capture.snapshot({ session: 'default', interactiveOnly: true }); - - assert.deepEqual(result.warnings, [ - 'Android snapshot helper unavailable; using stock UIAutomator dump, which can time out on busy React Native UIs. Reason: helper artifact missing', - ]); -}); - test('runtime snapshot warns when iOS interactive output is root-only', async () => { const device = createSnapshotOnlyDevice({ nodes: [{ ref: 'e1', index: 0, depth: 0, type: 'Application' }], diff --git a/src/commands/capture/runtime/snapshot.ts b/src/commands/capture/runtime/snapshot.ts index ce1764c54..ce495084e 100644 --- a/src/commands/capture/runtime/snapshot.ts +++ b/src/commands/capture/runtime/snapshot.ts @@ -240,11 +240,6 @@ function buildSnapshotWarnings(params: { warnings.push(...buildMergedAccessibilityLeafWarnings(params.snapshot.nodes)); } - const helperFallbackWarning = formatAndroidHelperFallbackWarning( - params.annotations.androidSnapshot, - ); - if (helperFallbackWarning) warnings.push(helperFallbackWarning); - const reactNativeOverlayWarning = formatReactNativeOverlayWarning(params.snapshot.nodes); if (reactNativeOverlayWarning) warnings.push(reactNativeOverlayWarning); @@ -335,14 +330,6 @@ function buildEmptyAndroidInteractiveWarnings(params: { return warnings; } -function formatAndroidHelperFallbackWarning( - androidSnapshot: SnapshotCaptureAnnotations['androidSnapshot'], -): string | undefined { - if (androidSnapshot?.backend !== 'uiautomator-dump') return undefined; - const reason = androidSnapshot.fallbackReason ? ` Reason: ${androidSnapshot.fallbackReason}` : ''; - return `Android snapshot helper unavailable; using stock UIAutomator dump, which can time out on busy React Native UIs.${reason}`; -} - function formatRecentSnapshotDropWarning(params: { annotations: SnapshotCaptureAnnotations; snapshot: SnapshotState; diff --git a/src/commands/management/app.ts b/src/commands/management/app.ts index f0538172d..a0ec90bba 100644 --- a/src/commands/management/app.ts +++ b/src/commands/management/app.ts @@ -2,7 +2,6 @@ import { PUBLIC_COMMANDS } from '../../command-catalog.ts'; import type { AppCloseOptions, AppOpenOptions } from '../../client/client-types.ts'; import { DEFAULT_APPS_FILTER } from '../../contracts/app-inventory.ts'; import { SESSION_SURFACES } from '../../contracts/session-surface.ts'; -import type { SessionRuntimeHints } from '../../kernel/contracts.ts'; import type { CommandSchemaOverride } from '../../cli-schema/types.ts'; import { assertResolvedAppsFilter } from './app-inventory-contract.ts'; import { @@ -18,8 +17,10 @@ import { import { defineExecutableCommand } from '../command-contract.ts'; import { commonInputFromFlags, direct, optionalString } from '../cli-grammar/common.ts'; import type { CliReader, CommandInput, DaemonWriter } from '../cli-grammar/types.ts'; +import { METRO_RELOAD_FLAGS } from '../cli-grammar/flag-groups.ts'; import { defineCommandFacet } from '../family/types.ts'; import { defineFieldCommandMetadata } from '../field-command-contract.ts'; +import { withCommandRuntimeHints } from '../runtime-hints.ts'; import { managementCliOutputFormatters } from './output.ts'; const appsCommandMetadata = defineFieldCommandMetadata('apps', 'List installed apps.', { @@ -85,27 +86,7 @@ function toAppOpenOptions( launchUrl?: string; }, ): AppOpenOptions { - const { metroHost, metroPort, bundleUrl, launchUrl, ...rest } = input; - const runtime = buildOpenRuntimeHints({ metroHost, metroPort, bundleUrl, launchUrl }); - return runtime ? { ...rest, runtime } : rest; -} - -function buildOpenRuntimeHints(hints: { - metroHost?: string; - metroPort?: number; - bundleUrl?: string; - launchUrl?: string; -}): SessionRuntimeHints | undefined { - const { metroHost, metroPort, bundleUrl, launchUrl } = hints; - if ( - metroHost === undefined && - metroPort === undefined && - bundleUrl === undefined && - launchUrl === undefined - ) { - return undefined; - } - return { metroHost, metroPort, bundleUrl, launchUrl }; + return withCommandRuntimeHints(input); } const closeCommandDefinition = defineExecutableCommand(closeCommandMetadata, (client, input) => @@ -134,9 +115,7 @@ const openCliSchema = { 'noRecord', 'relaunch', 'surface', - 'metroHost', - 'metroPort', - 'bundleUrl', + ...METRO_RELOAD_FLAGS, 'launchUrl', ], } as const satisfies CommandSchemaOverride; diff --git a/src/commands/replay/index.test.ts b/src/commands/replay/index.test.ts index a298ef63f..b0feaf418 100644 --- a/src/commands/replay/index.test.ts +++ b/src/commands/replay/index.test.ts @@ -50,6 +50,9 @@ describe('replay command interface', () => { replayUpdate: true, replayMaestro: true, replayEnv: ['FOO=bar'], + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', }), ), ).toEqual({ @@ -57,6 +60,9 @@ describe('replay command interface', () => { update: true, backend: 'maestro', env: ['FOO=bar'], + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', }); }); @@ -72,6 +78,9 @@ describe('replay command interface', () => { replayUpdate: true, replayMaestro: true, replayEnv: ['FOO=bar'], + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', failFast: true, timeoutMs: 10_000, retries: 2, @@ -86,6 +95,9 @@ describe('replay command interface', () => { update: true, backend: 'maestro', env: ['FOO=bar'], + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', failFast: true, timeoutMs: 10_000, retries: 2, @@ -119,6 +131,8 @@ describe('replay command interface', () => { const testRequest = testDaemonWriter({ paths: ['./suite.ad'], maestro: true, + metroHost: '127.0.0.1', + metroPort: 8083, reportJunit: './junit.xml', }); expect(testRequest).toMatchObject({ @@ -126,10 +140,48 @@ describe('replay command interface', () => { positionals: ['./suite.ad'], options: { replayBackend: 'maestro', + metroHost: '127.0.0.1', + metroPort: 8083, }, }); expect(testRequest.options).not.toHaveProperty('reportJunit'); }); + + test('folds replay and test Metro hints into the runtime request envelope', async () => { + const runCalls: unknown[] = []; + const testCalls: unknown[] = []; + const client = { + replay: { + run: async (input: unknown) => { + runCalls.push(input); + return {}; + }, + test: async (input: unknown) => { + testCalls.push(input); + return {}; + }, + }, + } as never; + const runtimeFlags = flags({ + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', + }); + + await replayCommandDefinition.invoke(client, replayCliReader(['./flow.ad'], runtimeFlags)); + await testCommandDefinition.invoke(client, testCliReader(['./suite'], runtimeFlags)); + + const runtime = { + metroHost: '127.0.0.1', + metroPort: 8083, + bundleUrl: 'http://127.0.0.1:8083/index.bundle', + launchUrl: undefined, + }; + expect(runCalls[0]).toMatchObject({ path: './flow.ad', runtime }); + expect(testCalls[0]).toMatchObject({ paths: ['./suite'], runtime }); + expect(runCalls[0]).not.toHaveProperty('metroHost'); + expect(testCalls[0]).not.toHaveProperty('metroPort'); + }); }); describe('replay resume (ADR 0012 decision 4 / migration step 5)', () => { diff --git a/src/commands/replay/index.ts b/src/commands/replay/index.ts index 365dbc879..a2e095662 100644 --- a/src/commands/replay/index.ts +++ b/src/commands/replay/index.ts @@ -19,7 +19,8 @@ import { requiredString, } from '../cli-grammar/common.ts'; import type { CliReader, CommandInput, DaemonWriter } from '../cli-grammar/types.ts'; -import { REPLAY_FLAGS } from '../cli-grammar/flag-groups.ts'; +import { METRO_RELOAD_FLAGS, REPLAY_FLAGS } from '../cli-grammar/flag-groups.ts'; +import { withCommandRuntimeHints } from '../runtime-hints.ts'; const REPLAY_COMMAND_NAME = 'replay'; const TEST_COMMAND_NAME = 'test'; @@ -38,6 +39,9 @@ export const replayCommandMetadata = defineFieldCommandMetadata( backend: stringField(), maestro: booleanField(), env: stringArrayField(), + metroHost: stringField('Metro/debug host hint inherited by replay-opened sessions.'), + metroPort: integerField('Metro/debug port hint inherited by replay-opened sessions.'), + bundleUrl: stringField('Bundle URL hint inherited by replay-opened sessions.'), // ADR 0012 decision 4 / migration step 5: replay-only resume. Named // `resumeFrom`/`resumePlanDigest` (not `from`/`planDigest`) because // `from` already means a gesture's `PointInput` on `CommandInput` @@ -61,6 +65,9 @@ export const testCommandMetadata = defineFieldCommandMetadata( backend: stringField(), maestro: booleanField(), env: stringArrayField(), + metroHost: stringField('Metro/debug host hint inherited by each test session.'), + metroPort: integerField('Metro/debug port hint inherited by each test session.'), + bundleUrl: stringField('Bundle URL hint inherited by each test session.'), failFast: booleanField(), timeoutMs: integerField(), retries: integerField(), @@ -73,11 +80,11 @@ export const testCommandMetadata = defineFieldCommandMetadata( export const replayCommandDefinition = defineExecutableCommand( replayCommandMetadata, - (client, input) => client.replay.run(input), + (client, input) => client.replay.run(withCommandRuntimeHints(input)), ); export const testCommandDefinition = defineExecutableCommand(testCommandMetadata, (client, input) => - client.replay.test(input), + client.replay.test(withCommandRuntimeHints(input)), ); const replayCliSchema = { @@ -91,6 +98,7 @@ const replayCliSchema = { 'replayMaestro', 'replayExportFormat', ...REPLAY_FLAGS, + ...METRO_RELOAD_FLAGS, 'replayFrom', 'replayPlanDigest', 'timeoutMs', @@ -109,6 +117,7 @@ const testCliSchema = { allowedFlags: [ 'replayMaestro', ...REPLAY_FLAGS, + ...METRO_RELOAD_FLAGS, 'failFast', 'timeoutMs', 'retries', @@ -127,6 +136,9 @@ export const replayCliReader: CliReader = (positionals, flags) => ({ update: flags.replayUpdate, backend: flags.replayMaestro ? 'maestro' : undefined, env: flags.replayEnv, + metroHost: flags.metroHost, + metroPort: flags.metroPort, + bundleUrl: flags.bundleUrl, resumeFrom: flags.replayFrom, resumePlanDigest: flags.replayPlanDigest, saveScript: flags.saveScript, @@ -138,6 +150,9 @@ export const testCliReader: CliReader = (positionals, flags) => ({ update: flags.replayUpdate, backend: flags.replayMaestro ? 'maestro' : undefined, env: flags.replayEnv, + metroHost: flags.metroHost, + metroPort: flags.metroPort, + bundleUrl: flags.bundleUrl, failFast: flags.failFast, timeoutMs: flags.timeoutMs, retries: flags.retries, diff --git a/src/commands/runtime-hints.ts b/src/commands/runtime-hints.ts new file mode 100644 index 000000000..9eb7bff8f --- /dev/null +++ b/src/commands/runtime-hints.ts @@ -0,0 +1,28 @@ +import type { SessionRuntimeHints } from '../kernel/contracts.ts'; + +export type CommandRuntimeHintInput = Pick< + SessionRuntimeHints, + 'metroHost' | 'metroPort' | 'bundleUrl' | 'launchUrl' +>; +type CommandRuntimeHintKey = keyof CommandRuntimeHintInput; + +function buildCommandRuntimeHints(hints: CommandRuntimeHintInput): SessionRuntimeHints | undefined { + const { metroHost, metroPort, bundleUrl, launchUrl } = hints; + if ( + metroHost === undefined && + metroPort === undefined && + bundleUrl === undefined && + launchUrl === undefined + ) { + return undefined; + } + return { metroHost, metroPort, bundleUrl, launchUrl }; +} + +export function withCommandRuntimeHints( + input: TInput, +): Omit & { runtime?: SessionRuntimeHints } { + const { metroHost, metroPort, bundleUrl, launchUrl, ...rest } = input; + const runtime = buildCommandRuntimeHints({ metroHost, metroPort, bundleUrl, launchUrl }); + return runtime ? { ...rest, runtime } : rest; +} diff --git a/src/compat/__tests__/replay-input.test.ts b/src/compat/__tests__/replay-input.test.ts index e79f6f636..3931d049e 100644 --- a/src/compat/__tests__/replay-input.test.ts +++ b/src/compat/__tests__/replay-input.test.ts @@ -3,51 +3,12 @@ import assert from 'node:assert/strict'; import { AppError } from '../../kernel/errors.ts'; import { parseReplayInput } from '../replay-input.ts'; -test('parseReplayInput routes compat replay scripts through the selected parser', () => { - const parsed = parseReplayInput( - `appId: com.callstack.agentdevicelab ---- -- launchApp -- tapOn: - id: submit-order -`, - { replayBackend: 'maestro' }, - ); - - assert.deepEqual( - parsed.actions.map((action) => [action.command, action.positionals]), - [ - ['open', ['com.callstack.agentdevicelab']], - ['__maestroTapOn', ['id="submit-order"']], - ], - ); -}); - -test('parseReplayInput applies replay env precedence before compat parsing', () => { - const parsed = parseReplayInput( - `appId: \${APP_ID} -env: - APP_ID: yaml-app - BUTTON_ID: yaml-button ---- -- launchApp -- tapOn: - id: \${BUTTON_ID} -`, - { - replayBackend: 'maestro', - replayShellEnv: { AD_VAR_APP_ID: 'shell-app', AD_VAR_BUTTON_ID: 'shell-button' }, - replayEnv: ['APP_ID=cli-app'], - }, - ); +test('parseReplayInput keeps .ad parsing generic for Maestro suites', () => { + const parsed = parseReplayInput('open Demo\n', { replayBackend: 'maestro' }); - assert.equal(parsed.metadata.env?.APP_ID, 'yaml-app'); assert.deepEqual( - parsed.actions.map((action) => [action.command, action.positionals]), - [ - ['open', ['cli-app']], - ['__maestroTapOn', ['id="shell-button"']], - ], + parsed.actions.map((action) => action.command), + ['open'], ); }); diff --git a/src/compat/maestro/__tests__/daemon-runtime-port-fixtures.ts b/src/compat/maestro/__tests__/daemon-runtime-port-fixtures.ts new file mode 100644 index 000000000..38ae94239 --- /dev/null +++ b/src/compat/maestro/__tests__/daemon-runtime-port-fixtures.ts @@ -0,0 +1,63 @@ +import type { DaemonInvokeFn, DaemonRequest } from '../../../daemon/types.ts'; +import type { SnapshotNode, SnapshotState } from '../../../kernel/snapshot.ts'; +import type { DaemonMaestroRuntimeDependencies } from '../daemon-runtime-port.ts'; + +export type CapturedDaemonRequest = Pick< + DaemonRequest, + 'command' | 'positionals' | 'input' | 'flags' +>; + +export const IOS_FRAME = { x: 0, y: 0, width: 402, height: 874 } as const; + +export function makeSnapshot( + nodes: Array & { ref?: string }>, +): SnapshotState { + return { + createdAt: 0, + nodes: nodes.map((node) => ({ ref: `e${node.index + 1}`, ...node })), + }; +} + +export function makeBaseRequest( + overrides: Partial> = {}, +): Omit { + return { + token: 'test-token', + session: 'maestro-test', + ...overrides, + }; +} + +export function makeDependencies( + now: { value: number } = { value: 0 }, +): DaemonMaestroRuntimeDependencies { + return { + now: () => now.value, + sleep: async (milliseconds) => { + now.value += milliseconds; + }, + }; +} + +export function makeInvoke( + requests: CapturedDaemonRequest[], + response: DaemonInvokeFn = async (request) => { + requests.push({ + command: request.command, + positionals: request.positionals, + input: request.input, + flags: request.flags, + }); + return { ok: true, data: {} }; + }, +): DaemonInvokeFn { + return async (request) => { + requests.push({ + command: request.command, + positionals: request.positionals, + input: request.input, + flags: request.flags, + }); + return await response(request); + }; +} diff --git a/src/compat/maestro/__tests__/daemon-runtime-port-observation.test.ts b/src/compat/maestro/__tests__/daemon-runtime-port-observation.test.ts new file mode 100644 index 000000000..37c490a72 --- /dev/null +++ b/src/compat/maestro/__tests__/daemon-runtime-port-observation.test.ts @@ -0,0 +1,403 @@ +import { expect, test } from 'vitest'; +import type { DaemonInvokeFn, DaemonRequest } from '../../../daemon/types.ts'; +import { createDaemonMaestroRuntimePort } from '../daemon-runtime-port.ts'; +import { + MAESTRO_INITIAL_SNAPSHOT_READY_TIMEOUT_MS, + MAESTRO_OBSERVATION_POLL_MS, +} from '../daemon-runtime-port-observation.ts'; +import { makeBaseRequest, makeDependencies } from './daemon-runtime-port-fixtures.ts'; + +test('does not pair an observation with a later same-generation snapshot', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + const targetX = snapshots >= 4 ? 40 : 200; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + { + index: 2, + parentIndex: 0, + type: 'Button', + identifier: 'continue', + rect: { x: targetX, y: 100, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(), + platform: 'ios', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 0, + generation: 0, + env: {}, + }); + await port.execute({ + command: { kind: 'waitForAnimationToEnd', source: { line: 2 }, timeout: 500 }, + generation: 0, + cachedObservation: observation, + env: {}, + }); + await port.execute({ + command: { + kind: 'tapOn', + source: { line: 3 }, + target: { space: 'target', selector: { id: 'continue' } }, + }, + generation: 0, + cachedObservation: observation, + env: {}, + }); + + expect(requests.map((request) => request.command)).toEqual([ + 'snapshot', + 'snapshot', + 'snapshot', + 'snapshot', + 'click', + ]); + expect(requests.at(-1)?.positionals).toEqual(['id="continue"']); +}); + +test('retries typed transient snapshot failures within the observation budget', async () => { + const requests: DaemonRequest[] = []; + const clock = { value: 0 }; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + if (snapshots === 1) { + return { + ok: false, + error: { + code: 'COMMAND_FAILED', + message: 'Foreground app window is transitioning.', + retriable: true, + }, + }; + } + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(clock), + platform: 'android', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 500, + generation: 0, + env: {}, + }); + + expect(observation.matched).toBe(true); + expect(requests.map((request) => request.command)).toEqual(['snapshot', 'snapshot']); + expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS); +}); + +test('starts the selector timeout after the first valid snapshot', async () => { + const clock = { value: 0 }; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + if (snapshots <= 3) { + return { + ok: false, + error: { + code: 'COMMAND_FAILED', + message: 'Foreground app window is transitioning.', + retriable: true, + }, + }; + } + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(clock), + platform: 'android', + }); + + await expect( + port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 500, + generation: 0, + env: {}, + }), + ).resolves.toMatchObject({ matched: true }); + expect(clock.value).toBe(3 * MAESTRO_OBSERVATION_POLL_MS); +}); + +test('bounds initial typed snapshot recovery independently of selector matching', async () => { + const clock = { value: 0 }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async () => ({ + ok: false, + error: { + code: 'COMMAND_FAILED', + message: 'Foreground app window is transitioning.', + retriable: true, + }, + }), + dependencies: makeDependencies(clock), + platform: 'android', + }); + + await expect( + port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 500, + generation: 0, + env: {}, + }), + ).rejects.toMatchObject({ message: 'Foreground app window is transitioning.' }); + expect(clock.value).toBe(MAESTRO_INITIAL_SNAPSHOT_READY_TIMEOUT_MS); +}); + +test('does not retry deterministic snapshot failures', async () => { + const requests: DaemonRequest[] = []; + const clock = { value: 0 }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + return { + ok: false, + error: { + code: 'COMMAND_FAILED', + message: 'Android snapshot helper is unavailable.', + }, + }; + }, + dependencies: makeDependencies(clock), + platform: 'android', + }); + + await expect( + port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 500, + generation: 0, + env: {}, + }), + ).rejects.toMatchObject({ message: 'Android snapshot helper is unavailable.' }); + expect(requests.map((request) => request.command)).toEqual(['snapshot']); + expect(clock.value).toBe(0); +}); + +test('fails scrollUntilVisible when the target stays absent', async () => { + const invoke: DaemonInvokeFn = async (request) => + request.command === 'snapshot' + ? { + ok: true, + data: { + createdAt: 0, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + ], + }, + } + : { ok: true, data: {} }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(), + platform: 'ios', + }); + + await expect( + port.execute({ + command: { + kind: 'scrollUntilVisible', + source: { line: 2 }, + element: { text: 'Discover' }, + direction: 'up', + timeout: 500, + }, + generation: 0, + env: {}, + }), + ).rejects.toMatchObject({ + code: 'COMMAND_FAILED', + message: 'Maestro scrollUntilVisible target did not become visible.', + }); +}); + +test.each([ + [{ kind: 'inputText', source: { line: 2 }, text: 'hello' }, 'hello'], + [{ kind: 'eraseText', source: { line: 2 }, charactersToErase: 3 }, '\b\b\b'], + [{ kind: 'pasteText', source: { line: 2 }, text: 'pasted' }, 'pasted'], +] as const)('waits for a stable snapshot after a Maestro $kind mutation', async (command, text) => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'TextField', + value: snapshots === 1 ? 'pending' : 'committed', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(), + platform: 'android', + }); + + await port.execute({ command, generation: 0, env: {} }); + + expect(requests.map((request) => request.command)).toEqual([ + 'type', + 'snapshot', + 'snapshot', + 'snapshot', + ]); + expect(requests[0]?.positionals).toEqual([text]); +}); + +test('commits Maestro input text before dispatching an immediate tap', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + let textCommitted = false; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command === 'click') { + if (!textCommitted) throw new Error('tap raced the text commit'); + return { ok: true, data: {} }; + } + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + textCommitted = snapshots >= 2; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Button', + identifier: 'navigate', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + { + index: 2, + parentIndex: 0, + type: 'TextField', + value: textCommitted ? 'hello' : '', + rect: { x: 20, y: 100, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(), + platform: 'android', + }); + + await port.execute({ + command: { kind: 'inputText', source: { line: 2 }, text: 'hello' }, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'tapOn', + source: { line: 3 }, + target: { space: 'target', selector: { id: 'navigate' } }, + }, + generation: 1, + env: {}, + }); + + expect(requests.map((request) => request.command)).toEqual([ + 'type', + 'snapshot', + 'snapshot', + 'snapshot', + 'snapshot', + 'click', + ]); +}); diff --git a/src/compat/maestro/__tests__/daemon-runtime-port-targets.test.ts b/src/compat/maestro/__tests__/daemon-runtime-port-targets.test.ts new file mode 100644 index 000000000..7a27cab92 --- /dev/null +++ b/src/compat/maestro/__tests__/daemon-runtime-port-targets.test.ts @@ -0,0 +1,341 @@ +import { expect, test } from 'vitest'; +import type { DaemonInvokeFn, DaemonRequest } from '../../../daemon/types.ts'; +import { createDaemonMaestroRuntimePort } from '../daemon-runtime-port.ts'; +import { makeBaseRequest, makeDependencies } from './daemon-runtime-port-fixtures.ts'; + +test('waits for a delayed input target using fresh snapshots', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const invoke: DaemonInvokeFn = async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + ...(snapshots < 3 + ? [] + : [ + { + index: 1, + parentIndex: 0, + type: 'Button', + identifier: 'delayedButton', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ]), + ], + }, + }; + }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(), + platform: 'android', + }); + + await expect( + port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { id: 'delayedButton' } }, + }, + generation: 0, + env: {}, + }), + ).resolves.toMatchObject({ mutated: true }); + expect(requests.map((request) => request.command)).toEqual([ + 'snapshot', + 'snapshot', + 'snapshot', + 'click', + ]); + expect(requests.at(-1)?.positionals).toEqual(['80', '62']); +}); + +test('atomically dispatches a unique exact iOS target from same-generation evidence', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const invoke: DaemonInvokeFn = async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + { + index: 2, + parentIndex: 0, + type: 'Button', + identifier: 'continue', + rect: { x: snapshots === 1 ? 20 : 160, y: 100, width: 120, height: 44 }, + }, + ], + }, + }; + }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(), + platform: 'ios', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 0, + generation: 0, + env: {}, + }); + await expect( + port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { id: 'continue' } }, + }, + generation: 0, + cachedObservation: observation, + env: {}, + }), + ).resolves.toMatchObject({ mutated: true }); + + expect(requests.map((request) => request.command)).toEqual(['snapshot', 'click']); + expect(requests.at(-1)?.positionals).toEqual(['id="continue"']); + expect(requests.at(-1)?.flags?.maestro).toEqual({ + allowNonHittableCoordinateFallback: true, + }); +}); + +test('falls back to fresh Maestro geometry when atomic iOS dispatch resolves off-screen', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + let clicks = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command === 'click') { + clicks += 1; + return clicks === 1 + ? { + ok: false, + error: { + code: 'ELEMENT_OFFSCREEN', + message: 'element resolved off-screen during dispatch', + }, + } + : { ok: true, data: {} }; + } + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + { + index: 2, + parentIndex: 0, + type: 'Button', + identifier: 'continue', + rect: { x: snapshots === 1 ? 20 : 160, y: 100, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(), + platform: 'ios', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 0, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { id: 'continue' } }, + }, + generation: 0, + cachedObservation: observation, + env: {}, + }); + + expect(requests.map((request) => request.command)).toEqual([ + 'snapshot', + 'click', + 'snapshot', + 'click', + ]); + expect(requests.at(-1)?.positionals).toEqual(['220', '122']); +}); + +test('refreshes filtered target geometry instead of reusing an observation rectangle', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + { + index: 2, + parentIndex: 0, + type: 'Button', + identifier: 'continue', + enabled: true, + rect: { x: snapshots === 1 ? 20 : 160, y: 100, width: 120, height: 44 }, + }, + ], + }, + }; + }, + dependencies: makeDependencies(), + platform: 'ios', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 0, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { id: 'continue', enabled: true } }, + }, + generation: 0, + cachedObservation: observation, + env: {}, + }); + + expect(requests.map((request) => request.command)).toEqual(['snapshot', 'snapshot', 'click']); + expect(requests.at(-1)?.positionals).toEqual(['220', '122']); +}); + +test('captures fresh target state immediately after a same-generation observation', async () => { + const requests: DaemonRequest[] = []; + let snapshots = 0; + const invoke: DaemonInvokeFn = async (request) => { + requests.push(request); + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: snapshots, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ...(snapshots === 1 + ? [] + : [ + { + index: 2, + parentIndex: 0, + type: 'Button', + identifier: 'continue', + rect: { x: 20, y: 100, width: 120, height: 44 }, + }, + ]), + ], + }, + }; + }; + const now = { value: 0 }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(now), + platform: 'ios', + }); + + const observation = await port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 0, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { id: 'continue' } }, + }, + generation: 0, + cachedObservation: observation, + env: {}, + }); + + expect(requests.map((request) => request.command)).toEqual(['snapshot', 'snapshot', 'click']); + expect(now.value).toBe(0); +}); diff --git a/src/compat/maestro/__tests__/daemon-runtime-port.test.ts b/src/compat/maestro/__tests__/daemon-runtime-port.test.ts new file mode 100644 index 000000000..a895bb226 --- /dev/null +++ b/src/compat/maestro/__tests__/daemon-runtime-port.test.ts @@ -0,0 +1,278 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test } from 'vitest'; +import type { DaemonInvokeFn, DaemonRequest } from '../../../daemon/types.ts'; +import { createDaemonMaestroRuntimePort } from '../daemon-runtime-port.ts'; +import { makeBaseRequest, makeDependencies } from './daemon-runtime-port-fixtures.ts'; + +test('delegates lifecycle and coordinate gestures through public daemon commands', async () => { + const requests: DaemonRequest[] = []; + const invoke: DaemonInvokeFn = async (request) => { + requests.push(request); + return { ok: true, data: {} }; + }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(), + platform: 'android', + }); + + await port.execute({ + command: { + kind: 'launchApp', + source: { line: 2 }, + appId: 'com.example.app', + clearState: true, + launchArguments: { kind: 'map', values: { seed: 7 } }, + }, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'swipe', + source: { line: 3 }, + gesture: { + kind: 'coordinates', + start: { space: 'absolute', x: 360, y: 400 }, + end: { space: 'absolute', x: 40, y: 400 }, + duration: 240, + }, + }, + generation: 1, + env: {}, + }); + + expect(requests).toEqual([ + expect.objectContaining({ + command: 'open', + positionals: ['com.example.app'], + flags: expect.objectContaining({ + clearAppState: true, + launchArgs: ['seed', '7'], + }), + }), + expect.objectContaining({ + command: 'swipe', + positionals: [], + input: { + from: { x: 360, y: 400 }, + to: { x: 40, y: 400 }, + durationMs: 240, + }, + }), + ]); +}); + +test('pairs percentage swipe geometry with the nested public gesture request', async () => { + const requests: DaemonRequest[] = []; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + requests.push(request); + if (request.command === 'snapshot') { + return { + ok: true, + data: { + createdAt: 0, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 10, y: 20, width: 400, height: 800 }, + }, + ], + }, + }; + } + return { ok: true, data: {} }; + }, + dependencies: makeDependencies(), + platform: 'android', + }); + + await port.execute({ + command: { + kind: 'swipe', + source: { line: 3 }, + gesture: { + kind: 'coordinates', + start: { space: 'percent', x: 90, y: 50 }, + end: { space: 'percent', x: 10, y: 50 }, + duration: 300, + }, + }, + generation: 0, + env: {}, + }); + + expect(requests.at(-1)).toMatchObject({ + command: 'swipe', + input: { + from: { x: 360, y: 400 }, + to: { x: 40, y: 400 }, + durationMs: 300, + }, + internal: { gestureViewport: { x: 0, y: 0, width: 400, height: 800 } }, + }); +}); + +test('does not leak replay and test controls into nested public commands', async () => { + const requests: DaemonRequest[] = []; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ + flags: { + platform: 'android', + replayBackend: 'maestro', + replayUpdate: true, + replayEnv: ['TOKEN=value'], + replayShellEnv: { AD_VAR_TOKEN: 'value' }, + replayFrom: 2, + replayPlanDigest: 'digest', + saveScript: true, + failFast: true, + timeoutMs: 30_000, + retries: 2, + recordVideo: true, + shardAll: 4, + shardSplit: 2, + shardCount: 4, + shardIndex: 1, + }, + }), + invoke: async (request) => { + requests.push(request); + return { ok: true, data: {} }; + }, + dependencies: makeDependencies(), + platform: 'android', + }); + + await port.execute({ + command: { kind: 'launchApp', source: { line: 2 }, appId: 'com.example.app' }, + generation: 0, + env: {}, + }); + await port.execute({ + command: { kind: 'back', source: { line: 3 } }, + generation: 1, + env: {}, + }); + + expect(requests).toHaveLength(2); + expect(requests[0]?.flags).toEqual({ platform: 'android', relaunch: true }); + expect(requests[1]?.flags).toEqual({ platform: 'android' }); +}); + +test('keeps absent negative observations, script output, and artifacts typed', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-daemon-port-')); + const sourcePath = path.join(root, 'flow.yaml'); + fs.writeFileSync(sourcePath, '---\n- runScript: setup.js\n'); + fs.writeFileSync(path.join(root, 'setup.js'), 'output.token = PREFIX + "-ready";\n'); + const invoke: DaemonInvokeFn = async (request) => { + if (request.command === 'snapshot') { + return { + ok: true, + data: { + createdAt: 0, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + ], + }, + }; + } + if (request.command === 'screenshot') { + return { ok: true, data: { path: path.join(root, 'shot.png') } }; + } + return { ok: true, data: {} }; + }; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }), + invoke, + dependencies: makeDependencies(), + platform: 'ios', + sourcePath, + }); + + await expect( + port.observe({ + condition: { kind: 'notVisible', selector: { id: 'loading' } }, + timeoutMs: 0, + generation: 0, + env: { PREFIX: 'typed' }, + }), + ).resolves.toMatchObject({ matched: true, candidateCount: 0 }); + await expect( + port.execute({ + command: { kind: 'runScript', source: { path: sourcePath, line: 2 }, file: 'setup.js' }, + generation: 0, + env: { PREFIX: 'typed' }, + }), + ).resolves.toMatchObject({ outputEnv: { 'output.token': 'typed-ready' } }); + await expect( + port.execute({ + command: { kind: 'takeScreenshot', source: { line: 3 }, path: 'shot.png' }, + generation: 0, + env: { PREFIX: 'typed' }, + }), + ).resolves.toMatchObject({ artifactPaths: [path.join(root, 'shot.png')] }); +}); + +test('takes one final observation when polling wakes after the deadline', async () => { + const now = { value: 0 }; + let snapshots = 0; + const port = createDaemonMaestroRuntimePort({ + baseReq: makeBaseRequest({ flags: { platform: 'android', replayBackend: 'maestro' } }), + invoke: async (request) => { + if (request.command !== 'snapshot') return { ok: true, data: {} }; + snapshots += 1; + return { + ok: true, + data: { + createdAt: now.value, + nodes: [ + { + index: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + ...(now.value < 500 + ? [] + : [ + { + index: 1, + parentIndex: 0, + type: 'Text', + identifier: 'ready', + rect: { x: 20, y: 40, width: 120, height: 44 }, + }, + ]), + ], + }, + }; + }, + dependencies: { + now: () => now.value, + sleep: async (milliseconds) => { + now.value += milliseconds + 1; + }, + }, + platform: 'android', + }); + + await expect( + port.observe({ + condition: { kind: 'visible', selector: { id: 'ready' } }, + timeoutMs: 500, + generation: 0, + env: {}, + }), + ).resolves.toMatchObject({ matched: true }); + expect(snapshots).toBe(3); +}); diff --git a/src/compat/maestro/__tests__/engine-context.test.ts b/src/compat/maestro/__tests__/engine-context.test.ts new file mode 100644 index 000000000..78471df3a --- /dev/null +++ b/src/compat/maestro/__tests__/engine-context.test.ts @@ -0,0 +1,21 @@ +import { expect, test } from 'vitest'; +import { createMaestroExecutionContext } from '../engine-context.ts'; + +test('resolves transitive scoped variables to their final value', () => { + const context = createMaestroExecutionContext(); + const leave = context.enter({ + TARGET: '${NEXT}', + NEXT: '${FINAL}', + FINAL: 'Done', + }); + + expect(context.resolve('${TARGET}')).toBe('Done'); + expect(context.expandedVariables).toEqual({ TARGET: 'Done' }); + leave(); +}); + +test('preserves cyclic references instead of recursing indefinitely', () => { + const context = createMaestroExecutionContext({ FIRST: '${SECOND}', SECOND: '${FIRST}' }); + + expect(context.resolve('${FIRST}')).toBe('${FIRST}'); +}); diff --git a/src/compat/maestro/__tests__/engine.test.ts b/src/compat/maestro/__tests__/engine.test.ts new file mode 100644 index 000000000..40413fb68 --- /dev/null +++ b/src/compat/maestro/__tests__/engine.test.ts @@ -0,0 +1,378 @@ +import { describe, expect, test, vi } from 'vitest'; +import { AppError } from '../../../kernel/errors.ts'; +import { executeMaestroProgram } from '../engine.ts'; +import { parseMaestroProgram } from '../program-ir-parser.ts'; +import type { + MaestroObservation, + MaestroRuntimePort, + MaestroRuntimeRequest, + MaestroRuntimeResult, +} from '../engine-types.ts'; + +describe('executeMaestroProgram', () => { + test('preserves authored percentage swipe intent without observing', async () => { + const port = makePort(); + const program = parseMaestroProgram( + ['---', '- swipe:', ' start: 90%, 50%', ' end: 10%, 50%', ' duration: 100'].join( + '\n', + ), + ); + + const result = await executeMaestroProgram(program, port); + + expect(port.execute).toHaveBeenCalledWith({ + command: { + kind: 'swipe', + source: { line: 2 }, + gesture: { + kind: 'coordinates', + start: { space: 'percent', x: 90, y: 50 }, + end: { space: 'percent', x: 10, y: 50 }, + duration: 100, + }, + }, + env: {}, + generation: 0, + }); + expect(port.observe).not.toHaveBeenCalled(); + expect(result).toEqual({ executed: 1, skipped: 0, generation: 1, artifactPaths: [] }); + }); + + test('reuses observations within a generation and invalidates them after mutation', async () => { + const observations: MaestroObservation[] = []; + const port = makePort({ + observe: vi.fn(async ({ generation }) => { + const observation = { + generation, + matched: true, + evidence: { + kind: 'selector' as const, + selector: { text: 'Ready' }, + visible: true, + candidateCount: 1, + }, + }; + observations.push(observation); + return observation; + }), + }); + const program = parseMaestroProgram( + [ + '---', + '- assertVisible: Ready', + '- assertVisible:', + ' id: ready', + '- tapOn:', + ' id: continue', + '- assertVisible: Done', + ].join('\n'), + ); + + const result = await executeMaestroProgram(program, port); + + expect(port.observe).toHaveBeenNthCalledWith( + 2, + expect.objectContaining({ generation: 0, cachedObservation: observations[0] }), + ); + expect(port.execute).toHaveBeenCalledWith( + expect.objectContaining({ generation: 0, cachedObservation: observations[1] }), + ); + expect(port.observe).toHaveBeenNthCalledWith( + 3, + expect.not.objectContaining({ cachedObservation: expect.anything() }), + ); + expect(result.generation).toBe(1); + }); + + test.each([ + ['tap', ' - tapOn: Retry'], + ['runScript', ' - runScript: setup.js'], + ['waitForAnimationToEnd', ' - waitForAnimationToEnd: 10'], + ])('invalidates cached observations after a failed %s attempt', async (_name, commandYaml) => { + const attempts: MaestroRuntimeRequest[] = []; + const observation: MaestroObservation = { generation: 0, matched: true }; + const port = makePort({ + observe: vi.fn(async () => observation), + execute: vi.fn(async (request) => { + attempts.push(request); + if (attempts.length === 1) throw new AppError('COMMAND_FAILED', 'retry me'); + return { mutated: true }; + }), + }); + const program = parseMaestroProgram( + [ + '---', + '- assertVisible: Ready', + '- retry:', + ' maxRetries: 1', + ' commands:', + commandYaml, + ].join('\n'), + ); + + const result = await executeMaestroProgram(program, port); + + expect(attempts[0]).toMatchObject({ generation: 0, cachedObservation: observation }); + expect(attempts[1]).toMatchObject({ generation: 1 }); + expect(attempts[1]).not.toHaveProperty('cachedObservation'); + expect(result.generation).toBe(2); + }); + + test('owns hooks, includes, scoped env, output env, repeat, and retry', async () => { + const executed: MaestroRuntimeRequest[] = []; + let failingAttempts = 0; + const port = makePort({ + execute: vi.fn(async (request) => { + executed.push(request); + if (request.command.kind === 'runScript') { + return { mutated: false, outputEnv: { TOKEN: 'generated' } }; + } + if ( + request.command.kind === 'tapOn' && + request.command.target.space === 'target' && + request.command.target.selector.text === 'Retry' + ) { + failingAttempts += 1; + if (failingAttempts === 1) throw new AppError('COMMAND_FAILED', 'retry me'); + } + return { mutated: request.command.kind !== 'takeScreenshot' }; + }), + }); + const main = parseMaestroProgram( + [ + 'appId: root.app', + 'env:', + ' COUNT: 2', + 'onFlowStart:', + ' - launchApp', + 'onFlowComplete:', + ' - takeScreenshot: final.png', + '---', + '- runScript: setup.js', + '- runFlow:', + ' file: child.yaml', + ' env:', + ' LABEL: ${TOKEN}', + ].join('\n'), + { sourcePath: '/flows/main.yaml' }, + ); + const child = parseMaestroProgram( + [ + '---', + '- repeat:', + ' times: ${COUNT}', + ' commands:', + ' - tapOn: ${LABEL}', + '- retry:', + ' maxRetries: 1', + ' commands:', + ' - tapOn: Retry', + ].join('\n'), + ); + + const result = await executeMaestroProgram(main, port, { + loadProgram: vi.fn(async () => child), + }); + + expect(executed.filter((entry) => entry.command.kind === 'tapOn')).toHaveLength(4); + expect(executed[0]).toEqual( + expect.objectContaining({ + appId: 'root.app', + command: expect.objectContaining({ kind: 'launchApp' }), + }), + ); + expect( + executed.some( + (entry) => + entry.command.kind === 'tapOn' && + entry.command.target.space === 'target' && + entry.command.target.selector.text === 'generated', + ), + ).toBe(true); + expect(result.artifactPaths).toEqual([]); + expect(executed.at(-1)?.command.kind).toBe('takeScreenshot'); + }); + + test('skips false conditions without loading their programs', async () => { + const loadProgram = vi.fn(); + const port = makePort(); + const program = parseMaestroProgram( + ['---', '- runFlow:', ' file: ios.yaml', ' when:', ' platform: iOS'].join('\n'), + ); + + const result = await executeMaestroProgram(program, port, { + platform: 'android', + loadProgram, + }); + + expect(loadProgram).not.toHaveBeenCalled(); + expect(port.execute).not.toHaveBeenCalled(); + expect(result).toEqual({ executed: 0, skipped: 1, generation: 0, artifactPaths: [] }); + }); + + test('rejects stale runtime observations with source context', async () => { + const port = makePort({ + observe: vi.fn(async () => ({ generation: 9, matched: true })), + }); + const program = parseMaestroProgram('---\n- assertVisible: Ready\n', { + sourcePath: '/flows/stale.yaml', + }); + + await expect(executeMaestroProgram(program, port)).rejects.toThrow( + /observation generation 9.*\/flows\/stale\.yaml:line 2/i, + ); + }); + + test('uses injected timing and deterministic when.true grammar', async () => { + const controller = new AbortController(); + const observe = vi.fn( + async ({ generation, condition }: Parameters[0]) => ({ + generation, + matched: true, + evidence: { + kind: 'selector' as const, + selector: condition.selector, + visible: condition.kind === 'visible', + candidateCount: 1, + }, + }), + ); + const port = makePort({ observe }); + const program = parseMaestroProgram( + [ + '---', + '- assertVisible: Ready', + '- assertNotVisible: Gone', + '- extendedWaitUntil:', + ' visible: Done', + '- runFlow:', + ' when:', + ' true: "${maestro.platform == \'ios\' && (true || false)}"', + ' visible: Gate', + ' commands:', + ' - inputText: included', + '- runFlow:', + ' when:', + ' true: "${maestro.platform == \'android\'}"', + ' commands:', + ' - inputText: skipped', + ].join('\n'), + ); + + await executeMaestroProgram(program, port, { + platform: 'ios', + timing: { + assertVisibleTimeoutMs: 101, + assertNotVisibleTimeoutMs: 202, + extendedWaitUntilTimeoutMs: 303, + runFlowConditionTimeoutMs: 404, + }, + signal: controller.signal, + }); + + expect(observe.mock.calls.map(([request]) => request.timeoutMs)).toEqual([101, 202, 303, 404]); + expect(observe.mock.calls[0]?.[0].signal).toBe(controller.signal); + expect(port.execute).toHaveBeenCalledWith( + expect.objectContaining({ + command: expect.objectContaining({ kind: 'inputText', text: 'included' }), + }), + ); + expect(port.execute).not.toHaveBeenCalledWith( + expect.objectContaining({ + command: expect.objectContaining({ kind: 'inputText', text: 'skipped' }), + }), + ); + }); + + test('preserves output variables across scopes while runtime env wins', async () => { + const seenTexts: string[] = []; + const child = parseMaestroProgram( + [ + 'env:', + ' CHILD_CONFIG: ${CHILD}', + ' OVERRIDE: child', + '---', + '- inputText: ${CHILD_CONFIG}', + '- inputText: ${OVERRIDE}', + '- runScript: child.js', + ].join('\n'), + ); + const port = makePort({ + execute: vi.fn(async (request) => { + if (request.command.kind === 'inputText') seenTexts.push(request.command.text); + if (request.command.kind === 'runScript') { + return { mutated: false, outputEnv: { OUTPUT: 'generated' } }; + } + return { mutated: true }; + }), + }); + const program = parseMaestroProgram( + [ + 'env:', + ' BASE: parent', + ' OVERRIDE: flow', + '---', + '- runFlow:', + ' file: child.yaml', + ' env:', + ' CHILD: ${BASE}', + '- inputText: ${OUTPUT}', + '- inputText: ${OVERRIDE}', + ].join('\n'), + ); + + await executeMaestroProgram(program, port, { + env: { OVERRIDE: 'runtime' }, + loadProgram: vi.fn(async () => child), + }); + + expect(seenTexts).toEqual(['parent', 'runtime', 'generated', 'runtime']); + }); + + test('rejects recursive file includes before loading the child', async () => { + const loadProgram = vi.fn(); + const program = parseMaestroProgram('---\n- runFlow: ./main.yaml\n', { + sourcePath: '/flows/main.yaml', + }); + + await expect(executeMaestroProgram(program, makePort(), { loadProgram })).rejects.toThrow( + /runFlow cycle detected.*\/flows\/main\.yaml/i, + ); + expect(loadProgram).not.toHaveBeenCalled(); + }); + + test('forwards AbortSignal and stops at the next cancellation checkpoint', async () => { + const controller = new AbortController(); + const execute = vi.fn(async (_request: MaestroRuntimeRequest) => { + controller.abort(); + return { mutated: false }; + }); + const port = makePort({ execute }); + const program = parseMaestroProgram('---\n- inputText: first\n- inputText: second\n'); + + await expect( + executeMaestroProgram(program, port, { signal: controller.signal }), + ).rejects.toMatchObject({ + code: 'COMMAND_FAILED', + details: { reason: 'request_canceled' }, + }); + expect(port.execute).toHaveBeenCalledTimes(1); + expect(execute.mock.calls[0]?.[0].signal).toBe(controller.signal); + }); +}); + +function makePort(overrides: Partial = {}): MaestroRuntimePort { + return { + execute: vi.fn( + async ({ command }): Promise => ({ + mutated: + command.kind !== 'takeScreenshot' && + command.kind !== 'runScript' && + command.kind !== 'waitForAnimationToEnd', + ...(command.kind === 'takeScreenshot' ? { artifactPaths: [command.path] } : {}), + }), + ), + observe: vi.fn(async ({ generation }) => ({ generation, matched: true })), + ...overrides, + }; +} diff --git a/src/compat/maestro/__tests__/export-points.test.ts b/src/compat/maestro/__tests__/export-points.test.ts new file mode 100644 index 000000000..839cfad98 --- /dev/null +++ b/src/compat/maestro/__tests__/export-points.test.ts @@ -0,0 +1,7 @@ +import { expect, test } from 'vitest'; +import { formatMaestroPoint } from '../export-points.ts'; + +test('formatMaestroPoint serializes coordinate pairs', () => { + expect(formatMaestroPoint(100, 200)).toBe('100,200'); + expect(formatMaestroPoint('50%', '75%')).toBe('50%,75%'); +}); diff --git a/src/compat/maestro/__tests__/file-execution.test.ts b/src/compat/maestro/__tests__/file-execution.test.ts new file mode 100644 index 000000000..a34b70702 --- /dev/null +++ b/src/compat/maestro/__tests__/file-execution.test.ts @@ -0,0 +1,37 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test, vi } from 'vitest'; +import { executeMaestroFile } from '../file-execution.ts'; +import type { MaestroRuntimePort } from '../engine-types.ts'; + +test('executes root and included Maestro files through one typed runtime port', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-file-')); + const childPath = path.join(root, 'child.yaml'); + const mainPath = path.join(root, 'main.yaml'); + fs.writeFileSync(childPath, '---\n- inputText: ${VALUE}\n'); + fs.writeFileSync( + mainPath, + ['appId: com.example.app', '---', '- runFlow:', ' file: child.yaml'].join('\n'), + ); + const execute = vi.fn(async () => ({ mutated: true })); + const port: MaestroRuntimePort = { + execute, + observe: vi.fn(async ({ generation }) => ({ generation, matched: true })), + }; + + const result = await executeMaestroFile({ + filePath: mainPath, + port, + env: { VALUE: 'runtime' }, + platform: 'android', + }); + + expect(execute).toHaveBeenCalledWith( + expect.objectContaining({ + appId: 'com.example.app', + command: expect.objectContaining({ kind: 'inputText', text: 'runtime' }), + }), + ); + expect(result).toMatchObject({ executed: 2, generation: 1 }); +}); diff --git a/src/compat/maestro/__tests__/launch-flow.test.ts b/src/compat/maestro/__tests__/launch-flow.test.ts deleted file mode 100644 index f66f9a3c1..000000000 --- a/src/compat/maestro/__tests__/launch-flow.test.ts +++ /dev/null @@ -1,32 +0,0 @@ -import assert from 'node:assert/strict'; -import { test } from 'vitest'; -import type { SessionAction } from '../../../daemon/types.ts'; -import { parseMaestroReplayFlow } from '../replay-flow.ts'; - -test('bare launchApp uses Maestro default stop-and-relaunch semantics', () => { - const parsed = parseMaestroReplayFlow(`appId: com.pagerviewexample ---- -- launchApp -`); - - assert.deepEqual(projectAction(parsed.actions[0]), [ - 'open', - ['com.pagerviewexample'], - { relaunch: true }, - ]); -}); - -test('launchApp stopApp false opts out of relaunch', () => { - const parsed = parseMaestroReplayFlow(`appId: com.pagerviewexample ---- -- launchApp: - stopApp: false -`); - - assert.deepEqual(projectAction(parsed.actions[0]), ['open', ['com.pagerviewexample'], {}]); -}); - -function projectAction(action: SessionAction | undefined) { - if (!action) throw new Error('expected parsed action'); - return [action.command, action.positionals, action.flags]; -} diff --git a/src/compat/maestro/__tests__/points.test.ts b/src/compat/maestro/__tests__/points.test.ts deleted file mode 100644 index 8bf4b3807..000000000 --- a/src/compat/maestro/__tests__/points.test.ts +++ /dev/null @@ -1,45 +0,0 @@ -import assert from 'node:assert/strict'; -import { test, expect } from 'vitest'; -import { AppError } from '../../../kernel/errors.ts'; -import { formatMaestroPoint, parseAbsolutePoint, parseMaestroPoint } from '../points.ts'; - -test('formatMaestroPoint serializes coordinate pairs', () => { - expect(formatMaestroPoint(100, 200)).toBe('100,200'); - expect(formatMaestroPoint('50%', '75%')).toBe('50%,75%'); -}); - -test('parseMaestroPoint parses absolute pixel coordinates', () => { - expect(parseMaestroPoint('100,200')).toEqual({ kind: 'absolute', x: 100, y: 200 }); - expect(parseMaestroPoint(' 320 , 640 ')).toEqual({ kind: 'absolute', x: 320, y: 640 }); -}); - -test('parseMaestroPoint parses percentage coordinates including decimals', () => { - expect(parseMaestroPoint('50%,75%')).toEqual({ kind: 'percent', x: 50, y: 75 }); - expect(parseMaestroPoint('12.5%, 99.9%')).toEqual({ kind: 'percent', x: 12.5, y: 99.9 }); -}); - -test('parseMaestroPoint rejects mixed or malformed coordinate expressions', () => { - for (const value of ['50%,75', '50,75%', '100;200', '-10,20', '100.5,200', '']) { - assert.throws( - () => parseMaestroPoint(value), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - error.message.includes( - 'Only Maestro swipe coordinates like "100,200" or "50%,75%" are supported.', - ), - `expected rejection for ${JSON.stringify(value)}`, - ); - } -}); - -test('parseAbsolutePoint accepts only absolute pixel coordinates', () => { - expect(parseAbsolutePoint('100,200')).toEqual({ x: 100, y: 200 }); - assert.throws( - () => parseAbsolutePoint('50%,50%'), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - error.message.includes('Only absolute Maestro point selectors like "100,200" are supported.'), - ); -}); diff --git a/src/compat/maestro/__tests__/program-ir-parser.test.ts b/src/compat/maestro/__tests__/program-ir-parser.test.ts new file mode 100644 index 000000000..f3b941481 --- /dev/null +++ b/src/compat/maestro/__tests__/program-ir-parser.test.ts @@ -0,0 +1,284 @@ +import assert from 'node:assert/strict'; +import { describe, test } from 'vitest'; +import { parseMaestroProgram } from '../program-ir-parser.ts'; +import type { MaestroCommand } from '../program-ir.ts'; + +describe('parseMaestroProgram', () => { + test('preserves config, hooks, conditions, nested blocks, and source lines', () => { + const program = parseMaestroProgram( + [ + 'name: Checkout', + 'appId: example.app', + 'env:', + ' COUNT: ${COUNT}', + 'onFlowStart:', + ' - launchApp:', + ' clearState: true', + 'onFlowComplete:', + ' - takeScreenshot: final.png', + '---', + '- runFlow:', + ' when:', + ' platform: iOS', + ' true: "${maestro.platform == \'ios\'}"', + ' env:', + ' CHILD: nested', + ' commands:', + ' - tapOn:', + ' id: checkout-form', + ' - repeat:', + ' times: ${COUNT}', + ' commands:', + ' - assertVisible: Ready', + '- retry:', + ' maxRetries: 2', + ' commands:', + ' - pressKey: Enter', + ].join('\n'), + { sourcePath: '/flows/checkout.yaml' }, + ); + + assert.deepEqual(program.source, { path: '/flows/checkout.yaml', line: 1 }); + assert.deepEqual(program.config.env, { COUNT: '${COUNT}' }); + assert.equal(program.config.onFlowStart?.[0]?.kind, 'launchApp'); + assert.deepEqual(program.config.onFlowStart?.[0]?.source, { + path: '/flows/checkout.yaml', + line: 6, + }); + assert.equal(program.config.onFlowComplete?.[0]?.kind, 'takeScreenshot'); + assert.deepEqual( + program.commands.map((command) => command.kind), + ['runFlow', 'retry'], + ); + + const runFlow = commandOfKind(program.commands[0], 'runFlow'); + assert.deepEqual(runFlow.source, { path: '/flows/checkout.yaml', line: 11 }); + assert.deepEqual(runFlow.when, { + platform: 'ios', + true: "${maestro.platform == 'ios'}", + }); + assert.deepEqual(runFlow.env, { CHILD: 'nested' }); + assert.equal(runFlow.include.kind, 'commands'); + const inline = runFlow.include as Extract; + assert.deepEqual( + inline.commands.map((command) => command.source.line), + [18, 20], + ); + const repeat = commandOfKind(inline.commands[1], 'repeat'); + assert.equal(repeat.times, '${COUNT}'); + assert.equal(repeat.commands[0]?.kind, 'assertVisible'); + assert.equal(repeat.commands[0]?.source.line, 23); + + const retry = commandOfKind(program.commands[1], 'retry'); + assert.equal(retry.maxRetries, 2); + assert.equal(retry.commands[0]?.kind, 'pressKey'); + assert.equal(retry.commands[0]?.source.line, 27); + }); + + test('parses flow tags as typed metadata and validates each tag', () => { + const program = parseMaestroProgram( + ['name: Pager', 'tags: [smoke, pager]', '---', '- launchApp'].join('\n'), + ); + + assert.deepEqual(program.config.tags, ['smoke', 'pager']); + assert.throws( + () => parseMaestroProgram(['tags: [smoke, 7]', '---', '- launchApp'].join('\n')), + /tags\[1\].*expects a string.*line 1/i, + ); + }); + + test('keeps authored absolute, percentage, and target gesture spaces', () => { + const program = parseMaestroProgram(`--- +- tapOn: + point: 20%, 30% +- tapOn: + id: submit +- doubleTapOn: + point: 100,200 +- longPressOn: + id: hold +- swipe: + start: 100, 200 + end: 300, 400 +- swipe: + start: 90%, 50% + end: 10%, 50% +- swipe: + from: + id: handle + direction: LEFT +`); + + const [ + percentTap, + targetTap, + absoluteDoubleTap, + targetLongPress, + absoluteSwipe, + percentSwipe, + targetSwipe, + ] = program.commands; + const percentTapCommand = commandOfKind(percentTap, 'tapOn'); + const targetTapCommand = commandOfKind(targetTap, 'tapOn'); + const absoluteDoubleTapCommand = commandOfKind(absoluteDoubleTap, 'doubleTapOn'); + const targetLongPressCommand = commandOfKind(targetLongPress, 'longPressOn'); + const absoluteSwipeCommand = commandOfKind(absoluteSwipe, 'swipe'); + const percentSwipeCommand = commandOfKind(percentSwipe, 'swipe'); + const targetSwipeCommand = commandOfKind(targetSwipe, 'swipe'); + + assert.deepEqual(percentTapCommand.target, { space: 'percent', x: 20, y: 30 }); + assert.deepEqual(targetTapCommand.target, { space: 'target', selector: { id: 'submit' } }); + assert.deepEqual(absoluteDoubleTapCommand.target, { space: 'absolute', x: 100, y: 200 }); + assert.deepEqual(targetLongPressCommand.target, { + space: 'target', + selector: { id: 'hold' }, + }); + assert.deepEqual(absoluteSwipeCommand.gesture, { + kind: 'coordinates', + start: { space: 'absolute', x: 100, y: 200 }, + end: { space: 'absolute', x: 300, y: 400 }, + }); + assert.deepEqual(percentSwipeCommand.gesture, { + kind: 'coordinates', + start: { space: 'percent', x: 90, y: 50 }, + end: { space: 'percent', x: 10, y: 50 }, + }); + assert.deepEqual(targetSwipeCommand.gesture, { + kind: 'target', + from: { id: 'handle' }, + direction: 'left', + }); + }); + + test('keeps selector-map keys aligned with the supported command subset', () => { + const program = parseMaestroProgram(['---', '- tapOn:', ' label: Save'].join('\n')); + const tap = commandOfKind(program.commands[0], 'tapOn'); + assert.deepEqual(tap.target, { space: 'target', selector: { label: 'Save' } }); + + assert.throws( + () => parseMaestroProgram(['---', '- doubleTapOn:', ' label: Save'].join('\n')), + /doubleTapOn field "label" is not supported.*line 3/i, + ); + assert.throws( + () => parseMaestroProgram(['---', '- assertVisible:', ' label: Save'].join('\n')), + /assertVisible field "label" is not supported.*line 3/i, + ); + }); + + test('preserves an include boundary and the authored include path', () => { + const program = parseMaestroProgram( + `appId: example.app +--- +- runFlow: helpers/child.yaml +- tapOn: Continue +`, + { sourcePath: '/flows/main.yaml' }, + ); + + const include = commandOfKind(program.commands[0], 'runFlow'); + assert.deepEqual(include.include, { kind: 'file', path: 'helpers/child.yaml' }); + assert.deepEqual(include.source, { path: '/flows/main.yaml', line: 3 }); + assert.deepEqual(program.commands[1]?.source, { path: '/flows/main.yaml', line: 4 }); + }); + + test('keeps supported command values typed instead of lowering them to arguments', () => { + const program = parseMaestroProgram(`appId: example.app +--- +- launchApp: + appId: child.app + stopApp: false + arguments: + - --mode + - preview + launchArguments: + feature: true +- inputText: + text: Ada \${USER} + label: Full name +- eraseText: + charactersToErase: 4 +- openLink: + link: https://example.test +- extendedWaitUntil: + visible: + id: ready + timeout: 2500 +- scrollUntilVisible: + element: Checkout + direction: DOWN + timeout: 5000 +- runScript: + file: setup.js + env: + SERVER: local +`); + + assert.deepEqual(program.commands[0], { + kind: 'launchApp', + source: { line: 3 }, + appId: 'child.app', + stopApp: false, + arguments: { kind: 'list', values: ['--mode', 'preview'] }, + launchArguments: { kind: 'map', values: { feature: true } }, + }); + assert.deepEqual(program.commands[1], { + kind: 'inputText', + source: { line: 11 }, + text: 'Ada ${USER}', + label: 'Full name', + }); + assert.deepEqual(program.commands[3], { + kind: 'openLink', + source: { line: 16 }, + link: 'https://example.test', + }); + const wait = commandOfKind(program.commands[4], 'extendedWaitUntil'); + assert.deepEqual(wait.visible, { id: 'ready' }); + assert.equal(wait.timeout, 2500); + const scroll = commandOfKind(program.commands[5], 'scrollUntilVisible'); + assert.equal(scroll.direction, 'down'); + assert.equal(scroll.timeout, 5000); + assert.deepEqual(program.commands[6], { + kind: 'runScript', + source: { line: 26 }, + file: 'setup.js', + env: { SERVER: 'local' }, + }); + }); + + test('reports source lines for unsupported and invalid command shapes', () => { + assert.throws( + () => + parseMaestroProgram(`--- +- unsupportedCommand: true +`), + /unsupported.*line 2/i, + ); + assert.throws( + () => + parseMaestroProgram(`--- +- swipe: + start: 10,20 + end: 50%,60% +`), + /same coordinate space.*line 2/i, + ); + assert.throws( + () => + parseMaestroProgram(`--- +- runFlow: + when: {} + commands: [] +`), + /when cannot be empty.*line 3/i, + ); + }); +}); + +function commandOfKind( + command: MaestroCommand | undefined, + kind: K, +): Extract { + assert.equal(command?.kind, kind); + return command as Extract; +} diff --git a/src/compat/maestro/__tests__/program-loader.test.ts b/src/compat/maestro/__tests__/program-loader.test.ts new file mode 100644 index 000000000..220aea221 --- /dev/null +++ b/src/compat/maestro/__tests__/program-loader.test.ts @@ -0,0 +1,41 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test, vi } from 'vitest'; +import { createMaestroProgramLoader, resolveMaestroIncludePath } from '../program-loader.ts'; + +test('resolves nested Maestro includes relative to their parent source', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-loader-')); + const flowDir = path.join(root, 'flows'); + fs.mkdirSync(flowDir); + const childPath = path.join(flowDir, 'child.yaml'); + fs.writeFileSync(childPath, '---\n- inputText: child\n'); + + const loader = createMaestroProgramLoader(root); + const program = await loader('./child.yaml', path.join(flowDir, 'parent.yaml')); + + expect(program.source.path).toBe(childPath); + expect(program.commands[0]).toMatchObject({ kind: 'inputText', text: 'child' }); + expect(resolveMaestroIncludePath('./root.yaml', undefined, root)).toBe( + path.join(root, 'root.yaml'), + ); +}); + +test('caches parsed programs by resolved path and honors cancellation before I/O', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-loader-cache-')); + const childPath = path.join(root, 'child.yaml'); + fs.writeFileSync(childPath, '---\n- back\n'); + const readFileSync = vi.spyOn(fs, 'readFileSync'); + const loader = createMaestroProgramLoader(root); + + await loader('child.yaml'); + await loader('./child.yaml'); + expect(readFileSync.mock.calls.filter(([entry]) => entry === childPath)).toHaveLength(1); + + const controller = new AbortController(); + controller.abort(); + await expect(loader('missing.yaml', undefined, controller.signal)).rejects.toMatchObject({ + details: { reason: 'request_canceled' }, + }); + readFileSync.mockRestore(); +}); diff --git a/src/compat/maestro/__tests__/progress.test.ts b/src/compat/maestro/__tests__/progress.test.ts new file mode 100644 index 000000000..fb816ba81 --- /dev/null +++ b/src/compat/maestro/__tests__/progress.test.ts @@ -0,0 +1,32 @@ +import { expect, test } from 'vitest'; +import { parseMaestroProgram } from '../program-ir-parser.ts'; +import { formatMaestroCommandProgress } from '../progress.ts'; + +test('formats progress directly from typed Maestro commands', () => { + const program = parseMaestroProgram( + [ + '---', + '- tapOn:', + ' id: next-page', + '- swipe:', + ' start: 90%, 50%', + ' end: 10%, 50%', + '- assertVisible: Page 2', + ].join('\n'), + ); + + expect(program.commands.map(formatMaestroCommandProgress)).toEqual([ + { command: 'tapOn', value: 'next-page' }, + { command: 'swipe', value: '90,50% to 10,50%' }, + { command: 'assertVisible', value: 'Page 2' }, + ]); +}); + +test('redacts typed values from progress output', () => { + const program = parseMaestroProgram('---\n- inputText: highly-sensitive\n- pasteText: secret\n'); + + expect(program.commands.map(formatMaestroCommandProgress)).toEqual([ + { command: 'inputText', value: '' }, + { command: 'pasteText', value: '' }, + ]); +}); diff --git a/src/compat/maestro/__tests__/replay-flow.test.ts b/src/compat/maestro/__tests__/replay-flow.test.ts deleted file mode 100644 index fb0c877d1..000000000 --- a/src/compat/maestro/__tests__/replay-flow.test.ts +++ /dev/null @@ -1,975 +0,0 @@ -import { test } from 'vitest'; -import assert from 'node:assert/strict'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { AppError } from '../../../kernel/errors.ts'; -import { parseMaestroReplayFlow } from '../replay-flow.ts'; - -test('parseMaestroReplayFlow converts a supported Maestro command subset', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab -env: - USER_NAME: Ada ---- -- launchApp -- tapOn: - id: home-open-form -- tapOn: - point: 20%,20% - label: Dismiss save password prompt -- doubleTapOn: - id: release-notice - delay: 150 -- longPressOn: - text: Agent Device Tester -- openLink: exp://localhost:8082 -- tapOn: Full name -- inputText: - text: Ada Lovelace - label: Full name -- assertVisible: - text: Checkout form -- assertNotVisible: - text: Missing banner -- extendedWaitUntil: - visible: - id: submit-order - timeout: 7000 -- scroll -- swipe: - start: 50%, 75% - end: 50%, 35% - duration: 300 -- swipe: - direction: LEFT -- scrollUntilVisible: - element: Discover - direction: UP -- takeScreenshot: ./screens/form.png -- hideKeyboard -- stopApp -`); - - assert.equal(parsed.metadata.env?.USER_NAME, 'Ada'); - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['open', ['com.callstack.agentdevicelab']], - ['__maestroTapOn', ['id="home-open-form"']], - ['__maestroTapPointPercent', ['20', '20']], - ['click', ['id="release-notice"']], - [ - 'click', - ['label="Agent Device Tester" || text="Agent Device Tester" || id="Agent Device Tester"'], - ], - ['open', ['exp://localhost:8082']], - ['__maestroTapOn', ['label="Full name" || text="Full name" || id="Full name"']], - ['type', ['Ada Lovelace']], - [ - '__maestroAssertVisible', - ['label="Checkout form" || text="Checkout form" || id="Checkout form"', '17000'], - ], - [ - '__maestroAssertNotVisible', - ['label="Missing banner" || text="Missing banner" || id="Missing banner"'], - ], - ['__maestroAssertVisible', ['id="submit-order"', '7000']], - ['scroll', ['down']], - ['__maestroSwipeScreen', ['percent', '50', '75', '50', '35', '300']], - ['__maestroSwipeScreen', ['direction', 'left']], - [ - '__maestroScrollUntilVisible', - ['label="Discover" || text="Discover" || id="Discover"', '5000', 'up'], - ], - ['screenshot', ['./screens/form.png']], - ['keyboard', ['dismiss']], - ['close', ['com.callstack.agentdevicelab']], - ], - ); - assert.equal(parsed.actions[3]?.flags.doubleTap, true); - assert.equal(parsed.actions[3]?.flags.intervalMs, 150); - assert.equal(parsed.actions[4]?.flags.holdMs, 3000); - assert.equal(parsed.actions[1]?.flags.maestro?.allowNonHittableCoordinateFallback, true); - assert.equal(parsed.actions[6]?.flags?.maestro?.allowNonHittableCoordinateFallback, true); - assert.equal(parsed.actions[10]?.flags.maestro?.allowAlreadyPastLoading, true); -}); - -test('parseMaestroReplayFlow maps iOS openLink through the app id when available', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- openLink: exp://localhost:8082 -`, - { platform: 'ios' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['open', ['com.callstack.agentdevicelab', 'exp://localhost:8082']]], - ); - assert.equal(parsed.actions[0]?.flags.maestro?.prewarmRunnerBeforeOpen, true); -}); - -test('parseMaestroReplayFlow maps Android openLink through the app id when available', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- openLink: exp://localhost:8082 -`, - { platform: 'android' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['open', ['com.callstack.agentdevicelab', 'exp://localhost:8082']]], - ); -}); - -test('parseMaestroReplayFlow maps Android openLink without package binding when appId is absent', () => { - const parsed = parseMaestroReplayFlow( - `--- -- openLink: exp://localhost:8082 -`, - { platform: 'android' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['open', ['exp://localhost:8082']]], - ); -}); - -test('parseMaestroReplayFlow converts Maestro nested selector compatibility syntax', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- eraseText -- eraseText: 12 -- tapOn: - id: childActionButton - childOf: - id: parent-row-secondary -- tapOn: - id: overflowButton - index: 0 -- tapOn: - label: Profile name metadata - text: Profile name -- swipe: - label: Drag item down - from: - id: reorder-handle - direction: UP - duration: 350 -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['type', ['\b'.repeat(50)]], - ['type', ['\b'.repeat(12)]], - [ - '__maestroTapOn', - ['id="childActionButton"', JSON.stringify({ childOf: 'id="parent-row-secondary"' })], - ], - ['__maestroTapOn', ['id="overflowButton"', JSON.stringify({ index: 0 })]], - ['__maestroTapOn', ['label="Profile name" || text="Profile name" || id="Profile name"']], - ['__maestroSwipeOn', ['id="reorder-handle"', 'up', '350']], - ], - ); -}); - -test('parseMaestroReplayFlow preserves runScript as an ordered runtime action', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-runscript-')); - const scriptPath = path.join(root, 'setup.js'); - const flowPath = path.join(root, 'flow.yml'); - fs.writeFileSync(scriptPath, `output.result = SERVER_PATH`); - - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- runScript: - file: ./setup.js - env: - SERVER_PATH: local -- inputText: \${output.result} -`, - { sourcePath: flowPath }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroRunScript', [scriptPath]], - ['type', ['${output.result}']], - ], - ); - assert.deepEqual(parsed.actions[0]?.flags.maestro?.runScriptEnv, { SERVER_PATH: 'local' }); -}); - -test('parseMaestroReplayFlow keeps focused inputText and pressKey Enter as separate actions', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- inputText: hello -- pressKey: Enter -- inputText: world -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['type', ['hello']], - ['__maestroPressEnter', []], - ['type', ['world']], - ], - ); - assert.deepEqual(parsed.actionLines, [3, 4, 5]); -}); - -test('parseMaestroReplayFlow keeps tapOn inputText without Enter on Maestro path', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- tapOn: - id: editableNameInput -- inputText: Saved list -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroTapOn', ['id="editableNameInput"']], - ['type', ['Saved list']], - ], - ); - assert.deepEqual(parsed.actionLines, [3, 5]); - assert.equal(parsed.actions[0]?.flags?.maestro?.allowNonHittableCoordinateFallback, true); -}); - -test('parseMaestroReplayFlow preserves optional tapOn before inputText without Enter', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- tapOn: - id: editableNameInput - optional: true -- inputText: Saved list -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroTapOn', ['id="editableNameInput"']], - ['type', ['Saved list']], - ], - ); - assert.deepEqual(parsed.actionLines, [3, 6]); - assert.equal(parsed.actions[0]?.flags?.maestro?.optional, true); - assert.equal(parsed.actions[0]?.flags?.maestro?.allowNonHittableCoordinateFallback, true); -}); - -test('parseMaestroReplayFlow coalesces tapOn inputText while preserving pressKey Enter submit', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- tapOn: - id: e2eProxyHeaderInput -- inputText: \${output.result} -- pressKey: Enter -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['wait', ['id="e2eProxyHeaderInput"', '30000']], - ['fill', ['id="e2eProxyHeaderInput"', '${output.result}']], - ['__maestroPressEnter', []], - ], - ); - assert.deepEqual(parsed.actionLines, [3, 3, 6]); - assert.equal(parsed.actions[1]?.flags?.maestro?.allowNonHittableCoordinateFallback, true); -}); - -test('parseMaestroReplayFlow does not coalesce text entry for non-input-looking targets', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- tapOn: Continue -- inputText: unexpected -- pressKey: Enter -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroTapOn', ['label="Continue" || text="Continue" || id="Continue"']], - ['type', ['unexpected']], - ['__maestroPressEnter', []], - ], - ); - assert.equal(parsed.actions[0]?.flags?.maestro?.allowNonHittableCoordinateFallback, undefined); -}); - -test('parseMaestroReplayFlow maps focused input commands to native type and keyboard actions', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- inputText: hello -- eraseText: - charactersToErase: 3 -- pasteText: pasted -- pressKey: Return -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['type', ['hello']], - ['type', ['\b'.repeat(3)]], - ['type', ['pasted']], - ['__maestroPressEnter', []], - ], - ); -}); - -test('parseMaestroReplayFlow rejects relative runScript paths without source path', () => { - assert.throws( - () => - parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- runScript: ./setup.js -`), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /runScript file paths/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow rejects unsupported Maestro commands', () => { - assert.throws( - () => parseMaestroReplayFlow('---\n- travelThroughTime: Save\n'), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /travelThroughTime/.test(error.message) && - /issues\/558/.test(error.message) && - /issues\/new/.test(error.message) && - /line 2/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow preserves selector state and absolute swipe commands', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- assertVisible: - id: shipping-pickup - selected: true -- swipe: - start: 100, 500 - end: 100, 200 - duration: 300 -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroAssertVisible', ['id="shipping-pickup" selected="true"', '17000']], - ['swipe', ['100', '500', '100', '200', '300']], - ], - ); - assert.deepEqual(parsed.actionLines, [3, 6]); -}); - -test('parseMaestroReplayFlow maps extendedWaitUntil.notVisible through Maestro visibility assertions', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- extendedWaitUntil: - notVisible: - text: Loading - timeout: 1200 -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroAssertNotVisible', ['label="Loading" || text="Loading" || id="Loading"', '1200']]], - ); -}); - -test('parseMaestroReplayFlow applies the Maestro default to extendedWaitUntil.visible', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- extendedWaitUntil: - visible: - text: Ready -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroAssertVisible', ['label="Ready" || text="Ready" || id="Ready"', '17000']]], - ); -}); - -test('parseMaestroReplayFlow rejects deferred Maestro utility commands loudly', () => { - assert.throws( - () => parseMaestroReplayFlow('---\n- assertTrue: "${READY}"\n'), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /assertTrue/.test(error.message) && - /issues\/558/.test(error.message) && - /line 2/.test(error.message), - ); - - assert.throws( - () => parseMaestroReplayFlow('---\n- setPermissions:\n camera: allow\n'), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /setPermissions/.test(error.message) && - /issues\/558/.test(error.message) && - /line 2/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow rejects unsupported fields instead of ignoring them', () => { - assert.throws( - () => - parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- tapOn: - id: submit-order - retryTapIfNoChange: true -`), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /retryTapIfNoChange/.test(error.message) && - /issues\/558/.test(error.message) && - /line 3/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow reports top-level command lines around nested lists', () => { - assert.throws( - () => - parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- runFlow: - commands: - - tapOn: Nested -- travelThroughTime: Save -`), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /travelThroughTime/.test(error.message) && - /line 6/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow flattens hooks, file runFlow, inline runFlow, env, and repeat times', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-flow-')); - const childPath = path.join(root, 'child.yaml'); - fs.writeFileSync( - childPath, - `appId: com.child.app ---- -- tapOn: "\${CHILD_LABEL}" -- repeat: - times: \${COUNT} - commands: - - tapOn: - id: child-repeat -`, - ); - - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab -env: - COUNT: "2" -onFlowStart: - - tapOn: Before -onFlowComplete: - - tapOn: After ---- -- runFlow: - file: child.yaml - env: - CHILD_LABEL: Nested -- runFlow: - when: - platform: iOS - commands: - - tapOn: iOS only -- repeat: - times: 2 - commands: - - tapOn: Again -`, - { sourcePath: path.join(root, 'main.yaml'), platform: 'ios' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [ - ['__maestroTapOn', ['label="Before" || text="Before" || id="Before"']], - ['__maestroTapOn', ['label="Nested" || text="Nested" || id="Nested"']], - ['__maestroTapOn', ['id="child-repeat"']], - ['__maestroTapOn', ['id="child-repeat"']], - ['__maestroTapOn', ['label="iOS only" || text="iOS only" || id="iOS only"']], - ['__maestroTapOn', ['label="Again" || text="Again" || id="Again"']], - ['__maestroTapOn', ['label="Again" || text="Again" || id="Again"']], - ['__maestroTapOn', ['label="After" || text="After" || id="After"']], - ], - ); -}); - -// ADR 0012 migration step 2: a `runFlow` include must not lose provenance — -// every inlined action reports the INCLUDE's own resolved path + line, not -// the including `runFlow:` command's line. Regression coverage for the -// live-reproduced "Replay failed at step 5 (__maestroTapOn ...)" bug, where -// no file or line appeared anywhere in the failure. -test('parseMaestroReplayFlow preserves the include file path and line through runFlow inlining', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-provenance-')); - const launchPath = path.join(root, 'launch.yml'); - const flowsDir = path.join(root, 'flows'); - fs.mkdirSync(flowsDir, { recursive: true }); - const mainPath = path.join(flowsDir, 'main.yaml'); - fs.writeFileSync( - launchPath, - `appId: com.callstack.agentdevicelab ---- -- launchApp -- tapOn: Welcome -- tapOn: Push Input -`, - ); - fs.writeFileSync( - mainPath, - `appId: com.callstack.agentdevicelab ---- -- tapOn: Before -- runFlow: - file: ../launch.yml -- tapOn: After -`, - ); - - const parsed = parseMaestroReplayFlow(fs.readFileSync(mainPath, 'utf8'), { - sourcePath: mainPath, - platform: 'ios', - }); - - assert.deepEqual( - parsed.actions.map((entry) => entry.command), - ['__maestroTapOn', 'open', '__maestroTapOn', '__maestroTapOn', '__maestroTapOn'], - ); - // "tapOn: Before" (main.yaml line 3) — no include, so the top-level path - // stays `undefined` (the caller's own file). - assert.equal(parsed.actionSourcePaths?.[0], undefined); - assert.equal(parsed.actionLines[0], 3); - // Every action inlined from the include (launchApp/tapOn Welcome/tapOn - // Push Input) reports launch.yml's OWN path and its OWN line — never - // main.yaml's `runFlow:` line (4). - for (const index of [1, 2, 3]) { - assert.equal(parsed.actionSourcePaths?.[index], launchPath); - } - assert.equal(parsed.actionLines[1], 3); // launchApp - assert.equal(parsed.actionLines[2], 4); // tapOn: Welcome - assert.equal(parsed.actionLines[3], 5); // tapOn: Push Input — the live-bug target - // "tapOn: After" (main.yaml line 6) returns to the top-level file. - assert.equal(parsed.actionSourcePaths?.[4], undefined); - assert.equal(parsed.actionLines[4], 6); -}); - -test('parseMaestroReplayFlow preserves provenance through a nested (include-of-include) runFlow', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-provenance-nested-')); - const grandchildPath = path.join(root, 'grandchild.yaml'); - const childPath = path.join(root, 'child.yaml'); - const mainPath = path.join(root, 'main.yaml'); - fs.writeFileSync( - grandchildPath, - `appId: com.callstack.agentdevicelab ---- -- tapOn: Deepest -`, - ); - fs.writeFileSync( - childPath, - `appId: com.callstack.agentdevicelab ---- -- tapOn: Shallow -- runFlow: - file: grandchild.yaml -`, - ); - fs.writeFileSync( - mainPath, - `appId: com.callstack.agentdevicelab ---- -- runFlow: - file: child.yaml -`, - ); - - const parsed = parseMaestroReplayFlow(fs.readFileSync(mainPath, 'utf8'), { - sourcePath: mainPath, - platform: 'ios', - }); - - assert.deepEqual( - parsed.actions.map((entry) => entry.command), - ['__maestroTapOn', '__maestroTapOn'], - ); - assert.equal(parsed.actionSourcePaths?.[0], childPath); - assert.equal(parsed.actionLines[0], 3); // tapOn: Shallow, in child.yaml - assert.equal(parsed.actionSourcePaths?.[1], grandchildPath); - assert.equal(parsed.actionLines[1], 3); // tapOn: Deepest, in grandchild.yaml -}); - -// Regression: provenance travels ONLY in the parallel structures -// (actionSourcePaths / replayControl.actionSources) — no per-action -// provenance field may (re)appear anywhere in parsed output. -function collectReplaySourceLeaks(actions: unknown[], leaks: string[] = [], prefix = ''): string[] { - for (const [index, entry] of actions.entries()) { - const action = entry as { - command?: string; - replaySource?: unknown; - replayControl?: { actions?: unknown[] }; - }; - if (action.replaySource !== undefined) { - leaks.push(`${prefix}[${index}] ${String(action.command)}`); - } - if (Array.isArray(action.replayControl?.actions)) { - collectReplaySourceLeaks(action.replayControl.actions, leaks, `${prefix}[${index}].control`); - } - } - return leaks; -} - -test('parseMaestroReplayFlow carries retry-wrapped include provenance in replayControl.actionSources', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-retry-provenance-')); - const childPath = path.join(root, 'child.yaml'); - const mainPath = path.join(root, 'main.yaml'); - fs.writeFileSync( - childPath, - `appId: com.callstack.agentdevicelab ---- -- back -- tapOn: Push Input -`, - ); - fs.writeFileSync( - mainPath, - `appId: com.callstack.agentdevicelab ---- -- retry: - maxRetries: 2 - commands: - - runFlow: - file: child.yaml -`, - ); - - const parsed = parseMaestroReplayFlow(fs.readFileSync(mainPath, 'utf8'), { - sourcePath: mainPath, - platform: 'ios', - }); - - // One wrapping retry action; the wrapper itself is a root-file step. - assert.equal(parsed.actions.length, 1); - const wrapper = parsed.actions[0]!; - assert.equal(wrapper.command, 'retry'); - assert.equal(parsed.actionSourcePaths?.[0], undefined); - assert.equal(parsed.actionLines[0], 3); - - // No per-action provenance field anywhere in the parsed output. - assert.deepEqual(collectReplaySourceLeaks(parsed.actions), []); - - // The include's provenance lives in replayControl.actionSources. - const control = wrapper.replayControl; - if (control?.kind !== 'retry') throw new Error('expected retry control'); - assert.equal(control.actions.length, 2); - assert.deepEqual(control.actionSources, [ - { path: childPath, line: 3 }, // back - { path: childPath, line: 4 }, // tapOn: Push Input - ]); -}); - -test('parseMaestroReplayFlow carries when-wrapped include provenance in replayControl.actionSources', () => { - const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-when-provenance-')); - const childPath = path.join(root, 'child.yaml'); - const mainPath = path.join(root, 'main.yaml'); - fs.writeFileSync( - childPath, - `appId: com.callstack.agentdevicelab ---- -- back -`, - ); - fs.writeFileSync( - mainPath, - `appId: com.callstack.agentdevicelab ---- -- runFlow: - file: child.yaml - when: - visible: Continue -`, - ); - - const parsed = parseMaestroReplayFlow(fs.readFileSync(mainPath, 'utf8'), { - sourcePath: mainPath, - platform: 'ios', - }); - - assert.equal(parsed.actions.length, 1); - const wrapper = parsed.actions[0]!; - assert.equal(wrapper.command, 'runFlow.when'); - assert.deepEqual(collectReplaySourceLeaks(parsed.actions), []); - - const control = wrapper.replayControl; - if (control?.kind !== 'maestroRunFlowWhen') throw new Error('expected runFlow.when control'); - assert.deepEqual(control.actionSources, [{ path: childPath, line: 3 }]); -}); - -test('parseMaestroReplayFlow skips platform-gated runFlow commands for other platforms', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- runFlow: - when: - platform: Android - commands: - - tapOn: Android only -- tapOn: Shared -`, - { platform: 'ios' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroTapOn', ['label="Shared" || text="Shared" || id="Shared"']]], - ); -}); - -test('parseMaestroReplayFlow treats Web platform gates as non-native branches', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- runFlow: - when: - platform: Web - commands: - - tapOn: Web only -- tapOn: Native -`, - { platform: 'ios' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroTapOn', ['label="Native" || text="Native" || id="Native"']]], - ); -}); - -test('parseMaestroReplayFlow evaluates simple runFlow.when.true platform expressions', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- runFlow: - when: - true: \${maestro.platform == 'android' || maestro.platform == 'ios'} - commands: - - tapOn: Native -- runFlow: - when: - true: \${maestro.platform == 'web' || maestro.platform == 'android'} - commands: - - tapOn: Not iOS -`, - { platform: 'ios' }, - ); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroTapOn', ['label="Native" || text="Native" || id="Native"']]], - ); -}); - -test('parseMaestroReplayFlow keeps visible-gated runFlow commands for runtime evaluation', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- runFlow: - when: - visible: Continue - commands: - - tapOn: Continue -`, - { platform: 'ios' }, - ); - - assert.equal(parsed.actions[0]?.command, 'runFlow.when'); - assert.deepEqual(parsed.actions[0]?.positionals, [ - 'visible', - 'label="Continue" || text="Continue" || id="Continue"', - ]); - const control = parsed.actions[0]?.replayControl; - assert.equal(control?.kind, 'maestroRunFlowWhen'); - if (control?.kind !== 'maestroRunFlowWhen') throw new Error('expected runFlow.when control'); - assert.equal(control.mode, 'visible'); - assert.equal(control.selector, 'label="Continue" || text="Continue" || id="Continue"'); - assert.deepEqual( - control.actions.map((entry) => [entry.command, entry.positionals, entry.flags]), - [ - [ - '__maestroTapOn', - ['label="Continue" || text="Continue" || id="Continue"'], - { maestro: { allowNonHittableCoordinateFallback: true } }, - ], - ], - ); -}); - -test('parseMaestroReplayFlow keeps retry commands for runtime evaluation', () => { - const parsed = parseMaestroReplayFlow( - `appId: com.callstack.agentdevicelab ---- -- retry: - maxRetries: 3 - commands: - - openLink: - link: \${APP_SCHEME}details - - assertVisible: Article -`, - { env: { APP_SCHEME: 'example://' } }, - ); - - assert.equal(parsed.actions[0]?.command, 'retry'); - assert.deepEqual(parsed.actions[0]?.positionals, ['3']); - const control = parsed.actions[0]?.replayControl; - assert.equal(control?.kind, 'retry'); - if (control?.kind !== 'retry') throw new Error('expected retry control'); - assert.equal(control.maxRetries, 3); - assert.deepEqual( - control.actions.map((entry) => [entry.command, entry.positionals, entry.flags]), - [ - ['open', ['example://details'], {}], - [ - '__maestroAssertVisible', - ['label="Article" || text="Article" || id="Article"', '17000'], - {}, - ], - ], - ); -}); - -test('parseMaestroReplayFlow accepts launchApp reset options', () => { - const parsed = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- launchApp: - clearState: true - arguments: - "-EXDevMenuIsOnboardingFinished": true - launchArguments: - "-Example": "ignored" - stopApp: true -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals, entry.flags]), - [ - [ - 'open', - ['com.callstack.agentdevicelab'], - { - clearAppState: true, - launchArgs: ['-EXDevMenuIsOnboardingFinished', 'true', '-Example', 'ignored'], - }, - ], - ], - ); -}); - -test('parseMaestroReplayFlow rejects clearKeychain instead of ignoring it', () => { - assert.throws( - () => - parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- launchApp: - clearKeychain: true -`), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /clearKeychain/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow relaunches launchApp only when clearState is absent', () => { - const withLaunchArgs = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- launchApp: - arguments: - "-Example": "value" -`); - const withStopApp = parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- launchApp: - stopApp: true -`); - - assert.equal(withLaunchArgs.actions[0]?.flags.relaunch, true); - assert.equal(withStopApp.actions[0]?.flags.relaunch, true); -}); - -test('parseMaestroReplayFlow rejects unsupported runtime-dependent flow control', () => { - assert.throws( - () => - parseMaestroReplayFlow(`appId: com.callstack.agentdevicelab ---- -- repeat: - while: - notVisible: Done - times: 3 - commands: - - tapOn: Again -`), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /repeat.while/.test(error.message) && - /line 3/.test(error.message), - ); -}); - -test('parseMaestroReplayFlow parses the test-app Maestro suite fixture', () => { - const fixturePath = path.resolve('examples/test-app/maestro/checkout-form.yaml'); - const parsed = parseMaestroReplayFlow(fs.readFileSync(fixturePath, 'utf8'), { - sourcePath: fixturePath, - platform: 'ios', - }); - - assert.deepEqual( - parsed.actions.map((entry) => entry.command), - [ - 'open', - '__maestroAssertVisible', - '__maestroScrollUntilVisible', - '__maestroTapOn', - '__maestroAssertVisible', - '__maestroTapOn', - 'type', - '__maestroTapOn', - 'type', - '__maestroTapOn', - '__maestroAssertVisible', - '__maestroAssertVisible', - '__maestroSwipeScreen', - '__maestroTapOn', - '__maestroAssertVisible', - '__maestroTapOn', - '__maestroAssertVisible', - '__maestroTapOn', - '__maestroAssertVisible', - '__maestroAssertVisible', - ], - ); - assert.equal(parsed.actions[0]?.flags.clearAppState, true); -}); diff --git a/src/compat/maestro/__tests__/replay-plan.test.ts b/src/compat/maestro/__tests__/replay-plan.test.ts new file mode 100644 index 000000000..b5c2b4447 --- /dev/null +++ b/src/compat/maestro/__tests__/replay-plan.test.ts @@ -0,0 +1,137 @@ +import { describe, expect, test, vi } from 'vitest'; +import { executeMaestroProgram } from '../engine.ts'; +import type { MaestroRuntimePort } from '../engine-types.ts'; +import { parseMaestroProgram } from '../program-ir-parser.ts'; +import { compileMaestroReplayPlan, evaluateMaestroReplayResume } from '../replay-plan.ts'; + +describe('typed Maestro replay plan', () => { + test('expands static hooks, includes, and repeats while retaining dynamic controls', async () => { + const child = parseMaestroProgram('---\n- inputText: child\n', { + sourcePath: '/flows/child.yaml', + }); + const program = parseMaestroProgram( + [ + 'appId: com.example.app', + 'env:', + ' FLOW: config', + 'onFlowStart:', + ' - inputText: start', + 'onFlowComplete:', + ' - inputText: complete', + '---', + '- repeat:', + ' times: 2', + ' commands:', + ' - back', + '- runFlow: ${INCLUDE}', + '- runFlow:', + ' when:', + ' platform: iOS', + ' commands:', + ' - inputText: omitted', + '- retry:', + ' maxRetries: 1', + ' commands:', + ' - inputText: retry-body', + '- runScript: setup.js', + ].join('\n'), + { sourcePath: '/flows/main.yaml' }, + ); + const loadProgram = vi.fn(async () => child); + + const plan = await compileMaestroReplayPlan(program, { + platform: 'android', + target: 'simulator', + runtimeHints: { platform: 'android', metroHost: '127.0.0.1', metroPort: 8083 }, + defaults: { BUILTIN: 'default' }, + env: { INCLUDE: 'child.yaml', FLOW: 'runtime' }, + loadProgram, + }); + + expect(plan.steps.map((step) => step.command.kind)).toEqual([ + 'inputText', + 'back', + 'back', + 'inputText', + 'retry', + 'runScript', + 'inputText', + ]); + expect(plan.steps[3]?.source.path).toBe('/flows/child.yaml'); + expect(plan.steps[4]).toMatchObject({ + kind: 'opaque', + body: [expect.objectContaining({ command: expect.objectContaining({ kind: 'inputText' }) })], + }); + expect(plan.initialStaticEnv).toEqual({ + BUILTIN: 'default', + FLOW: 'runtime', + INCLUDE: 'child.yaml', + }); + expect(plan.runtimeHints).toEqual({ + platform: 'android', + metroHost: '127.0.0.1', + metroPort: 8083, + }); + expect(loadProgram).toHaveBeenCalledWith('child.yaml', '/flows/main.yaml'); + expect(Object.isFrozen(plan)).toBe(true); + + const changed = await compileMaestroReplayPlan(program, { + platform: 'android', + target: 'simulator', + env: { INCLUDE: 'other.yaml' }, + loadProgram, + }); + expect(changed.digest).not.toBe(plan.digest); + + const changedRuntime = await compileMaestroReplayPlan(program, { + platform: 'android', + target: 'simulator', + runtimeHints: { platform: 'android', metroHost: '127.0.0.1', metroPort: 8084 }, + defaults: { BUILTIN: 'default' }, + env: { INCLUDE: 'child.yaml', FLOW: 'runtime' }, + loadProgram, + }); + expect(changedRuntime.digest).not.toBe(plan.digest); + + expect(evaluateMaestroReplayResume(plan, { from: 4, planDigest: plan.digest })).toEqual({ + allowed: true, + startIndex: 3, + }); + expect(evaluateMaestroReplayResume(plan, { from: 5, planDigest: plan.digest })).toMatchObject({ + allowed: false, + reason: expect.stringContaining('cannot be resumed'), + }); + expect(evaluateMaestroReplayResume(plan, { from: 6, planDigest: plan.digest })).toMatchObject({ + allowed: false, + reason: expect.stringContaining('cannot be skipped'), + }); + }); + + test('executes from a stable plan index and reports plan ordinals', async () => { + const program = parseMaestroProgram('---\n- inputText: first\n- inputText: second\n'); + const execute = vi.fn(async () => ({ mutated: true })); + const observer = { commandStarted: vi.fn() }; + const port: MaestroRuntimePort = { + execute, + observe: vi.fn(async ({ generation }) => ({ generation, matched: true })), + }; + + const initialPlan = await compileMaestroReplayPlan(program); + await executeMaestroProgram(program, port, { + from: 2, + planDigest: initialPlan.digest, + observer, + }); + + expect(execute).toHaveBeenCalledOnce(); + expect(execute).toHaveBeenCalledWith( + expect.objectContaining({ + command: expect.objectContaining({ text: 'second' }), + env: {}, + }), + ); + expect(observer.commandStarted).toHaveBeenCalledWith( + expect.objectContaining({ stepIndex: 2, stepTotal: 2 }), + ); + }); +}); diff --git a/src/compat/maestro/__tests__/run-script.test.ts b/src/compat/maestro/__tests__/run-script.test.ts new file mode 100644 index 000000000..39e2e0845 --- /dev/null +++ b/src/compat/maestro/__tests__/run-script.test.ts @@ -0,0 +1,97 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test } from 'vitest'; +import { AppError } from '../../../kernel/errors.ts'; +import { executeRunScriptFile } from '../run-script-execution.ts'; + +test('executeRunScriptFile exposes env and serializes output values', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-run-script-')); + const scriptPath = path.join(root, 'setup.js'); + fs.writeFileSync( + scriptPath, + ` +output.text = SERVER_PATH +output.number = 42 +output.boolean = false +output.object = { ready: true } +`, + ); + + try { + expect(executeRunScriptFile({ scriptPath, env: { SERVER_PATH: 'local' } })).toEqual({ + 'output.text': 'local', + 'output.number': '42', + 'output.boolean': 'false', + 'output.object': '{"ready":true}', + }); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}); + +test('executeRunScriptFile rejects output keys that cannot become replay variables', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-run-script-')); + const scriptPath = path.join(root, 'setup.js'); + fs.writeFileSync(scriptPath, `output['nested.value'] = 'ambiguous'`); + + try { + expect(() => executeRunScriptFile({ scriptPath, env: {} })).toThrowError(AppError); + try { + executeRunScriptFile({ scriptPath, env: {} }); + } catch (error) { + expect(error).toBeInstanceOf(AppError); + expect((error as AppError).code).toBe('INVALID_ARGS'); + expect((error as AppError).message).toContain('output key cannot contain'); + expect((error as AppError).details).toEqual({ scriptPath, key: 'nested.value' }); + } + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}); + +test('executeRunScriptFile keeps recovery guidance separate from its bounded error message', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-run-script-')); + const scriptPath = path.join(root, 'setup.js'); + fs.writeFileSync(scriptPath, `output.result = json('').value`); + + try { + executeRunScriptFile({ scriptPath, env: {} }); + throw new Error('expected runScript to fail'); + } catch (error) { + expect(error).toBeInstanceOf(AppError); + expect((error as AppError).message).toBe( + 'Maestro runScript failed: Maestro runScript json() received an empty body.', + ); + expect((error as AppError).details).toEqual({ + hint: 'Check the preceding HTTP response status and setup server output.', + scriptPath, + }); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}); + +test('executeRunScriptFile strips prototype keys from json output', () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-run-script-')); + const scriptPath = path.join(root, 'setup.js'); + fs.writeFileSync( + scriptPath, + ` +const parsed = json('{"safe":1,"__proto__":{"polluted":true},"nested":{"prototype":{"polluted":true},"ok":2}}') +output.result = [ + Object.prototype.hasOwnProperty.call(parsed, '__proto__'), + Object.prototype.hasOwnProperty.call(parsed.nested, 'prototype'), + parsed.nested.ok +].join(':') +`, + ); + + try { + expect(executeRunScriptFile({ scriptPath, env: {} })).toEqual({ + 'output.result': 'false:false:2', + }); + } finally { + fs.rmSync(root, { recursive: true, force: true }); + } +}); diff --git a/src/compat/maestro/__tests__/runtime-assertions.test.ts b/src/compat/maestro/__tests__/runtime-assertions.test.ts deleted file mode 100644 index 2fd14409d..000000000 --- a/src/compat/maestro/__tests__/runtime-assertions.test.ts +++ /dev/null @@ -1,969 +0,0 @@ -import assert from 'node:assert/strict'; -import fs from 'node:fs'; -import os from 'node:os'; -import path from 'node:path'; -import { afterEach, test, vi } from 'vitest'; -import { - invokeMaestroAssertNotVisible, - invokeMaestroAssertVisible, -} from '../runtime-assertions.ts'; -import { invokeMaestroSwipeScreen, invokeMaestroTapOn } from '../runtime-interactions.ts'; -import { rememberMaestroRecoverableInteraction } from '../runtime-support.ts'; -import type { DaemonRequest, DaemonResponse } from '../../../daemon/types.ts'; -import type { SnapshotState } from '../../../kernel/snapshot.ts'; - -afterEach(() => { - vi.restoreAllMocks(); - vi.useRealTimers(); -}); - -test('invokeMaestroAssertVisible takes a terminal snapshot when the last miss started before the deadline', async () => { - vi.spyOn(Date, 'now') - .mockReturnValueOnce(0) - .mockReturnValueOnce(1000) - .mockReturnValueOnce(6500) - .mockReturnValueOnce(6500) - .mockReturnValueOnce(6600); - - let snapshots = 0; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['id="details-preloaded"', '5000'], - invoke: async (): Promise => { - snapshots += 1; - if (snapshots === 1) { - return { ok: true, data: { createdAt: 1, nodes: [] } }; - } - return { - ok: true, - data: { - createdAt: 2, - nodes: [node('Details is preloaded!', { identifier: 'details-preloaded' })], - }, - }; - }, - }); - - assert.equal(response.ok, true); - assert.equal(snapshots, 2); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.nodeLabel, 'Details is preloaded!'); - assert.equal(response.data.waitedMs, 6500); - } -}); - -test('invokeMaestroAssertVisible retries transient snapshot failures until a later match', async () => { - vi.useFakeTimers(); - - let snapshots = 0; - const responsePromise = invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['id="ready-state"', '1000'], - invoke: async (): Promise => { - snapshots += 1; - if (snapshots === 1) { - return { - ok: false, - error: { code: 'SNAPSHOT_FAILED', message: 'Snapshot temporarily unavailable.' }, - }; - } - return { - ok: true, - data: { - createdAt: 2, - nodes: [node('Ready', { identifier: 'ready-state' })], - }, - }; - }, - }); - - await vi.advanceTimersByTimeAsync(250); - const response = await responsePromise; - - assert.equal(response.ok, true); - assert.equal(snapshots, 2); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.nodeLabel, 'Ready'); - assert.equal(response.data.waitedMs, 250); - } -}); - -test('invokeMaestroAssertVisible uses native wait for short simple iOS assertions', async () => { - const calls: Array<[string, string[] | undefined]> = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['label="Ready" || text="Ready" || id="Ready"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { ok: true, data: { matches: 1 } }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [['wait', ['Ready', '1000']]]); -}); - -test('invokeMaestroAssertVisible uses the Maestro default timeout when omitted', async () => { - const calls: Array<[string, string[] | undefined]> = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['label="Ready" || text="Ready" || id="Ready"'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { ok: true, data: { matches: 1 } }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [['wait', ['Ready', '17000']]]); -}); - -test('invokeMaestroAssertVisible verifies Android native wait success with exact snapshot matching', async () => { - const calls: Array<[string, string[] | undefined]> = []; - let snapshots = 0; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['label="Albums" || text="Albums" || id="Albums"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { ok: true, data: { matches: 1 } }; - } - if (req.command === 'snapshot') { - snapshots += 1; - return { - ok: true, - data: snapshot([snapshots === 1 ? node('Push albums') : node('Albums')], snapshots), - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [ - ['wait', ['Albums', '1000']], - ['snapshot', []], - ['snapshot', []], - ]); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.nodeLabel, 'Albums'); - } -}); - -test('invokeMaestroAssertVisible falls back to one snapshot after native wait misses', async () => { - const calls: Array<[string, string[] | undefined]> = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['label="Ready" || text="Ready" || id="Ready"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Ready' }, - }; - } - if (req.command === 'snapshot') { - return { - ok: true, - data: snapshot([node('Ready')]), - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [ - ['wait', ['Ready', '1000']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertVisible re-resolves the previous Android tap when its target remains visible', async () => { - const scope = { values: {} }; - const calls: Array<[string, string[] | undefined]> = []; - rememberMaestroRecoverableInteraction(scope, { - kind: 'tap', - selector: 'label="Go to Contacts" || text="Go to Contacts" || id="Go to Contacts"', - point: { x: 999, y: 999 }, - }); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: ['label="Marissa Castillo" || text="Marissa Castillo" || id="Marissa Castillo"'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - const waitCalls = calls.filter(([command]) => command === 'wait').length; - if (waitCalls === 2) return { ok: true, data: { matches: 1 } }; - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Marissa Castillo' }, - }; - } - if (req.command === 'snapshot') { - return { - ok: true, - data: snapshot([ - node('Go to Contacts', { - type: 'android.widget.Button', - identifier: 'go-to-contacts', - }), - ]), - }; - } - if (req.command === 'click') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [ - ['wait', ['Marissa Castillo', '17000']], - ['snapshot', []], - ['click', ['80', '100']], - ['wait', ['Marissa Castillo', '5000']], - ]); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.retryTap, true); - } -}); - -test('invokeMaestroAssertVisible retries previous Android text tap when point resolution misses', async () => { - const scope = { values: {} }; - const calls: Array<[string, string[] | undefined]> = []; - rememberMaestroRecoverableInteraction(scope, { - kind: 'tap', - selector: 'label="Push article" || text="Push article" || id="Push article"', - point: { x: 999, y: 999 }, - }); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: [ - 'label="Article by The Doctor" || text="Article by The Doctor" || id="Article by The Doctor"', - ], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - const waitCalls = calls.filter(([command]) => command === 'wait').length; - if (waitCalls === 2) return { ok: true, data: { matches: 1 } }; - return { - ok: false, - error: { - code: 'COMMAND_FAILED', - message: 'wait timed out for text: Article by The Doctor', - }, - }; - } - if (req.command === 'snapshot') { - return { - ok: true, - data: snapshot([ - node('Push article', { - type: 'android.widget.Button', - identifier: undefined, - rect: undefined, - }), - ]), - }; - } - if (req.command === 'find') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual(calls, [ - ['wait', ['Article by The Doctor', '17000']], - ['snapshot', []], - ['find', ['Push article', 'click']], - ['wait', ['Article by The Doctor', '5000']], - ]); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.retryTap, true); - } -}); - -test('invokeMaestroAssertVisible does not retry stale Android taps after swipes', async () => { - const scope = { values: {} }; - const calls: Array<[string, string[] | undefined]> = []; - rememberMaestroRecoverableInteraction(scope, { - kind: 'tap', - selector: 'label="Contacts" || text="Contacts" || id="Contacts"', - point: { x: 120, y: 720 }, - }); - - const swipeResponse = await invokeMaestroSwipeScreen({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: ['direction', 'left', '300'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'gesture') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - assert.equal(swipeResponse.ok, true); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: [ - 'label="What is Lorem Ipsum?" || text="What is Lorem Ipsum?" || id="What is Lorem Ipsum?"', - '2000', - ], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { - code: 'COMMAND_FAILED', - message: 'wait timed out for text: What is Lorem Ipsum?', - }, - }; - } - if (req.command === 'snapshot') { - return { - ok: true, - data: snapshot([node('Contacts'), node('Albums')]), - }; - } - if (req.command === 'gesture') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - assert.deepEqual(calls, [ - ['gesture', []], - ['wait', ['What is Lorem Ipsum?', '2000']], - ['snapshot', []], - ['gesture', []], - ['wait', ['What is Lorem Ipsum?', '2000']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertVisible does not replay a previous iOS swipe after an AX miss', async () => { - const scope = { values: {} }; - const calls: Array<[string, string[] | undefined]> = []; - rememberMaestroRecoverableInteraction(scope, { - kind: 'swipe', - command: 'gesture', - input: { kind: 'swipe', preset: 'left', durationMs: 300 }, - }); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - scope, - positionals: ['label="Second page" || text="Second page" || id="Second page"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Second page' }, - }; - } - if (req.command === 'snapshot') return { ok: true, data: snapshot([node('First page')]) }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - assert.deepEqual(calls, [ - ['wait', ['Second page', '1000']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertVisible does not retry stale Android taps after fuzzy taps', async () => { - const scope = { values: {} }; - const calls: Array<[string, string[] | undefined]> = []; - rememberMaestroRecoverableInteraction(scope, { - kind: 'tap', - selector: 'label="Contacts" || text="Contacts" || id="Contacts"', - point: { x: 120, y: 720 }, - }); - - const tapResponse = await invokeMaestroTapOn({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: ['label="Search" || text="Search" || id="Search"'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'snapshot') return { ok: true, data: snapshot([node('Search')]) }; - if (req.command === 'click') { - return { ok: false, error: { code: 'COMMAND_FAILED', message: 'coordinate miss' } }; - } - if (req.command === 'find') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(tapResponse.ok, true); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - scope, - positionals: [ - 'label="Marissa Castillo" || text="Marissa Castillo" || id="Marissa Castillo"', - '0', - ], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Marissa Castillo' }, - }; - } - if (req.command === 'snapshot') return { ok: true, data: snapshot([node('Settings')]) }; - if (req.command === 'click') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - assert.deepEqual(calls, [ - ['snapshot', []], - ['click', ['80', '100']], - ['find', ['Search', 'click']], - ['wait', ['Marissa Castillo', '0']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertVisible does not use raw fallback for iOS snapshot misses', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['id="chat"', '1000'], - invoke: async (req): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: snapshot([]) }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - assert.ok(snapshotFlags.length > 1); - assert.equal( - snapshotFlags.some((flags) => flags?.snapshotRaw === true), - false, - ); -}); - -test('invokeMaestroAssertVisible does not use raw fallback for Android identifiers', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['id="album-0"', '1000'], - invoke: async (req): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { - ok: true, - data: snapshot([node('Album item', { identifier: 'album-0' })]), - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.equal(snapshotFlags.length, 1); - assert.equal(snapshotFlags[0]?.snapshotRaw, undefined); -}); - -test('invokeMaestroAssertVisible retries Android id-only selectors with a raw snapshot after a presentation miss', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['id="material-top-bar-post-auth-screen"', '1000'], - invoke: async (req): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { - ok: true, - data: snapshot( - req.flags?.snapshotRaw === true - ? [ - node('', { - type: 'android.view.ViewGroup', - identifier: 'material-top-bar-post-auth-screen', - rect: { x: 0, y: 240, width: 1080, height: 1900 }, - }), - ] - : [], - ), - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - snapshotFlags.map((flags) => flags?.snapshotRaw), - [undefined, true], - ); -}); - -test('invokeMaestroAssertNotVisible does not use Android raw fallback for absent id-only selectors', async () => { - vi.spyOn(Date, 'now').mockReturnValue(0); - - const snapshotFlags: Array = []; - const response = await invokeMaestroAssertNotVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['id="archived-banner"', '0'], - invoke: async (req): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: snapshot([]) }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - snapshotFlags.map((flags) => flags?.snapshotRaw), - [undefined], - ); -}); - -test('invokeMaestroAssertVisible does not use Android raw fallback for generated text selectors', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['label="Chat" || text="Chat" || id="Chat"', '0'], - invoke: async (req): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: snapshot([]) }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - assert.equal( - snapshotFlags.some((flags) => flags?.snapshotRaw === true), - false, - ); -}); - -test('invokeMaestroAssertVisible bounds Android verification retries after native wait succeeds', async () => { - vi.useFakeTimers(); - - const calls: Array<[string, string[] | undefined]> = []; - const responsePromise = invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['label="Input" || text="Input" || id="Input"', '60000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'snapshot') { - return { ok: true, data: snapshot([node('Loading')]) }; - } - if (req.command === 'wait') return { ok: true, data: { matches: 1 } }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - await vi.advanceTimersByTimeAsync(6500); - const response = await responsePromise; - - assert.equal(response.ok, false); - assert.deepEqual(calls.slice(0, 3), [ - ['wait', ['Input', '60000']], - ['snapshot', []], - ['snapshot', []], - ]); - assert.ok(calls.filter(([command]) => command === 'snapshot').length < 40); -}); - -test('invokeMaestroAssertVisible writes terminal snapshot artifacts for failed attempts', async () => { - const artifactsDir = fs.mkdtempSync(path.join(os.tmpdir(), 'maestro-assert-artifacts-')); - try { - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android', artifactsDir }, - }, - positionals: ['id="album-0"', '0'], - invoke: async (): Promise => ({ - ok: true, - data: snapshot([ - node('Chat', { identifier: 'chat-tab', type: 'android.widget.Button' }), - node('Contacts', { identifier: 'contacts-tab', type: 'android.widget.Button' }), - ]), - }), - }); - - assert.equal(response.ok, false); - if (!response.ok) { - const artifactPaths = response.error.details?.artifactPaths; - assert.deepEqual(artifactPaths, [ - path.join(artifactsDir, 'failure-snapshot.json'), - path.join(artifactsDir, 'failure-snapshot.txt'), - ]); - } - assert.match( - fs.readFileSync(path.join(artifactsDir, 'failure-snapshot.txt'), 'utf8'), - /@e1 \[button\] "Chat"/, - ); - assert.match( - fs.readFileSync(path.join(artifactsDir, 'failure-snapshot.json'), 'utf8'), - /"identifier": "chat-tab"/, - ); - } finally { - fs.rmSync(artifactsDir, { recursive: true, force: true }); - } -}); - -test('invokeMaestroAssertVisible treats an elapsed ellipsis loading gate as already past loading', async () => { - vi.spyOn(Date, 'now').mockReturnValueOnce(0).mockReturnValueOnce(0).mockReturnValueOnce(250); - - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { - platform: 'ios', - maestro: { allowAlreadyPastLoading: true }, - }, - }, - positionals: ['label="Loading…" || text="Loading…" || id="Loading…"', '1000'], - invoke: async (req): Promise => { - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Loading…' }, - }; - } - return { - ok: true, - data: snapshot([node('Dashboard')]), - }; - }, - }); - - assert.equal(response.ok, true); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.alreadyPastLoading, true); - assert.equal(response.data.timeoutMs, 1000); - } -}); - -test('invokeMaestroAssertVisible reports React Native overlays during snapshot assertions', async () => { - const calls: Array<[string, string[] | undefined]> = []; - let snapshots = 0; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['label="Ready" || text="Ready" || id="Ready"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Ready' }, - }; - } - if (req.command === 'snapshot') { - snapshots += 1; - return { - ok: true, - data: snapshot( - snapshots === 1 - ? [ - node('Ready'), - node('Runtime Error', { - index: 2, - ref: 'e2', - rect: { x: 0, y: 0, width: 390, height: 80 }, - }), - node('Minimize', { - index: 3, - ref: 'e3', - type: 'Button', - rect: { x: 300, y: 20, width: 80, height: 44 }, - }), - ] - : [node('Ready')], - snapshots, - ), - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - if (!response.ok) { - assert.match(response.error.message, /React Native overlay is covering app content/); - } - assert.deepEqual(calls, [ - ['wait', ['Ready', '1000']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertVisible fails fast when a RedBox has no dismiss target', async () => { - const calls: Array<[string, string[] | undefined]> = []; - const response = await invokeMaestroAssertVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - positionals: ['label="Ready" || text="Ready" || id="Ready"', '1000'], - invoke: async (req): Promise => { - calls.push([req.command, req.positionals]); - if (req.command === 'wait') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out for text: Ready' }, - }; - } - if (req.command === 'snapshot') { - return { - ok: true, - data: snapshot([ - node("Uncaught (in promise): Error: Unable to download asset from url: 'x'", { - type: 'Other', - rect: { x: 0, y: 0, width: 390, height: 80 }, - }), - ]), - }; - } - if (req.command === 'react-native') { - return { - ok: false, - error: { - code: 'COMMAND_FAILED', - message: 'React Native overlay detected, but no safe dismiss target was found', - }, - }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - assert.equal(response.ok, false); - if (!response.ok) { - assert.match(response.error.message, /React Native overlay is covering app content/); - } - assert.deepEqual(calls, [ - ['wait', ['Ready', '1000']], - ['snapshot', []], - ]); -}); - -test('invokeMaestroAssertNotVisible passes after a slow hidden sample exhausts the timeout', async () => { - vi.spyOn(Date, 'now').mockReturnValueOnce(0).mockReturnValueOnce(0).mockReturnValueOnce(3500); - - const calls: DaemonRequest[] = []; - const response = await invokeMaestroAssertNotVisible({ - baseReq: { - token: 't', - session: 's', - flags: {}, - }, - positionals: ['id="tab-4"'], - invoke: async (req): Promise => { - calls.push(req); - return { - ok: true, - data: { - createdAt: 1, - nodes: [], - }, - }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - calls.map((call) => [call.command, call.positionals]), - [['snapshot', []]], - ); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.stableSamples, 1); - assert.equal(response.data.waitedMs, 3500); - } -}); - -test('invokeMaestroAssertNotVisible ignores matched nodes without visible rects', async () => { - vi.spyOn(Date, 'now').mockReturnValueOnce(0).mockReturnValueOnce(0).mockReturnValueOnce(3500); - - const response = await invokeMaestroAssertNotVisible({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - positionals: ['label="📌" || text="📌" || id="📌"'], - invoke: async (): Promise => ({ - ok: true, - data: { - createdAt: 1, - nodes: [node('📌', { value: '📌', enabled: true, depth: 21, rect: undefined })], - }, - }), - }); - - assert.equal(response.ok, true); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.stableSamples, 1); - } -}); - -function snapshot(nodes: SnapshotState['nodes'], createdAt = 1): SnapshotState { - return { createdAt, nodes }; -} - -function node( - label: string, - overrides: Partial = {}, -): SnapshotState['nodes'][number] { - return { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label, - rect: { x: 20, y: 80, width: 120, height: 40 }, - depth: 8, - ...overrides, - }; -} - -test('invokeMaestroAssertNotVisible accepts timeout overrides for short extended waits', async () => { - vi.spyOn(Date, 'now').mockReturnValueOnce(0).mockReturnValueOnce(0).mockReturnValueOnce(300); - - const response = await invokeMaestroAssertNotVisible({ - baseReq: { - token: 't', - session: 's', - flags: {}, - }, - positionals: ['id="toast"', '1'], - invoke: async (): Promise => ({ - ok: true, - data: { - createdAt: 1, - nodes: [], - }, - }), - }); - - assert.equal(response.ok, true); - if (response.ok) { - assert.ok(response.data); - assert.equal(response.data.stableSamples, 1); - assert.equal(response.data.timeoutMs, 1); - } -}); diff --git a/src/compat/maestro/__tests__/runtime-flow.test.ts b/src/compat/maestro/__tests__/runtime-flow.test.ts deleted file mode 100644 index fdf195b6e..000000000 --- a/src/compat/maestro/__tests__/runtime-flow.test.ts +++ /dev/null @@ -1,170 +0,0 @@ -import assert from 'node:assert/strict'; -import { test } from 'vitest'; -import type { DaemonRequest, DaemonResponse, SessionAction } from '../../../daemon/types.ts'; -import { invokeMaestroRunFlowWhenControl } from '../runtime-flow.ts'; - -test('invokeMaestroRunFlowWhenControl waits briefly for visible conditions', async () => { - let snapshots = 0; - const invokedActions: SessionAction[] = []; - const actions: SessionAction[] = [ - { ts: Date.now(), command: 'click', positionals: ['label="Dismiss"'], flags: {} }, - ]; - - const response = await invokeMaestroRunFlowWhenControl({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - control: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: 'label="Dismiss" || text="Dismiss" || id="Dismiss"', - actions, - }, - line: 12, - step: 4, - invoke: async (req: DaemonRequest): Promise => { - assert.equal(req.command, 'snapshot'); - snapshots += 1; - return { - ok: true, - data: { - createdAt: Date.now(), - nodes: - snapshots === 1 - ? [] - : [ - { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label: 'Dismiss', - rect: { x: 201, y: 2180, width: 138, height: 55 }, - depth: 20, - }, - ], - }, - }; - }, - invokeReplayAction: async ({ action }): Promise => { - invokedActions.push(action); - return { ok: true, data: { clicked: true } }; - }, - }); - - assert.equal(response.ok, true); - assert.equal(snapshots, 2); - assert.deepEqual( - invokedActions.map((action) => [action.command, action.positionals]), - [['click', ['label="Dismiss"']]], - ); - if (response.ok) { - assert.equal(response.data?.ran, 1); - } -}); - -test('invokeMaestroRunFlowWhenControl uses regular iOS snapshots for visible conditions', async () => { - const snapshotFlags: Array = []; - const invokedActions: SessionAction[] = []; - const actions: SessionAction[] = [ - { ts: Date.now(), command: 'click', positionals: ['label="Continue"'], flags: {} }, - ]; - - const response = await invokeMaestroRunFlowWhenControl({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'ios' }, - }, - control: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: 'label="Continue" || text="Continue" || id="Continue"', - actions, - }, - line: 12, - step: 4, - invoke: async (req: DaemonRequest): Promise => { - assert.equal(req.command, 'snapshot'); - snapshotFlags.push(req.flags); - return { - ok: true, - data: { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'Button', - label: 'Continue', - rect: { x: 100, y: 420, width: 120, height: 44 }, - depth: 4, - }, - ], - }, - }; - }, - invokeReplayAction: async ({ action }): Promise => { - invokedActions.push(action); - return { ok: true, data: { clicked: true } }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - snapshotFlags.map((flags) => flags?.snapshotRaw), - [undefined], - ); - assert.deepEqual( - invokedActions.map((action) => [action.command, action.positionals]), - [['click', ['label="Continue"']]], - ); -}); - -test('invokeMaestroRunFlowWhenControl keeps notVisible conditions immediate', async () => { - let snapshots = 0; - const response = await invokeMaestroRunFlowWhenControl({ - baseReq: { - token: 't', - session: 's', - flags: { platform: 'android' }, - }, - control: { - kind: 'maestroRunFlowWhen', - mode: 'notVisible', - selector: 'label="Loading" || text="Loading" || id="Loading"', - actions: [{ ts: Date.now(), command: 'click', positionals: ['label="Continue"'], flags: {} }], - }, - line: 14, - step: 7, - invoke: async (): Promise => { - snapshots += 1; - return { - ok: true, - data: { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label: 'Loading', - rect: { x: 120, y: 420, width: 160, height: 48 }, - depth: 8, - }, - ], - }, - }; - }, - invokeReplayAction: async (): Promise => { - throw new Error('notVisible should skip while the selector is visible'); - }, - }); - - assert.equal(response.ok, true); - assert.equal(snapshots, 1); - if (response.ok) { - assert.equal(response.data?.skipped, true); - } -}); diff --git a/src/compat/maestro/__tests__/runtime-geometry.test.ts b/src/compat/maestro/__tests__/runtime-geometry.test.ts deleted file mode 100644 index fbc35f042..000000000 --- a/src/compat/maestro/__tests__/runtime-geometry.test.ts +++ /dev/null @@ -1,80 +0,0 @@ -import { expect, test } from 'vitest'; -import { pointForMaestroTapOnTarget, swipeCoordinatesFromTarget } from '../runtime-geometry.ts'; - -test('pointForMaestroTapOnTarget centers resolved broad text containers', () => { - const point = pointForMaestroTapOnTarget({ - node: { - index: 5, - ref: 'e5', - type: 'scroll-area', - label: 'Article', - rect: { x: 0, y: 117, width: 402, height: 180 }, - }, - rect: { x: 0, y: 117, width: 402, height: 180 }, - frame: { referenceWidth: 402, referenceHeight: 874 }, - }); - - expect(point).toEqual({ x: 201, y: 207 }); -}); - -test('pointForMaestroTapOnTarget centers tall Android bottom-tab containers', () => { - const point = pointForMaestroTapOnTarget({ - node: { - index: 40, - ref: 'e41', - type: 'android.widget.FrameLayout', - label: 'Albums', - rect: { x: 540, y: 2054, width: 270, height: 220 }, - }, - rect: { x: 540, y: 2054, width: 270, height: 220 }, - frame: { referenceWidth: 1080, referenceHeight: 2340 }, - }); - - expect(point).toEqual({ x: 675, y: 2164 }); -}); - -test('swipeCoordinatesFromTarget preserves Maestro target-relative swipe distance', () => { - const swipe = swipeCoordinatesFromTarget( - { - node: { - index: 12, - ref: 'e12', - type: 'Cell', - label: 'Card', - rect: { x: 100, y: 200, width: 100, height: 80 }, - }, - rect: { x: 100, y: 200, width: 100, height: 80 }, - frame: { referenceWidth: 402, referenceHeight: 874 }, - }, - 'right', - ); - - expect(swipe).toEqual({ - ok: true, - start: { x: 150, y: 240 }, - end: { x: 300, y: 240 }, - }); -}); - -test('swipeCoordinatesFromTarget clamps swipe endpoints to the viewport margin', () => { - const swipe = swipeCoordinatesFromTarget( - { - node: { - index: 12, - ref: 'e12', - type: 'Cell', - label: 'Card', - rect: { x: 340, y: 200, width: 100, height: 80 }, - }, - rect: { x: 340, y: 200, width: 100, height: 80 }, - frame: { referenceWidth: 402, referenceHeight: 874 }, - }, - 'right', - ); - - expect(swipe).toEqual({ - ok: true, - start: { x: 390, y: 240 }, - end: { x: 394, y: 240 }, - }); -}); diff --git a/src/compat/maestro/__tests__/runtime-interactions.test.ts b/src/compat/maestro/__tests__/runtime-interactions.test.ts deleted file mode 100644 index 8179774b3..000000000 --- a/src/compat/maestro/__tests__/runtime-interactions.test.ts +++ /dev/null @@ -1,866 +0,0 @@ -import { expect, test, vi } from 'vitest'; -import type { DaemonRequest, DaemonResponse } from '../../../daemon/types.ts'; -import type { SnapshotState } from '../../../kernel/snapshot.ts'; -import { - invokeMaestroSwipeScreen, - invokeMaestroSwipeOn, - invokeMaestroTapOn, - invokeMaestroTapPointPercent, -} from '../runtime-interactions.ts'; -import { - captureMaestroSnapshot, - consumeMaestroRecoverableInteraction, -} from '../runtime-support.ts'; - -test('invokeMaestroTapOn resolves mutating taps from the current snapshot', async () => { - const selector = - 'label="Article by Gandalf" || text="Article by Gandalf" || id="Article by Gandalf"'; - - const { response, clicks, clickFlags, snapshots } = await runTapOn(selector, () => - currentBreadcrumbSnapshot(), - ); - - expect(response.ok).toBe(true); - expect(snapshots).toBe(1); - expect(clicks).toEqual([['86', '89']]); - expect(clickFlags[0]?.postGestureStabilization).toBe(true); - expect(clickFlags[0]?.interactionOutcome).toBeUndefined(); -}); - -test('invokeMaestroTapOn uses raw snapshots for target resolution', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroTapOn({ - baseReq: { - token: 'test', - session: 'article', - flags: { platform: 'ios' }, - }, - positionals: [ - 'label="Article by Gandalf" || text="Article by Gandalf" || id="Article by Gandalf"', - ], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: currentBreadcrumbSnapshot() }; - } - if (req.command === 'click') return { ok: true, data: {} }; - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(snapshotFlags).toHaveLength(1); - expect(snapshotFlags[0]?.noRecord).toBe(true); - expect(snapshotFlags[0]?.snapshotInteractiveOnly).toBeUndefined(); - expect(snapshotFlags[0]?.snapshotRaw).toBe(true); - expect(snapshotFlags[0]?.snapshotForceFull).toBeUndefined(); -}); - -test('invokeMaestroTapOn resolves drawer targets from raw snapshots', async () => { - const snapshotFlags: Array = []; - const clicks: string[][] = []; - const response = await invokeMaestroTapOn({ - baseReq: { - token: 'test', - session: 'drawer', - flags: { platform: 'ios' }, - }, - positionals: ['label="Feed" || text="Feed" || id="Feed"'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { - ok: true, - data: req.flags?.snapshotRaw === true ? drawerRawSnapshot() : pagesOnlySnapshot(), - }; - } - if (req.command === 'click') { - clicks.push(req.positionals ?? []); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(snapshotFlags.map((flags) => flags?.snapshotRaw)).toEqual([true]); - expect(clicks).toEqual([['201', '202']]); -}); - -test('invokeMaestroTapOn retries raw target snapshots without interactive fallback', async () => { - vi.useFakeTimers(); - const snapshotFlags: Array = []; - const responsePromise = invokeMaestroTapOn({ - baseReq: { - token: 'test', - session: 'bottom-tabs', - flags: { platform: 'ios' }, - }, - positionals: ['id="article"'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: truncatedContentSnapshot() }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - try { - await vi.advanceTimersByTimeAsync(30_000); - const response = await responsePromise; - - expect(response.ok).toBe(false); - expect(snapshotFlags.length).toBeGreaterThan(1); - expect(snapshotFlags.some((flags) => flags?.snapshotInteractiveOnly === true)).toBe(false); - expect(snapshotFlags.every((flags) => flags?.snapshotRaw === true)).toBe(true); - } finally { - vi.useRealTimers(); - } -}); - -test('invokeMaestroSwipeOn does not use interactive fallback for truncated regular snapshot misses', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroSwipeOn({ - baseReq: { - token: 'test', - session: 'bottom-tabs', - flags: { platform: 'ios' }, - }, - positionals: ['id="article"', 'left', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: truncatedContentSnapshot() }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(false); - expect(snapshotFlags).toHaveLength(1); - expect(snapshotFlags[0]?.snapshotInteractiveOnly).toBeUndefined(); -}); - -test('invokeMaestroSwipeOn does not use interactive fallback for complete regular snapshot misses', async () => { - const snapshotFlags: Array = []; - const response = await invokeMaestroSwipeOn({ - baseReq: { - token: 'test', - session: 'complete-miss', - flags: { platform: 'ios' }, - }, - positionals: ['id="article"', 'left', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { ok: true, data: { ...truncatedContentSnapshot(), truncated: false } }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(false); - expect(snapshotFlags).toHaveLength(1); - expect(snapshotFlags[0]?.snapshotInteractiveOnly).toBeUndefined(); -}); - -test('invokeMaestroTapOn resolves visible Android non-interactive text from a regular snapshot', async () => { - const snapshotFlags: Array = []; - const clicks: string[][] = []; - const response = await invokeMaestroTapOn({ - baseReq: { - token: 'test', - session: 'android-header', - flags: { platform: 'android' }, - }, - positionals: ['label="Albums" || text="Albums" || id="Albums"'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { - ok: true, - data: { - nodes: [ - appNode(), - windowNode(), - { - index: 56, - ref: 'e56', - type: 'android.view.View', - label: 'Albums', - value: 'Albums', - depth: 20, - parentIndex: 1, - rect: { x: 154, y: 194, width: 188, height: 74 }, - visibleToUser: true, - enabled: true, - }, - ], - }, - }; - } - if (req.command === 'click') { - clicks.push(req.positionals ?? []); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(snapshotFlags).toHaveLength(1); - expect(snapshotFlags[0]?.snapshotInteractiveOnly).toBeUndefined(); - expect(snapshotFlags[0]?.snapshotRaw).toBe(true); - expect(clicks).toEqual([['248', '231']]); -}); - -test('invokeMaestroTapOn taps resolved iOS buttons by coordinates', async () => { - const { response, clicks } = await runTapOn( - 'label="Pop to top" || text="Pop to top" || id="Pop to top"', - () => buttonSnapshot('Pop to top'), - ); - - expect(response.ok).toBe(true); - expect(clicks).toEqual([['201', '149']]); -}); - -test('invokeMaestroTapOn clicks normal Close/Dismiss buttons when no React Native overlay is present', async () => { - const { response, commands } = await runTapOn( - 'label="Dismiss" || text="Dismiss" || id="Dismiss"', - () => buttonSnapshot('Dismiss'), - ); - - expect(response.ok).toBe(true); - expect(commands).toEqual(['snapshot', 'click']); -}); - -test('invokeMaestroTapOn clicks explicit React Native overlay controls directly', async () => { - const { response, commands, clicks } = await runTapOn( - 'label="Dismiss" || text="Dismiss" || id="Dismiss"', - () => overlayDismissButtonSnapshot(), - ); - - expect(response.ok).toBe(true); - expect(commands).toEqual(['snapshot', 'click']); - expect(clicks).toEqual([['355', '30']]); -}); - -test('invokeMaestroSwipeScreen delegates directional swipes to core gestures', async () => { - const scope = { values: {} }; - const gestures: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'pager', - flags: { platform: 'android' }, - }, - scope, - positionals: ['direction', 'left', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'gesture') { - gestures.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(gestures).toEqual([{ kind: 'swipe', preset: 'left', durationMs: 300 }]); - expect(consumeMaestroRecoverableInteraction(scope)).toEqual({ - kind: 'swipe', - command: 'gesture', - input: { kind: 'swipe', preset: 'left', durationMs: 300 }, - }); -}); - -test('invokeMaestroSwipeScreen delegates mirrored Android directional swipes to core gestures', async () => { - const gestures: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'pager', - flags: { platform: 'android' }, - }, - positionals: ['direction', 'right', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'gesture') { - gestures.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(gestures).toEqual([{ kind: 'swipe', preset: 'right', durationMs: 300 }]); -}); - -test('invokeMaestroSwipeScreen delegates Android vertical down swipes to core gestures', async () => { - const scope = { values: {} }; - const gestures: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'pager', - flags: { platform: 'android' }, - }, - scope, - positionals: ['direction', 'down', '100'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'gesture') { - gestures.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(gestures).toEqual([{ kind: 'swipe', preset: 'down', durationMs: 100 }]); - expect(consumeMaestroRecoverableInteraction(scope)).toEqual({ - kind: 'swipe', - command: 'gesture', - input: { kind: 'swipe', preset: 'down', durationMs: 100 }, - }); -}); - -test('invokeMaestroSwipeScreen delegates Android vertical up swipes to core gestures', async () => { - const gestures: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'pager', - flags: { platform: 'android' }, - }, - positionals: ['direction', 'up'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'gesture') { - gestures.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(gestures).toEqual([{ kind: 'swipe', preset: 'up' }]); -}); - -test('invokeMaestroSwipeOn resolves visible non-interactive text from a regular snapshot', async () => { - const snapshotFlags: Array = []; - const swipes: Array = []; - const swipeFlags: Array = []; - const response = await invokeMaestroSwipeOn({ - baseReq: { - token: 'test', - session: 'android-carousel', - flags: { platform: 'android', postGestureStabilization: true }, - }, - positionals: ['label="Gallery" || text="Gallery" || id="Gallery"', 'left', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshotFlags.push(req.flags); - return { - ok: true, - data: { - nodes: [ - appNode(), - windowNode(), - { - index: 4, - ref: 'e4', - type: 'android.view.View', - label: 'Gallery', - value: 'Gallery', - depth: 2, - parentIndex: 1, - rect: { x: 100, y: 200, width: 200, height: 100 }, - visibleToUser: true, - enabled: true, - }, - ], - }, - }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - swipeFlags.push(req.flags); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(snapshotFlags).toHaveLength(1); - expect(snapshotFlags[0]?.snapshotInteractiveOnly).toBeUndefined(); - expect(snapshotFlags[0]?.snapshotRaw).toBe(true); - expect(swipes).toEqual([{ from: { x: 200, y: 250 }, to: { x: 8, y: 250 }, durationMs: 300 }]); - expect(swipeFlags[0]?.postGestureStabilization).toBe(true); -}); - -test('invokeMaestroSwipeScreen preserves vertical percentage endpoints', async () => { - const swipes: Array = []; - const swipeFlags: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'article', - flags: { platform: 'ios' }, - }, - positionals: ['percent', '50', '75', '50', '35', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - return { ok: true, data: fullScreenSnapshot(400, 800) }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - swipeFlags.push(req.flags); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(swipes).toEqual([{ from: { x: 200, y: 600 }, to: { x: 200, y: 280 }, durationMs: 300 }]); - expect(swipeFlags[0]?.postGestureStabilization).toBeUndefined(); -}); - -test('invokeMaestroSwipeScreen preserves iOS horizontal percentage endpoints', async () => { - const swipes: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'ios-pager', - flags: { platform: 'ios' }, - }, - positionals: ['percent', '10', '50', '90', '50', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - return { ok: true, data: fullScreenSnapshot(400, 800) }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(swipes).toEqual([{ from: { x: 40, y: 400 }, to: { x: 360, y: 400 }, durationMs: 300 }]); -}); - -test('invokeMaestroSwipeScreen preserves Android midpoint percentage endpoints', async () => { - const swipes: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'pager', - flags: { platform: 'android' }, - }, - positionals: ['percent', '90', '50', '10', '50', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - return { ok: true, data: fullScreenSnapshot(390, 600) }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(swipes).toEqual([{ from: { x: 351, y: 300 }, to: { x: 39, y: 300 }, durationMs: 300 }]); -}); - -test('invokeMaestroSwipeScreen refreshes a cached frame before resolving percentages', async () => { - const scope = { values: {} }; - const swipes: Array = []; - let snapshots = 0; - const baseReq = { - token: 'test', - session: 'rotated-pager', - flags: { platform: 'android' as const }, - }; - const invoke = async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - snapshots += 1; - return { - ok: true, - data: snapshots === 1 ? fullScreenSnapshot(400, 800) : fullScreenSnapshot(800, 400), - }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }; - - await captureMaestroSnapshot({ baseReq, invoke, scope }); - const response = await invokeMaestroSwipeScreen({ - baseReq, - positionals: ['percent', '90', '50', '10', '50', '300'], - invoke, - scope, - }); - - expect(response.ok).toBe(true); - expect(snapshots).toBe(2); - expect(swipes).toEqual([{ from: { x: 720, y: 200 }, to: { x: 80, y: 200 }, durationMs: 300 }]); -}); - -test('invokeMaestroSwipeScreen preserves an explicit Android horizontal percentage lane', async () => { - const swipes: Array = []; - const response = await invokeMaestroSwipeScreen({ - baseReq: { - token: 'test', - session: 'nested-pager', - flags: { platform: 'android' }, - }, - positionals: ['percent', '90', '30', '10', '30', '300'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - return { ok: true, data: fullScreenSnapshot(390, 600) }; - } - if (req.command === 'swipe') { - swipes.push(req.input); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(swipes).toEqual([{ from: { x: 351, y: 180 }, to: { x: 39, y: 180 }, durationMs: 300 }]); -}); - -test('invokeMaestroTapPointPercent bounds percentage points to valid frame pixels', async () => { - const clicks: string[][] = []; - const clickFlags: Array = []; - const response = await invokeMaestroTapPointPercent({ - baseReq: { - token: 'test', - session: 'article', - flags: { platform: 'ios' }, - }, - positionals: ['125', '-10'], - invoke: async (req: DaemonRequest): Promise => { - if (req.command === 'snapshot') { - return { ok: true, data: fullScreenSnapshot(400, 800) }; - } - if (req.command === 'click') { - clicks.push(req.positionals ?? []); - clickFlags.push(req.flags); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - - expect(response.ok).toBe(true); - expect(clicks).toEqual([['399', '0']]); - expect(clickFlags[0]?.interactionOutcome).toBeUndefined(); - expect(clickFlags[0]?.postGestureStabilization).toBe(true); -}); - -function currentBreadcrumbSnapshot(): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - appNode(), - windowNode(), - { - index: 2, - ref: 'e3', - type: 'ScrollView', - label: 'Article by Gandalf', - depth: 4, - parentIndex: 1, - rect: { x: 0, y: 58.33333333333333, width: 402, height: 58.33333333333333 }, - }, - { - index: 3, - ref: 'e4', - type: 'Cell', - label: 'Article by Gandalf', - depth: 5, - parentIndex: 2, - rect: { x: 8, y: 65.33333587646484, width: 155, height: 48 }, - }, - ], - }; -} - -function pagesOnlySnapshot(): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - appNode(), - windowNode(), - { - index: 2, - ref: 'e3', - type: 'StaticText', - label: 'Pages', - depth: 4, - parentIndex: 1, - rect: { - x: 176.6666717529297, - y: 75.66666603088379, - width: 48.666656494140625, - height: 20.333335876464844, - }, - }, - ], - }; -} - -function drawerRawSnapshot(): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - appNode(), - windowNode(), - { - index: 2, - ref: 'e3', - type: 'StaticText', - label: 'Pages', - depth: 4, - parentIndex: 1, - rect: { - x: 176.6666717529297, - y: 75.66666603088379, - width: 48.666656494140625, - height: 20.333335876464844, - }, - }, - { - index: 3, - ref: 'e4', - type: 'ScrollView', - label: 'Article', - depth: 4, - parentIndex: 1, - rect: { x: 0, y: 116.66666412353516, width: 402, height: 757.3333333333333 }, - }, - { - index: 4, - ref: 'e5', - type: 'Button', - label: 'Article', - depth: 5, - parentIndex: 3, - rect: { x: 12, y: 120.66666412353516, width: 378, height: 54.00000762939453 }, - }, - { - index: 5, - ref: 'e6', - type: 'Button', - label: 'Feed', - depth: 5, - parentIndex: 3, - rect: { x: 12, y: 174.66666412353516, width: 378, height: 54 }, - }, - { - index: 6, - ref: 'e7', - type: 'Button', - label: 'Albums', - depth: 5, - parentIndex: 3, - rect: { x: 12, y: 228.66666412353516, width: 378, height: 53.99998474121094 }, - }, - { - index: 7, - ref: 'e8', - type: 'StaticText', - label: 'Feed', - depth: 4, - parentIndex: 1, - rect: { - x: 181.3333282470703, - y: 75.66666603088379, - width: 39.333343505859375, - height: 20.333335876464844, - }, - }, - ], - }; -} - -async function runTapOn( - selector: string, - readSnapshot: (snapshotIndex: number) => SnapshotState, -): Promise<{ - response: DaemonResponse; - commands: string[]; - clicks: string[][]; - clickFlags: Array; - snapshots: number; -}> { - const commands: string[] = []; - const clicks: string[][] = []; - const clickFlags: Array = []; - let snapshots = 0; - const response = await invokeMaestroTapOn({ - baseReq: { - token: 'test', - session: 'nav', - flags: { platform: 'ios' }, - }, - positionals: [selector], - invoke: async (req: DaemonRequest): Promise => { - commands.push(req.command); - if (req.command === 'snapshot') { - snapshots += 1; - return { ok: true, data: readSnapshot(snapshots) }; - } - if (req.command === 'click') { - clicks.push(req.positionals ?? []); - clickFlags.push(req.flags); - return { ok: true, data: {} }; - } - return { ok: false, error: { code: 'UNEXPECTED_COMMAND', message: req.command } }; - }, - }); - return { response, commands, clicks, clickFlags, snapshots }; -} - -function fullScreenSnapshot(width: number, height: number): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - { - index: 0, - ref: 'e1', - type: 'Application', - label: 'Android Test App', - depth: 0, - rect: { x: 0, y: 0, width, height }, - }, - { - index: 1, - ref: 'e2', - type: 'Window', - depth: 1, - parentIndex: 0, - rect: { x: 0, y: 0, width, height }, - }, - ], - }; -} - -function buttonSnapshot(label: string): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - appNode(), - windowNode(), - { - index: 2, - ref: 'e3', - type: 'Button', - label, - depth: 4, - parentIndex: 1, - rect: { x: 142, y: 128.66666412353516, width: 118, height: 40 }, - }, - ], - }; -} - -function overlayDismissButtonSnapshot(): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - appNode(), - windowNode(), - { - index: 10, - ref: 'e10', - type: 'StaticText', - label: 'Runtime Error', - depth: 2, - parentIndex: 1, - rect: { x: 0, y: 0, width: 402, height: 40 }, - }, - { - index: 11, - ref: 'e11', - type: 'Button', - label: 'Dismiss', - depth: 2, - parentIndex: 1, - rect: { x: 320, y: 12, width: 70, height: 36 }, - }, - { - index: 12, - ref: 'e12', - type: 'StaticText', - label: 'Call Stack', - depth: 2, - parentIndex: 1, - rect: { x: 0, y: 52, width: 402, height: 40 }, - }, - ], - }; -} - -function truncatedContentSnapshot(): SnapshotState { - return { - createdAt: Date.now(), - truncated: true, - nodes: [ - appNode(), - windowNode(), - { - index: 2, - ref: 'e3', - type: 'ScrollView', - label: 'Contacts', - depth: 2, - parentIndex: 1, - rect: { x: 0, y: 92, width: 402, height: 699 }, - }, - { - index: 3, - ref: 'e4', - type: 'StaticText', - label: 'Marissa Castillo', - depth: 3, - parentIndex: 2, - rect: { x: 16, y: 128, width: 160, height: 22 }, - }, - ], - }; -} - -function appNode(): SnapshotState['nodes'][number] { - return { - index: 0, - ref: 'e1', - type: 'Application', - label: 'React Navigation Example', - depth: 0, - rect: { x: 0, y: 0, width: 402, height: 874 }, - }; -} - -function windowNode(): SnapshotState['nodes'][number] { - return { - index: 1, - ref: 'e2', - type: 'Window', - depth: 1, - parentIndex: 0, - rect: { x: 0, y: 0, width: 402, height: 874 }, - }; -} diff --git a/src/compat/maestro/__tests__/runtime-port-fixtures.ts b/src/compat/maestro/__tests__/runtime-port-fixtures.ts new file mode 100644 index 000000000..37f192e56 --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-port-fixtures.ts @@ -0,0 +1,68 @@ +import type { + MaestroRuntimeOperationContext, + MaestroRuntimeOperations, +} from '../runtime-port-types.ts'; + +export type RecordedCall = { + kind: string; + input: unknown; + generation: number; + appId?: string; +}; + +export function makeOperations( + overrides: Partial = {}, +): MaestroRuntimeOperations { + const noOp = async (): Promise => undefined; + return { + resolveTarget: async ({ selector }, context) => ({ + generation: context.generation, + matched: true, + visible: true, + candidateCount: 1, + rect: { x: 100, y: 200, width: 100, height: 80 }, + viewport: { x: 0, y: 0, width: 402, height: 874 }, + ref: selector.id ? 'e1' : undefined, + }), + observe: async (_input, context) => ({ + generation: context.generation, + matched: true, + visible: true, + candidateCount: 1, + }), + resolveGestureViewport: async () => ({ x: 0, y: 0, width: 402, height: 874 }), + launchApp: noOp, + stopApp: noOp, + openLink: noOp, + tapOn: noOp, + doubleTapOn: noOp, + longPressOn: noOp, + gesture: noOp, + inputText: noOp, + eraseText: noOp, + pasteText: noOp, + scroll: noOp, + scrollUntilVisible: noOp, + pressKey: noOp, + back: noOp, + hideKeyboard: noOp, + waitForAnimationToEnd: noOp, + takeScreenshot: noOp, + runScript: noOp, + ...overrides, + }; +} + +export function record( + calls: RecordedCall[], + kind: string, + input: unknown, + context: MaestroRuntimeOperationContext, +): void { + calls.push({ + kind, + input, + generation: context.generation, + ...(context.appId === undefined ? {} : { appId: context.appId }), + }); +} diff --git a/src/compat/maestro/__tests__/runtime-port-gestures.test.ts b/src/compat/maestro/__tests__/runtime-port-gestures.test.ts new file mode 100644 index 000000000..e28faeb64 --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-port-gestures.test.ts @@ -0,0 +1,148 @@ +import { expect, test, vi } from 'vitest'; +import { createMaestroRuntimePort } from '../runtime-port.ts'; +import { makeOperations } from './runtime-port-fixtures.ts'; + +test('uses the structured gesture contract without observing absolute swipes', async () => { + const resolveGestureViewport = vi.fn(async () => ({ x: 10, y: 20, width: 400, height: 800 })); + const gesture = vi.fn(async () => undefined); + const operations = makeOperations({ resolveGestureViewport, gesture }); + const port = createMaestroRuntimePort(operations); + + await port.execute({ + command: { + kind: 'swipe', + source: { line: 2 }, + gesture: { + kind: 'coordinates', + start: { space: 'absolute', x: 100, y: 200 }, + end: { space: 'absolute', x: 300, y: 200 }, + duration: 240, + }, + }, + generation: 0, + env: {}, + }); + await port.execute({ + command: { + kind: 'swipe', + source: { line: 3 }, + gesture: { + kind: 'coordinates', + start: { space: 'percent', x: 90, y: 50 }, + end: { space: 'percent', x: 10, y: 50 }, + }, + }, + generation: 1, + env: {}, + }); + await port.execute({ + command: { + kind: 'swipe', + source: { line: 4 }, + gesture: { kind: 'screen', direction: 'down', duration: 300 }, + }, + generation: 2, + env: {}, + }); + await port.execute({ + command: { + kind: 'swipe', + source: { line: 5 }, + gesture: { kind: 'screen', direction: 'left' }, + }, + generation: 3, + env: {}, + }); + + expect(resolveGestureViewport).toHaveBeenCalledTimes(2); + expect(gesture).toHaveBeenNthCalledWith( + 1, + { + intent: 'pan', + origin: { x: 100, y: 200 }, + delta: { x: 200, y: 0 }, + durationMs: 240, + executionProfile: 'endpoint-hold', + }, + expect.objectContaining({ + generation: 0, + authoredSwipe: { + kind: 'coordinates', + start: { space: 'absolute', x: 100, y: 200 }, + end: { space: 'absolute', x: 300, y: 200 }, + duration: 240, + }, + }), + ); + expect(gesture).toHaveBeenNthCalledWith( + 2, + { + intent: 'pan', + origin: { x: 370, y: 420 }, + delta: { x: -320, y: 0 }, + durationMs: 400, + executionProfile: 'endpoint-hold', + }, + expect.objectContaining({ + generation: 1, + authoredSwipe: expect.objectContaining({ kind: 'coordinates' }), + gestureViewport: { x: 10, y: 20, width: 400, height: 800 }, + }), + ); + expect(gesture).toHaveBeenNthCalledWith( + 3, + { + intent: 'pan', + origin: { x: 210, y: 140 }, + delta: { x: 0, y: 560 }, + durationMs: 300, + executionProfile: 'endpoint-hold', + }, + expect.objectContaining({ + generation: 2, + authoredSwipe: { kind: 'screen', direction: 'down', duration: 300 }, + gestureViewport: { x: 10, y: 20, width: 400, height: 800 }, + }), + ); + expect(gesture).toHaveBeenNthCalledWith( + 4, + { + intent: 'pan', + preset: 'left', + durationMs: 400, + executionProfile: 'endpoint-hold', + }, + expect.objectContaining({ + generation: 3, + authoredSwipe: { kind: 'screen', direction: 'left' }, + }), + ); +}); + +test('rejects stale typed selector evidence before input execution', async () => { + const tapOn = vi.fn(async () => undefined); + const operations = makeOperations({ + resolveTarget: vi.fn(async () => ({ + generation: 9, + matched: true, + visible: true, + candidateCount: 1, + rect: { x: 0, y: 0, width: 10, height: 10 }, + })), + tapOn, + }); + const port = createMaestroRuntimePort(operations); + + await expect( + port.execute({ + command: { + kind: 'tapOn', + source: { line: 2 }, + target: { space: 'target', selector: { text: 'Continue' } }, + }, + generation: 0, + env: {}, + }), + ).rejects.toThrow(/evidence generation 9 does not match 0/); + expect(tapOn).not.toHaveBeenCalled(); +}); diff --git a/src/compat/maestro/__tests__/runtime-port.test.ts b/src/compat/maestro/__tests__/runtime-port.test.ts new file mode 100644 index 000000000..08906ee48 --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-port.test.ts @@ -0,0 +1,290 @@ +import { describe, expect, test, vi } from 'vitest'; +import { executeMaestroProgram } from '../engine.ts'; +import { parseMaestroProgram } from '../program-ir-parser.ts'; +import { createMaestroRuntimePort } from '../runtime-port.ts'; +import type { + MaestroRuntimeOperationContext, + MaestroRuntimeOperations, + MaestroTargetMatch, +} from '../runtime-port-types.ts'; +import { makeOperations, record, type RecordedCall } from './runtime-port-fixtures.ts'; + +describe('MaestroRuntimePort', () => { + test('delegates typed lifecycle, input, keyboard, screenshot, and script operations', async () => { + const calls: RecordedCall[] = []; + const operations = makeOperations({ + launchApp: vi.fn(async (input, context) => record(calls, 'launchApp', input, context)), + openLink: vi.fn(async (input, context) => record(calls, 'openLink', input, context)), + inputText: vi.fn(async (input, context) => record(calls, 'inputText', input, context)), + eraseText: vi.fn(async (input, context) => record(calls, 'eraseText', input, context)), + pasteText: vi.fn(async (input, context) => record(calls, 'pasteText', input, context)), + scroll: vi.fn(async (input, context) => record(calls, 'scroll', input, context)), + scrollUntilVisible: vi.fn(async (input, context) => + record(calls, 'scrollUntilVisible', input, context), + ), + hideKeyboard: vi.fn(async (input, context) => record(calls, 'hideKeyboard', input, context)), + pressKey: vi.fn(async (input, context) => record(calls, 'pressKey', input, context)), + back: vi.fn(async (input, context) => record(calls, 'back', input, context)), + waitForAnimationToEnd: vi.fn(async (input, context) => + record(calls, 'waitForAnimationToEnd', input, context), + ), + takeScreenshot: vi.fn(async (input, context) => { + record(calls, 'takeScreenshot', input, context); + return { artifactPaths: ['artifact://checkout.png'] }; + }), + runScript: vi.fn(async (input, context) => { + record(calls, 'runScript', input, context); + return { outputEnv: { TOKEN: 'generated' } }; + }), + }); + const program = parseMaestroProgram( + [ + 'appId: com.example.checkout', + '---', + '- launchApp:', + ' clearState: true', + ' launchArguments:', + ' seed: 7', + '- openLink: https://example.test/checkout', + '- inputText:', + ' text: ada@example.com', + ' label: email', + '- eraseText:', + ' charactersToErase: 3', + '- pasteText: pasted', + '- scroll', + '- scrollUntilVisible:', + ' element: Checkout', + ' direction: up', + ' timeout: 1200', + '- hideKeyboard', + '- pressKey: Enter', + '- back', + '- waitForAnimationToEnd: 50', + '- takeScreenshot: checkout.png', + '- runScript:', + ' file: setup.js', + ' env:', + ' SEED: 7', + ].join('\n'), + ); + + const result = await executeMaestroProgram(program, createMaestroRuntimePort(operations)); + + expect(result).toEqual({ + executed: 13, + skipped: 0, + generation: 12, + artifactPaths: ['artifact://checkout.png'], + }); + expect(calls.map(({ kind }) => kind)).toEqual([ + 'launchApp', + 'openLink', + 'inputText', + 'eraseText', + 'pasteText', + 'scroll', + 'scrollUntilVisible', + 'hideKeyboard', + 'pressKey', + 'back', + 'waitForAnimationToEnd', + 'takeScreenshot', + 'runScript', + ]); + expect(calls[0]).toMatchObject({ + kind: 'launchApp', + input: { + appId: 'com.example.checkout', + clearState: true, + launchArguments: { kind: 'map', values: { seed: 7 } }, + }, + generation: 0, + appId: 'com.example.checkout', + }); + expect(calls[6]).toMatchObject({ + kind: 'scrollUntilVisible', + input: { selector: { text: 'Checkout' }, direction: 'up', timeoutMs: 1200 }, + }); + expect(calls[7]).toMatchObject({ kind: 'hideKeyboard', input: {}, generation: 7 }); + expect(calls[8]).toMatchObject({ kind: 'pressKey', input: { key: 'enter' }, generation: 8 }); + expect(calls[10]).toMatchObject({ + kind: 'waitForAnimationToEnd', + input: { timeoutMs: 50 }, + generation: 10, + }); + expect(calls[12]).toMatchObject({ + kind: 'runScript', + input: { file: 'setup.js', env: { SEED: 7 } }, + generation: 11, + }); + }); + + test('invalidates observation evidence after successful waits and scripts', async () => { + const operations = makeOperations({ + waitForAnimationToEnd: vi.fn(async () => undefined), + runScript: vi.fn(async () => undefined), + }); + const port = createMaestroRuntimePort(operations); + + await expect( + port.execute({ + command: { kind: 'waitForAnimationToEnd', source: { line: 2 }, timeout: 50 }, + generation: 4, + env: {}, + }), + ).resolves.toEqual({ mutated: true }); + await expect( + port.execute({ + command: { kind: 'runScript', source: { line: 3 }, file: 'setup.js' }, + generation: 5, + env: {}, + }), + ).resolves.toEqual({ mutated: true }); + }); + + test('keeps selector evidence and target geometry in the current generation', async () => { + const resolved: Record = { + ready: { + generation: 0, + matched: true, + visible: true, + candidateCount: 1, + rect: { x: 24, y: 44, width: 120, height: 48 }, + viewport: { x: 0, y: 0, width: 402, height: 874 }, + ref: 'e5', + }, + pager: { + generation: 1, + matched: true, + visible: true, + candidateCount: 1, + rect: { x: 100, y: 200, width: 100, height: 80 }, + viewport: { x: 0, y: 0, width: 402, height: 874 }, + ref: 'e12', + }, + }; + const observe = vi.fn( + async ({ condition }: Parameters[0]) => ({ + generation: 0, + matched: true, + visible: true, + candidateCount: 1, + rect: { x: 20, y: 40, width: 120, height: 48 }, + viewport: { x: 0, y: 0, width: 402, height: 874 }, + ref: condition.selector.id === 'ready' ? 'e4' : undefined, + }), + ); + const resolveTarget = vi.fn( + async ( + { selector }: Parameters[0], + context: MaestroRuntimeOperationContext, + ) => ({ + ...(resolved[selector.id ?? selector.text ?? ''] ?? { + generation: context.generation, + matched: false, + visible: false, + candidateCount: 0, + }), + }), + ); + const tapOn = vi.fn(async () => undefined); + const gesture = vi.fn(async () => undefined); + const operations = makeOperations({ + observe, + resolveTarget, + tapOn, + gesture, + resolveGestureViewport: vi.fn(async () => ({ x: 0, y: 0, width: 402, height: 874 })), + }); + const program = parseMaestroProgram( + [ + '---', + '- assertVisible:', + ' id: ready', + '- tapOn:', + ' id: ready', + '- swipe:', + ' from:', + ' id: pager', + ' direction: right', + ].join('\n'), + ); + + const result = await executeMaestroProgram(program, createMaestroRuntimePort(operations)); + + expect(result.generation).toBe(2); + expect(resolveTarget).toHaveBeenCalledTimes(2); + expect(tapOn).toHaveBeenCalledWith( + expect.objectContaining({ + target: expect.objectContaining({ + point: { x: 84, y: 68 }, + resolution: expect.objectContaining({ + ref: 'e5', + rect: { x: 24, y: 44, width: 120, height: 48 }, + }), + }), + }), + expect.objectContaining({ generation: 0 }), + ); + expect(gesture).toHaveBeenCalledWith( + { + intent: 'pan', + origin: { x: 150, y: 240 }, + delta: { x: 150, y: 0 }, + durationMs: 400, + executionProfile: 'endpoint-hold', + }, + expect.objectContaining({ + generation: 1, + authoredSwipe: { kind: 'target', from: { id: 'pager' }, direction: 'right' }, + swipeTarget: expect.objectContaining({ ref: 'e12' }), + }), + ); + expect(operations.resolveGestureViewport).not.toHaveBeenCalled(); + }); + + test('skips an absent optional tap without hiding resolver infrastructure failures', async () => { + const tapOn = vi.fn(async () => undefined); + const missingOperations = makeOperations({ + resolveTarget: vi.fn(async ({ purpose }, context) => ({ + generation: context.generation, + matched: false, + visible: false, + candidateCount: 0, + ...(purpose === 'tap' ? {} : { ref: 'unexpected' }), + })), + tapOn, + }); + const command = parseMaestroProgram('---\n- tapOn:\n text: Missing\n optional: true\n') + .commands[0]!; + + const result = await createMaestroRuntimePort(missingOperations).execute({ + command: command as Extract, + generation: 0, + env: {}, + }); + + expect(result).toEqual({ mutated: false }); + expect(missingOperations.resolveTarget).toHaveBeenCalledWith( + expect.objectContaining({ purpose: 'tap', selector: { text: 'Missing' } }), + expect.any(Object), + ); + expect(tapOn).not.toHaveBeenCalled(); + + const failure = new Error('snapshot transport failed'); + const failingOperations = makeOperations({ + resolveTarget: vi.fn(async () => { + throw failure; + }), + tapOn, + }); + await expect( + createMaestroRuntimePort(failingOperations).execute({ + command: command as Extract, + generation: 0, + env: {}, + }), + ).rejects.toBe(failure); + }); +}); diff --git a/src/compat/maestro/__tests__/runtime-target-fixtures.ts b/src/compat/maestro/__tests__/runtime-target-fixtures.ts new file mode 100644 index 000000000..8c075217e --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-target-fixtures.ts @@ -0,0 +1,12 @@ +import type { SnapshotNode, SnapshotState } from '../../../kernel/snapshot.ts'; + +export const IOS_TAB_FRAME = { referenceWidth: 402, referenceHeight: 874 } as const; + +export type SnapshotNodeFixture = Omit & { ref?: string }; + +export function makeSnapshot(nodes: SnapshotNodeFixture[]): SnapshotState { + return { + createdAt: Date.now(), + nodes: nodes.map((node) => ({ ref: `e${node.index}`, ...node })), + }; +} diff --git a/src/compat/maestro/__tests__/runtime-target-matching.test.ts b/src/compat/maestro/__tests__/runtime-target-matching.test.ts new file mode 100644 index 000000000..d2cc7f3c1 --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-target-matching.test.ts @@ -0,0 +1,14 @@ +import { expect, test } from 'vitest'; +import { findMaestroTypedSelectorMatches } from '../runtime-target-matching.ts'; +import { makeSnapshot } from './runtime-target-fixtures.ts'; + +test('typed target matching preserves snapshot read order', () => { + const snapshot = makeSnapshot([ + { index: 1, label: 'Save', rect: { x: 10, y: 10, width: 100, height: 40 } }, + { index: 2, label: 'Save', rect: { x: 10, y: 80, width: 100, height: 40 } }, + ]); + + expect( + findMaestroTypedSelectorMatches(snapshot, { text: 'Save' }).map((node) => node.index), + ).toEqual([1, 2]); +}); diff --git a/src/compat/maestro/__tests__/runtime-target-policy.test.ts b/src/compat/maestro/__tests__/runtime-target-policy.test.ts new file mode 100644 index 000000000..4411ee35a --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-target-policy.test.ts @@ -0,0 +1,35 @@ +import { expect, test } from 'vitest'; +import { matchesMaestroTypedSelector } from '../runtime-target-policy.ts'; +import { makeSnapshot } from './runtime-target-fixtures.ts'; + +test('typed Maestro text selectors match visible text and state without expression strings', () => { + const node = makeSnapshot([ + { + index: 1, + type: 'TextView', + value: 'Subtotal: $42.10', + enabled: true, + selected: true, + }, + ]).nodes[0]!; + + expect( + matchesMaestroTypedSelector(node, { text: '^Subtotal.*', enabled: true, selected: true }), + ).toBe(true); + expect(matchesMaestroTypedSelector(node, { text: 'Subtotal', selected: false })).toBe(false); +}); + +test('typed Maestro id and label selectors keep their primary field semantics', () => { + const node = makeSnapshot([ + { + index: 1, + identifier: 'checkout-submit', + label: 'Submit order', + value: 'Submit order', + }, + ]).nodes[0]!; + + expect(matchesMaestroTypedSelector(node, { id: 'checkout-submit' })).toBe(true); + expect(matchesMaestroTypedSelector(node, { label: 'Submit' })).toBe(false); + expect(matchesMaestroTypedSelector(node, { label: '^Submit.*' })).toBe(true); +}); diff --git a/src/compat/maestro/__tests__/runtime-target-ranking.test.ts b/src/compat/maestro/__tests__/runtime-target-ranking.test.ts new file mode 100644 index 000000000..0433074ec --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-target-ranking.test.ts @@ -0,0 +1,54 @@ +import { expect, test } from 'vitest'; +import { selectMaestroSnapshotMatch } from '../runtime-target-ranking.ts'; +import { IOS_TAB_FRAME, makeSnapshot } from './runtime-target-fixtures.ts'; + +test('target ranking promotes actionable controls over matching static text', () => { + const snapshot = makeSnapshot([ + { + index: 1, + type: 'StaticText', + label: 'Save', + rect: { x: 24, y: 100, width: 120, height: 44 }, + }, + { + index: 2, + type: 'Button', + label: 'Save', + rect: { x: 24, y: 300, width: 120, height: 44 }, + }, + ]); + + expect( + selectMaestroSnapshotMatch(snapshot.nodes, snapshot.nodes, undefined, 'Save', IOS_TAB_FRAME), + ).toMatchObject({ node: { index: 2 } }); +}); + +test('target ranking promotes a matched label to a useful actionable ancestor', () => { + const snapshot = makeSnapshot([ + { + index: 1, + type: 'Button', + rect: { x: 20, y: 100, width: 220, height: 64 }, + hittable: true, + }, + { + index: 2, + parentIndex: 1, + type: 'StaticText', + label: 'Continue', + rect: { x: 40, y: 112, width: 120, height: 40 }, + }, + ]); + + expect( + selectMaestroSnapshotMatch( + snapshot.nodes, + [snapshot.nodes[1]!], + undefined, + 'Continue', + IOS_TAB_FRAME, + false, + true, + ), + ).toMatchObject({ node: { index: 1 }, rect: { x: 20, y: 100, width: 220, height: 64 } }); +}); diff --git a/src/compat/maestro/__tests__/runtime-targets-typed.test.ts b/src/compat/maestro/__tests__/runtime-targets-typed.test.ts new file mode 100644 index 000000000..ca1a9c827 --- /dev/null +++ b/src/compat/maestro/__tests__/runtime-targets-typed.test.ts @@ -0,0 +1,71 @@ +import { expect, test } from 'vitest'; +import { resolveMaestroTargetFromSnapshot } from '../runtime-targets.ts'; +import { IOS_TAB_FRAME, makeSnapshot } from './runtime-target-fixtures.ts'; + +test('typed target resolution returns target geometry and structured evidence', () => { + const snapshot = makeSnapshot([ + { + index: 1, + type: 'Button', + identifier: 'continue', + label: 'Continue', + rect: { x: 24, y: 100, width: 180, height: 48 }, + enabled: true, + selected: false, + }, + ]); + + const result = resolveMaestroTargetFromSnapshot( + snapshot, + { selector: { id: 'continue', enabled: true } }, + 'ios', + IOS_TAB_FRAME, + ); + + expect(result).toMatchObject({ + ok: true, + node: { index: 1 }, + rect: { x: 24, y: 100, width: 180, height: 48 }, + matches: 1, + evidence: { + selector: { id: 'continue', enabled: true }, + matched: true, + visible: true, + candidateCount: 1, + ref: 'e1', + }, + }); +}); + +test('typed target resolution applies typed childOf and reports structured misses', () => { + const snapshot = makeSnapshot([ + { index: 1, identifier: 'row', rect: { x: 0, y: 0, width: 320, height: 80 } }, + { + index: 2, + parentIndex: 1, + type: 'Button', + label: 'Delete', + rect: { x: 220, y: 16, width: 64, height: 48 }, + }, + ]); + + const result = resolveMaestroTargetFromSnapshot( + snapshot, + { selector: { text: 'Delete' }, childOf: { id: 'row' } }, + 'android', + { referenceWidth: 1080, referenceHeight: 2340 }, + ); + const missingParent = resolveMaestroTargetFromSnapshot( + snapshot, + { selector: { text: 'Delete' }, childOf: { id: 'missing' } }, + 'android', + { referenceWidth: 1080, referenceHeight: 2340 }, + ); + + expect(result).toMatchObject({ ok: true, node: { index: 2 }, evidence: { candidateCount: 1 } }); + expect(missingParent).toMatchObject({ + ok: false, + message: 'Maestro childOf parent did not match.', + evidence: { matched: true, visible: false, candidateCount: 1 }, + }); +}); diff --git a/src/compat/maestro/__tests__/runtime-targets.test.ts b/src/compat/maestro/__tests__/runtime-targets.test.ts deleted file mode 100644 index b2c66e824..000000000 --- a/src/compat/maestro/__tests__/runtime-targets.test.ts +++ /dev/null @@ -1,1624 +0,0 @@ -import assert from 'node:assert/strict'; -import { test, expect } from 'vitest'; -import type { SnapshotNode, SnapshotState } from '../../../kernel/snapshot.ts'; -import { AppError } from '../../../kernel/errors.ts'; -import { - extractMaestroVisibleTextQuery, - readMaestroSelectorPlatform, - resolveMaestroFuzzyTextNodeFromSnapshot, - resolveMaestroNodeFromSnapshot, - resolveVisibleMaestroNodeFromSnapshot, -} from '../runtime-targets.ts'; - -const IOS_TAB_FRAME = { referenceWidth: 402, referenceHeight: 874 } as const; - -type SnapshotNodeFixture = Omit & { ref?: string }; -type ResolveMaestroOptions = NonNullable[5]>; -type MaestroTapOptions = Parameters[2]; - -function makeSnapshot(nodes: SnapshotNodeFixture[]): SnapshotState { - return { - createdAt: Date.now(), - nodes: nodes.map((node) => ({ ref: `e${node.index}`, ...node })), - }; -} - -function maestroTextSelector(text: string): string { - return `label="${text}" || text="${text}" || id="${text}"`; -} - -function resolveIosTabTarget( - snapshot: SnapshotState, - text: string, - options: ResolveMaestroOptions, -) { - return resolveMaestroNodeFromSnapshot( - snapshot, - maestroTextSelector(text), - {}, - 'ios', - IOS_TAB_FRAME, - options, - ); -} - -function resolveIosNode( - snapshot: SnapshotState, - selector: string, - options: MaestroTapOptions = {}, -) { - return resolveMaestroNodeFromSnapshot(snapshot, selector, options, 'ios', IOS_TAB_FRAME); -} - -test('resolveVisibleMaestroNodeFromSnapshot treats app content behind React Native overlays as hidden', () => { - const snapshot = makeReactNativeOverlaySnapshot(); - - const appContent = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Article title" || text="Article title" || id="Article title"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - const overlayControl = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Minimize" || text="Minimize" || id="Minimize"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(appContent).toMatchObject({ - ok: false, - message: expect.stringContaining('React Native overlay is covering app content'), - }); - expect(overlayControl).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Minimize' }), - }); -}); - -test('resolveMaestroNodeFromSnapshot blocks taps on app content behind React Native overlays', () => { - const snapshot = makeReactNativeOverlaySnapshot(); - - const appContent = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Article title" || text="Article title" || id="Article title"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - const overlayControl = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Dismiss" || text="Dismiss" || id="Dismiss"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(appContent).toMatchObject({ - ok: false, - message: expect.stringContaining('React Native overlay is covering app content'), - }); - expect(overlayControl).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Dismiss' }), - }); -}); - -test('resolveVisibleMaestroNodeFromSnapshot ignores hidden React Native overlay controls', () => { - const snapshot = makeReactNativeOverlaySnapshot(); - snapshot.nodes = snapshot.nodes.map((node) => - node.label === 'Dismiss' || node.label === 'Minimize' - ? { ...node, visibleToUser: false } - : node, - ); - - const appContent = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Article title" || text="Article title" || id="Article title"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(appContent).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Article title' }), - }); -}); - -test('resolveVisibleMaestroNodeFromSnapshot blocks content behind control-less RedBox text', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'StaticText', - label: 'Article title', - rect: { x: 24, y: 120, width: 200, height: 44 }, - depth: 4, - }, - { - index: 2, - ref: 'e2', - type: 'Other', - label: - "Uncaught (in promise): Error: Unable to download asset from url: 'http://localhost:8081/assets/icon.ttf'", - rect: { x: 0, y: 0, width: 402, height: 100 }, - depth: 2, - }, - ], - }; - - const target = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Article title" || text="Article title" || id="Article title"', - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - ); - - expect(target).toMatchObject({ - ok: false, - message: expect.stringContaining('React Native overlay is covering app content'), - }); -}); - -test('resolveMaestroNodeFromSnapshot does not match plain text as a substring', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.Button', - label: 'Push feed', - rect: { x: 32, y: 320, width: 280, height: 96 }, - enabled: true, - hittable: true, - depth: 5, - }, - { - index: 2, - ref: 'e2', - type: 'StaticText', - label: 'Albums, back', - rect: { x: 120, y: 80, width: 180, height: 48 }, - depth: 5, - }, - ], - }; - - const plain = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Feed" || text="Feed" || id="Feed"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - const regex = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label=".*feed" || text=".*feed" || id=".*feed"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - const compositeAssertion = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Albums" || text="Albums" || id="Albums"', - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - ); - const compositeTap = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Albums" || text="Albums" || id="Albums"', - {}, - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - ); - - expect(plain).toMatchObject({ - ok: false, - message: expect.stringContaining('Feed'), - }); - expect(regex).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Push feed' }), - }); - expect(compositeAssertion).toMatchObject({ - ok: false, - message: expect.stringContaining('Albums'), - }); - expect(compositeTap).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Albums, back' }), - }); -}); - -test('resolveVisibleMaestroNodeFromSnapshot does not block content behind collapsed React Native warnings', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label: 'Morning Favorites', - rect: { x: 24, y: 420, width: 320, height: 54 }, - depth: 8, - }, - { - index: 2, - ref: 'e2', - type: 'android.view.ViewGroup', - label: 'Open debugger to view warnings', - rect: { x: 0, y: 2190, width: 1080, height: 96 }, - depth: 6, - }, - ], - }; - - const appContent = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Morning Favorites" || text="Morning Favorites" || id="Morning Favorites"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(appContent).toMatchObject({ - ok: true, - node: expect.objectContaining({ label: 'Morning Favorites' }), - }); -}); - -test('resolveMaestroNodeFromSnapshot childOf resolves parent links by node index', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 7, - ref: 'e7', - identifier: 'other-row', - rect: { x: 0, y: 0, width: 320, height: 80 }, - }, - { - index: 99, - ref: 'e99', - parentIndex: 42, - identifier: 'childActionButton', - rect: { x: 240, y: 120, width: 64, height: 48 }, - }, - { - index: 42, - ref: 'e42', - identifier: 'parent-row-secondary', - rect: { x: 0, y: 96, width: 320, height: 80 }, - }, - { - index: 8, - ref: 'e8', - parentIndex: 7, - identifier: 'childActionButton', - rect: { x: 240, y: 16, width: 64, height: 48 }, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'id="childActionButton"', - { childOf: 'id="parent-row-secondary"' }, - 'ios', - { referenceWidth: 320, referenceHeight: 640 }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 99 }), - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers foreground duplicate matches', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'button', - label: 'Show Dialog', - rect: { x: 24, y: 220, width: 240, height: 72 }, - depth: 8, - }, - { - index: 2, - ref: 'e2', - type: 'button', - label: 'Show Dialog', - rect: { x: 24, y: 220, width: 240, height: 72 }, - depth: 8, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Show Dialog" || text="Show Dialog" || id="Show Dialog"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - }); -}); - -test('resolveMaestroNodeFromSnapshot preserves read order for duplicate matches in different rects', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'button', - label: 'Open details', - rect: { x: 24, y: 520, width: 240, height: 72 }, - depth: 8, - }, - { - index: 2, - ref: 'e2', - type: 'button', - label: 'Open details', - rect: { x: 24, y: 320, width: 240, height: 72 }, - depth: 8, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Open details" || text="Open details" || id="Open details"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers duplicate text on foreground overlapping screen', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.ScrollView', - label: 'Article, Go back, Show Dialog', - rect: { x: 0, y: 120, width: 1080, height: 1800 }, - depth: 6, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.Button', - label: 'Show Dialog', - rect: { x: 720, y: 980, width: 280, height: 88 }, - enabled: true, - hittable: true, - depth: 14, - parentIndex: 1, - }, - { - index: 30, - ref: 'e30', - type: 'android.widget.ScrollView', - label: 'NewsFeed, Push NewsFeed, Show Dialog', - rect: { x: 0, y: 120, width: 1080, height: 1800 }, - depth: 6, - }, - { - index: 31, - ref: 'e31', - type: 'android.widget.Button', - label: 'Show Dialog', - rect: { x: 720, y: 1320, width: 280, height: 88 }, - enabled: true, - hittable: true, - depth: 14, - parentIndex: 30, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Show Dialog" || text="Show Dialog" || id="Show Dialog"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 31 }), - rect: { x: 720, y: 1320, width: 280, height: 88 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers full-width screen over stale side navigation duplicate', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.ScrollView', - label: 'Article, Go back, Replace params', - rect: { x: 0, y: 319, width: 1080, height: 2021 }, - depth: 17, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.Button', - label: 'Go back', - rect: { x: 33, y: 667, width: 1014, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 1, - }, - { - index: 30, - ref: 'e30', - type: 'android.widget.ScrollView', - label: 'Albums, Go back, Go to Article', - rect: { x: 0, y: 319, width: 816, height: 2021 }, - depth: 17, - }, - { - index: 31, - ref: 'e31', - type: 'android.widget.Button', - label: 'Go back', - rect: { x: 0, y: 581, width: 783, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 30, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Go back" || text="Go back" || id="Go back"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 33, y: 667, width: 1014, height: 110 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot uses visible assertion context for equal overlapping screens', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 6, - ref: 'e6', - type: 'android.view.View', - label: 'Albums', - value: 'Albums', - rect: { x: 154, y: 194, width: 188, height: 74 }, - depth: 22, - parentIndex: 25, - }, - { - index: 7, - ref: 'e7', - type: 'android.widget.ScrollView', - rect: { x: 0, y: 319, width: 1080, height: 2021 }, - depth: 17, - }, - { - index: 8, - ref: 'e8', - type: 'android.widget.Button', - label: 'Push albums', - rect: { x: 33, y: 352, width: 361, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 7, - }, - { - index: 12, - ref: 'e12', - type: 'android.widget.Button', - label: 'Push article', - rect: { x: 33, y: 495, width: 340, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 7, - }, - { - index: 13, - ref: 'e13', - type: 'android.widget.TextView', - label: 'Push article', - value: 'Push article', - rect: { x: 99, y: 523, width: 208, height: 55 }, - depth: 20, - parentIndex: 12, - }, - { - index: 25, - ref: 'e25', - type: 'android.widget.ScrollView', - rect: { x: 0, y: 319, width: 1080, height: 2021 }, - depth: 17, - }, - { - index: 26, - ref: 'e26', - type: 'android.widget.Button', - label: 'Push article', - rect: { x: 33, y: 352, width: 340, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 25, - }, - { - index: 27, - ref: 'e27', - type: 'android.widget.TextView', - label: 'Push article', - value: 'Push article', - rect: { x: 99, y: 380, width: 208, height: 55 }, - depth: 20, - parentIndex: 26, - }, - { - index: 30, - ref: 'e30', - type: 'android.widget.Button', - label: 'Push albums', - rect: { x: 33, y: 495, width: 361, height: 110 }, - enabled: true, - hittable: true, - depth: 19, - parentIndex: 25, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Push article" || text="Push article" || id="Push article"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { - promoteTapTarget: true, - preferredContext: { - node: snapshot.nodes[0]!, - rect: snapshot.nodes[0]!.rect!, - }, - }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 12 }), - rect: { x: 33, y: 495, width: 340, height: 110 }, - }); -}); - -test('resolveVisibleMaestroNodeFromSnapshot requires visible text matches to be on screen', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label: 'Library', - rect: { x: 0, y: 2340, width: 120, height: 48 }, - depth: 8, - }, - ], - }; - - const target = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Library" || text="Library" || id="Library"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(target).toMatchObject({ - ok: false, - message: expect.stringContaining('none were visible'), - }); -}); - -test('resolveVisibleMaestroNodeFromSnapshot ignores Android rectless hidden navigation labels', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.view.ViewGroup', - label: '', - rect: { x: 0, y: 0, width: 1080, height: 2340 }, - depth: 1, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.Button', - label: 'Chat', - enabled: true, - hittable: true, - depth: 2, - parentIndex: 1, - }, - { - index: 3, - ref: 'e3', - type: 'android.widget.TextView', - label: 'Chat', - value: 'Chat', - depth: 3, - parentIndex: 2, - }, - ], - }; - - const target = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - 'label="Chat" || text="Chat" || id="Chat"', - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - ); - - expect(target).toMatchObject({ - ok: false, - message: expect.stringContaining('none were visible'), - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers concrete Android tab rect over hidden drawer label', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.view.ViewGroup', - label: '', - rect: { x: 0, y: 0, width: 1080, height: 2340 }, - depth: 1, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.FrameLayout', - label: 'Albums', - rect: { x: 540, y: 2054, width: 270, height: 220 }, - enabled: true, - hittable: true, - depth: 16, - parentIndex: 1, - }, - { - index: 3, - ref: 'e3', - type: 'android.view.ViewGroup', - label: '', - rect: { x: 0, y: 0, width: 816, height: 2340 }, - depth: 1, - }, - { - index: 4, - ref: 'e4', - type: 'android.widget.Button', - label: '\udb80\udeea, Albums', - enabled: true, - hittable: true, - depth: 18, - parentIndex: 3, - }, - { - index: 5, - ref: 'e5', - type: 'android.widget.TextView', - label: 'Albums', - value: 'Albums', - enabled: true, - hittable: false, - depth: 19, - parentIndex: 4, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Albums" || text="Albums" || id="Albums"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 540, y: 2054, width: 270, height: 220 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot ignores Android nodes hidden from users', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.Button', - label: 'Settings', - rect: { x: 0, y: 0, width: 200, height: 80 }, - enabled: true, - hittable: true, - visibleToUser: false, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.Button', - label: 'Settings', - rect: { x: 300, y: 700, width: 200, height: 80 }, - enabled: true, - hittable: true, - visibleToUser: true, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Settings"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 300, y: 700, width: 200, height: 80 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers exact Android tab label over normalized header icon text', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.FrameLayout', - label: 'Search', - rect: { x: 810, y: 2054, width: 270, height: 132 }, - enabled: true, - hittable: true, - depth: 16, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.Button', - label: 'search', - rect: { x: 673, y: 165, width: 132, height: 132 }, - enabled: true, - hittable: true, - depth: 22, - }, - { - index: 3, - ref: 'e3', - type: 'android.widget.TextView', - label: 'search', - value: 'search', - rect: { x: 706, y: 198, width: 66, height: 66 }, - enabled: true, - depth: 23, - parentIndex: 2, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Search" || text="Search" || id="Search"', - {}, - 'android', - { referenceWidth: 1080, referenceHeight: 2340 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 810, y: 2054, width: 270, height: 132 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot infers missing selected tab slot from tab-strip children', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'ScrollView', - label: 'Chat', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 3, - }, - { - index: 2, - ref: 'e2', - type: 'Cell', - label: 'Contacts', - rect: { x: 134, y: 116.66666412353516, width: 134, height: 48 }, - depth: 4, - parentIndex: 1, - }, - { - index: 3, - ref: 'e3', - type: 'Cell', - label: 'Albums', - rect: { x: 268, y: 116.66666412353516, width: 134, height: 48 }, - depth: 4, - parentIndex: 1, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Chat" || text="Chat" || id="Chat"', - {}, - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 0, y: 116.66666412353516, width: 134, height: 48 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot infers leading tab slot with selected sibling and content context', () => { - const snapshot = makeSnapshot([ - { index: 1, type: 'Window', rect: { x: 0, y: 0, width: 402, height: 874 }, depth: 1 }, - { - index: 4, - type: 'ScrollView', - label: 'Chat', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 3, - parentIndex: 1, - }, - { - index: 5, - type: 'Cell', - label: 'Contacts', - rect: { x: 134, y: 116.66666412353516, width: 134, height: 48.00000762939453 }, - depth: 4, - parentIndex: 4, - }, - { - index: 6, - type: 'Cell', - label: 'Albums', - selected: true, - rect: { x: 268, y: 116.66666412353516, width: 134, height: 48.00000762939453 }, - depth: 4, - parentIndex: 4, - }, - { - index: 10, - type: 'Other', - label: 'album-0', - identifier: 'album-0', - rect: { x: 16, y: 220, width: 100, height: 100 }, - depth: 7, - parentIndex: 1, - }, - ]); - const contentNode = snapshot.nodes[4]!; - - const target = resolveIosTabTarget(snapshot, 'Chat', { - promoteTapTarget: true, - preferredContext: { node: contentNode, rect: contentNode.rect! }, - }); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 4 }), - rect: { x: 0, y: 116.66666412353516, width: 134, height: 48 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot infers leading tab slot from broad matching cell', () => { - const snapshot = makeSnapshot([ - { - index: 4, - type: 'ScrollView', - label: 'Article', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 3, - }, - { - index: 5, - type: 'Cell', - label: 'Article', - rect: { x: -9, y: 116.66666412353516, width: 560, height: 48.00000762939453 }, - depth: 4, - parentIndex: 4, - }, - { - index: 6, - type: 'Cell', - label: 'Contacts', - selected: true, - rect: { x: 141, y: 116.66666412353516, width: 120, height: 48.00000762939453 }, - depth: 5, - parentIndex: 5, - }, - { - index: 7, - type: 'Cell', - label: 'Albums', - rect: { x: 281, y: 116.66666412353516, width: 120, height: 48.00000762939453 }, - depth: 5, - parentIndex: 5, - }, - { - index: 8, - type: 'Cell', - label: 'Chat', - rect: { x: 421, y: 116.66666412353516, width: 120, height: 48.00000762939453 }, - depth: 5, - parentIndex: 5, - }, - ]); - - const target = resolveIosTabTarget(snapshot, 'Article', { promoteTapTarget: true }); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 5 }), - rect: { x: -9, y: 116.66666412353516, width: 150, height: 48.00000762939453 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot rejects off-screen interaction targets when required', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'ScrollView', - label: 'Article', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 3, - }, - { - index: 2, - type: 'Other', - label: 'Contacts', - rect: { - x: -173.66666412353516, - y: 116.66666412353516, - width: 91.99999237060547, - height: 48, - }, - depth: 4, - parentIndex: 1, - }, - ]); - - const target = resolveIosTabTarget(snapshot, 'Contacts', { - promoteTapTarget: true, - requireOnScreen: true, - }); - - expect(target).toMatchObject({ - ok: false, - message: expect.stringContaining('none were visible'), - }); -}); - -test('resolveMaestroNodeFromSnapshot does not promote tab child to broad tab-strip cell', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'Cell', - label: 'Article', - rect: { - x: -150, - y: 116.66666412353516, - width: 701.6666870117188, - height: 48.00000762939453, - }, - depth: 4, - }, - { - index: 2, - type: 'Other', - label: 'Contacts', - rect: { - x: -44.666664123535156, - y: 116.66666412353516, - width: 91.99999237060547, - height: 48.00000762939453, - }, - depth: 5, - parentIndex: 1, - }, - ]); - - const target = resolveIosTabTarget(snapshot, 'Contacts', { - promoteTapTarget: true, - requireOnScreen: true, - }); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { - x: -44.666664123535156, - y: 116.66666412353516, - width: 91.99999237060547, - height: 48.00000762939453, - }, - }); -}); - -test('resolveMaestroNodeFromSnapshot infers missing selected tab slot from nested tab-strip children', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'Other', - label: 'Chat', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 4, - }, - { - index: 2, - type: 'ScrollView', - label: 'Chat', - rect: { x: 0, y: 116.66666412353516, width: 402, height: 48 }, - depth: 5, - parentIndex: 1, - }, - { - index: 3, - type: 'Other', - label: 'Contacts', - selected: true, - rect: { x: 134, y: 116.66666412353516, width: 134, height: 48 }, - depth: 6, - parentIndex: 2, - }, - { - index: 4, - type: 'Other', - label: 'Albums', - rect: { x: 268, y: 116.66666412353516, width: 134, height: 48 }, - depth: 6, - parentIndex: 2, - }, - ]); - - const target = resolveIosTabTarget(snapshot, 'Chat', { promoteTapTarget: true }); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 0, y: 116.66666412353516, width: 134, height: 48 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot keeps concrete child matches over tab-strip inference', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'ScrollView', - label: 'Article by Gandalf', - rect: { x: 0, y: 58.33333333333333, width: 402, height: 58.33333333333333 }, - depth: 4, - }, - { - index: 2, - ref: 'e2', - type: 'Cell', - label: 'Article by Gandalf', - rect: { x: 8, y: 65.33333587646484, width: 155, height: 48 }, - depth: 5, - parentIndex: 1, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Article by Gandalf" || text="Article by Gandalf" || id="Article by Gandalf"', - {}, - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 8, y: 65.33333587646484, width: 155, height: 48 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot prefers localized breadcrumb label over broad containers', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'Other', - label: 'Article by Gandalf', - rect: { x: 0, y: 0, width: 402, height: 116.66666412353516 }, - depth: 12, - }, - { - index: 2, - ref: 'e2', - type: 'ScrollView', - label: 'Article by Gandalf', - rect: { x: 0, y: 0, width: 402, height: 116.66666666666666 }, - depth: 13, - parentIndex: 1, - }, - { - index: 3, - ref: 'e3', - type: 'Other', - label: 'Article by Gandalf', - rect: { x: 0, y: 0, width: 232.3333282470703, height: 116.33333587646484 }, - depth: 14, - parentIndex: 2, - }, - { - index: 4, - ref: 'e4', - type: 'Other', - label: 'Article by Gandalf', - rect: { x: 0, y: 0, width: 232.3333282470703, height: 116.33333587646484 }, - depth: 15, - parentIndex: 3, - }, - { - index: 5, - ref: 'e5', - type: 'Other', - label: 'Article by Gandalf', - rect: { x: 8, y: 65.33333587646484, width: 155, height: 48 }, - depth: 16, - parentIndex: 4, - }, - { - index: 6, - ref: 'e6', - type: 'Other', - label: 'Feed', - rect: { x: 170.3333282470703, y: 65.33333587646484, width: 54, height: 48 }, - depth: 16, - parentIndex: 4, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Article by Gandalf" || text="Article by Gandalf" || id="Article by Gandalf"', - {}, - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 5 }), - rect: { x: 8, y: 65.33333587646484, width: 155, height: 48 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot infers leading breadcrumb slot when selected child is omitted', () => { - const snapshot: SnapshotState = { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'ScrollView', - label: 'Article by Gandalf', - rect: { x: 0, y: 58.33333333333333, width: 402, height: 58.33333333333333 }, - depth: 4, - }, - { - index: 2, - ref: 'e2', - type: 'Other', - label: 'Feed', - rect: { x: 170.3333282470703, y: 65.33333587646484, width: 54, height: 48 }, - depth: 5, - parentIndex: 1, - }, - { - index: 3, - ref: 'e3', - type: 'Other', - label: 'Albums', - rect: { x: 231.6666717529297, y: 65.33333587646484, width: 75, height: 48 }, - depth: 5, - parentIndex: 1, - }, - ], - }; - - const target = resolveMaestroNodeFromSnapshot( - snapshot, - 'label="Article by Gandalf" || text="Article by Gandalf" || id="Article by Gandalf"', - {}, - 'ios', - { referenceWidth: 402, referenceHeight: 874 }, - { promoteTapTarget: true }, - ); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 0, y: 58.33333333333333, width: 168, height: 58.33333333333333 }, - }); -}); - -test('readMaestroSelectorPlatform defaults to iOS unless flags request exactly "android"', () => { - const flagsWithPlatform = (platform: string) => - ({ platform }) as Parameters[0]; - - expect(readMaestroSelectorPlatform({ platform: 'android' })).toBe('android'); - expect(readMaestroSelectorPlatform({ platform: 'ios' })).toBe('ios'); - expect(readMaestroSelectorPlatform(undefined)).toBe('ios'); - // The comparison is case-sensitive and exact: anything but lowercase - // 'android' falls back to iOS. - expect(readMaestroSelectorPlatform(flagsWithPlatform('Android'))).toBe('ios'); - expect(readMaestroSelectorPlatform(flagsWithPlatform('tvos'))).toBe('ios'); -}); - -test('extractMaestroVisibleTextQuery extracts the shared text from text selector chains', () => { - expect(extractMaestroVisibleTextQuery('label="Save" || text="Save" || id="Save"')).toBe('Save'); - expect(extractMaestroVisibleTextQuery('label="Save"')).toBe('Save'); -}); - -test('extractMaestroVisibleTextQuery keeps exact selector paths for mixed selectors', () => { - expect(extractMaestroVisibleTextQuery('id="row-1"')).toBeNull(); - expect(extractMaestroVisibleTextQuery('label="Save" || text="Cancel"')).toBeNull(); - expect(extractMaestroVisibleTextQuery('role="button" label="Save"')).toBeNull(); -}); - -test('resolveMaestroFuzzyTextNodeFromSnapshot prefers exact normalized text over partial matches', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'StaticText', - label: 'Sign In Now', - rect: { x: 24, y: 100, width: 200, height: 44 }, - depth: 3, - }, - { - index: 2, - type: 'StaticText', - label: 'sign in', - rect: { x: 24, y: 200, width: 200, height: 44 }, - depth: 3, - }, - ]); - - const exact = resolveMaestroFuzzyTextNodeFromSnapshot(snapshot, 'Sign In', 'ios', IOS_TAB_FRAME); - const partial = resolveMaestroFuzzyTextNodeFromSnapshot(snapshot, 'In Now', 'ios', IOS_TAB_FRAME); - - expect(exact).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 24, y: 200, width: 200, height: 44 }, - }); - expect(partial).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 24, y: 100, width: 200, height: 44 }, - }); -}); - -test('resolveMaestroFuzzyTextNodeFromSnapshot reports unmatched and blank fuzzy queries', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'StaticText', - label: 'Welcome', - rect: { x: 24, y: 100, width: 200, height: 44 }, - depth: 3, - }, - ]); - - const blankQuery = ' '.repeat(3); - const missing = resolveMaestroFuzzyTextNodeFromSnapshot( - snapshot, - 'Checkout', - 'ios', - IOS_TAB_FRAME, - ); - const blank = resolveMaestroFuzzyTextNodeFromSnapshot(snapshot, blankQuery, 'ios', IOS_TAB_FRAME); - - expect(missing).toEqual({ ok: false, message: 'Maestro fuzzy text did not match: Checkout' }); - expect(blank).toEqual({ ok: false, message: `Maestro fuzzy text did not match: ${blankQuery}` }); -}); - -test('resolveMaestroNodeFromSnapshot index option selects matches in snapshot order', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'Button', - label: 'Delete', - rect: { x: 24, y: 100, width: 120, height: 44 }, - depth: 3, - }, - { - index: 2, - type: 'Button', - label: 'Delete', - rect: { x: 24, y: 300, width: 120, height: 44 }, - depth: 3, - }, - ]); - - const first = resolveIosNode(snapshot, 'label="Delete"', { index: 0 }); - const second = resolveIosNode(snapshot, 'label="Delete"', { index: 1 }); - - expect(first).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 24, y: 100, width: 120, height: 44 }, - }); - expect(second).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 24, y: 300, width: 120, height: 44 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot reports the requested index when out of range', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'Button', - label: 'Delete', - rect: { x: 24, y: 100, width: 120, height: 44 }, - depth: 3, - }, - ]); - - const target = resolveIosNode(snapshot, 'label="Delete"', { index: 5 }); - - expect(target).toEqual({ - ok: false, - message: 'Maestro selector did not match index 5: label="Delete"', - }); -}); - -test('resolveMaestroNodeFromSnapshot reports missing childOf parents', () => { - const snapshot = makeSnapshot([ - { index: 1, identifier: 'row', rect: { x: 0, y: 0, width: 320, height: 80 }, depth: 2 }, - { - index: 2, - parentIndex: 1, - identifier: 'action', - rect: { x: 240, y: 16, width: 64, height: 48 }, - depth: 3, - }, - ]); - - const target = resolveIosNode(snapshot, 'id="action"', { childOf: 'id="missing-parent"' }); - - expect(target).toEqual({ - ok: false, - message: 'Maestro childOf parent did not match: id="missing-parent"', - }); -}); - -test('resolveMaestroNodeFromSnapshot inherits the nearest ancestor rect for rectless matches', () => { - const snapshot = makeSnapshot([ - { index: 1, type: 'Other', rect: { x: 10, y: 20, width: 300, height: 50 }, depth: 1 }, - { index: 2, type: 'Button', label: 'Submit', hittable: true, depth: 2, parentIndex: 1 }, - ]); - - const target = resolveIosNode(snapshot, 'label="Submit"'); - - expect(target).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 2 }), - rect: { x: 10, y: 20, width: 300, height: 50 }, - }); -}); - -test('resolveMaestroNodeFromSnapshot reports matches hidden by zero-size rects', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'StaticText', - label: 'Ghost', - rect: { x: 0, y: 0, width: 0, height: 0 }, - depth: 2, - }, - ]); - - const target = resolveIosNode(snapshot, 'label="Ghost"'); - - expect(target).toEqual({ - ok: false, - message: 'Maestro selector matched 1 element(s), but none were visible: label="Ghost"', - }); -}); - -test('resolveMaestroNodeFromSnapshot treats regex-looking text terms as regular expressions', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'StaticText', - label: 'Subtotal: $42.10', - rect: { x: 24, y: 100, width: 240, height: 44 }, - depth: 3, - }, - ]); - - const regex = resolveIosNode(snapshot, 'label="^Subtotal.*"'); - const invalidRegex = resolveIosNode(snapshot, 'label="^[unclosed"'); - - expect(regex).toMatchObject({ - ok: true, - node: expect.objectContaining({ index: 1 }), - rect: { x: 24, y: 100, width: 240, height: 44 }, - }); - expect(invalidRegex).toEqual({ - ok: false, - message: 'Maestro selector did not match index 0: label="^[unclosed"', - }); -}); - -test('resolveMaestroNodeFromSnapshot propagates selector parser errors uncaught', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'Button', - label: 'Save', - rect: { x: 24, y: 100, width: 120, height: 44 }, - depth: 3, - }, - ]); - - // Exact parser wording is owned by src/selectors/arguments.ts and pinned - // in src/daemon/__tests__/selectors.test.ts; here we only assert that the - // parser error reaches the caller unchanged. - assert.throws( - () => resolveIosNode(snapshot, 'bogus="Save"'), - (error) => - error instanceof AppError && - error.code === 'INVALID_ARGS' && - /Unknown selector key/i.test(error.message), - ); -}); - -test('resolveVisibleMaestroNodeFromSnapshot reports the visible match count and prefers tap targets', () => { - const snapshot = makeSnapshot([ - { - index: 1, - type: 'StaticText', - label: 'Save', - rect: { x: 24, y: 100, width: 120, height: 44 }, - depth: 3, - }, - { - index: 2, - type: 'Button', - label: 'Save', - rect: { x: 24, y: 300, width: 120, height: 44 }, - depth: 3, - }, - ]); - - const target = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - maestroTextSelector('Save'), - 'ios', - IOS_TAB_FRAME, - ); - - expect(target).toMatchObject({ - ok: true, - matches: 2, - node: expect.objectContaining({ index: 2 }), - rect: { x: 24, y: 300, width: 120, height: 44 }, - }); -}); - -function makeReactNativeOverlaySnapshot(): SnapshotState { - return { - createdAt: Date.now(), - nodes: [ - { - index: 1, - ref: 'e1', - type: 'android.widget.TextView', - label: 'Article title', - rect: { x: 24, y: 420, width: 320, height: 54 }, - depth: 8, - }, - { - index: 2, - ref: 'e2', - type: 'android.widget.TextView', - label: 'AppStack.tsx (42:7)', - rect: { x: 28, y: 1304, width: 1025, height: 44 }, - depth: 8, - }, - { - index: 3, - ref: 'e3', - type: 'android.view.ViewGroup', - label: 'Dismiss', - rect: { x: 0, y: 2142, width: 540, height: 132 }, - depth: 6, - }, - { - index: 4, - ref: 'e4', - type: 'android.view.ViewGroup', - label: 'Minimize', - rect: { x: 540, y: 2142, width: 540, height: 132 }, - depth: 6, - }, - ], - }; -} diff --git a/src/compat/maestro/__tests__/swipe-flow.test.ts b/src/compat/maestro/__tests__/swipe-flow.test.ts deleted file mode 100644 index 2b65d43c7..000000000 --- a/src/compat/maestro/__tests__/swipe-flow.test.ts +++ /dev/null @@ -1,46 +0,0 @@ -import assert from 'node:assert/strict'; -import { test } from 'vitest'; -import { parseMaestroReplayFlow } from '../replay-flow.ts'; - -test('coordinate endpoints take precedence over from like Maestro', () => { - const parsed = parseMaestroReplayFlow(`appId: com.pagerviewexample ---- -- swipe: - from: - id: pager-view - start: 90%, 50% - end: 10%, 50% - duration: 100 -`); - - assert.deepEqual( - parsed.actions.map((entry) => [entry.command, entry.positionals]), - [['__maestroSwipeScreen', ['percent', '90', '50', '10', '50', '100']]], - ); -}); - -test('coordinate swipes require both endpoints even when from is present', () => { - assert.throws( - () => - parseMaestroReplayFlow(`--- -- swipe: - from: - id: pager-view - start: 90%, 50% -`), - /both start and end coordinates/i, - ); -}); - -test('coordinate swipes reject direction like Maestro', () => { - assert.throws( - () => - parseMaestroReplayFlow(`--- -- swipe: - direction: LEFT - start: 90%, 50% - end: 10%, 50% -`), - /cannot combine direction with start\/end coordinates/i, - ); -}); diff --git a/src/compat/maestro/command-mapper.ts b/src/compat/maestro/command-mapper.ts deleted file mode 100644 index df4974c71..000000000 --- a/src/compat/maestro/command-mapper.ts +++ /dev/null @@ -1,201 +0,0 @@ -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { convertLaunchApp, convertStopApp } from './device-actions.ts'; -import { - convertDoubleTapOn, - convertEraseText, - convertExtendedWaitUntil, - convertLongPressOn, - convertPressKey, - convertScroll, - convertScrollUntilVisible, - convertSwipe, - convertTapOn, - maestroSelector, - readInputText, -} from './interactions.ts'; -import { - action, - assertOnlyKeys, - isPlainRecord, - readTimeoutMs, - requireAppId, - requireStringValue, - resolveMaestroString, - unsupportedCommand, -} from './support.ts'; -import { convertRepeat, convertRetry, convertRunFlow } from './flow-control.ts'; -import { convertRunScript } from './run-script.ts'; -import { MAESTRO_RUNTIME_COMMAND } from './runtime-commands.ts'; -import type { - MaestroCommand, - MaestroCommandMapperDeps, - MaestroConvertedActions, - MaestroFlowConfig, - MaestroParseContext, -} from './types.ts'; - -// Handlers that cannot produce foreign-file actions return bare actions; -// toConvertedActions normalizes them to the pipeline's uniform shape. -type MaestroCommandHandler = (params: { - value: unknown; - config: MaestroFlowConfig; - context: MaestroParseContext; - deps: MaestroCommandMapperDeps; - name: string; -}) => SessionAction[] | MaestroConvertedActions; - -function toConvertedActions( - result: SessionAction[] | MaestroConvertedActions, -): MaestroConvertedActions { - if (Array.isArray(result)) { - return { actions: result, sources: result.map(() => undefined) }; - } - return result; -} - -const MAP_COMMAND_HANDLERS: Record = { - launchApp: ({ value, config, context }) => [convertLaunchApp(value, config, context)], - tapOn: ({ value, context }) => [convertTapOn(value, context)], - doubleTapOn: ({ value, context }) => [convertDoubleTapOn(value, context)], - longPressOn: ({ value, context }) => [convertLongPressOn(value, context)], - inputText: ({ value, context }) => [ - action('type', [resolveMaestroString(readInputText(value), context)]), - ], - eraseText: ({ value }) => [convertEraseText(value)], - pasteText: ({ value, context, name }) => [ - action('type', [resolveMaestroString(requireStringValue(name, value), context)]), - ], - openLink: ({ value, config, context, name }) => [convertOpenLink(value, config, context, name)], - assertVisible: ({ value, context, name }) => [ - action(MAESTRO_RUNTIME_COMMAND.assertVisible, [ - maestroSelector(value, name, [], context), - '17000', - ]), - ], - assertNotVisible: ({ value, context, name }) => [ - action(MAESTRO_RUNTIME_COMMAND.assertNotVisible, [maestroSelector(value, name, [], context)]), - ], - extendedWaitUntil: ({ value, context }) => convertExtendedWaitUntil(value, context), - takeScreenshot: ({ value, context, name }) => [ - action('screenshot', [resolveMaestroString(requireStringValue(name, value), context)]), - ], - scroll: ({ value }) => [convertScroll(value)], - scrollUntilVisible: ({ value, context }) => convertScrollUntilVisible(value, context), - swipe: ({ value, context }) => [convertSwipe(value, context)], - hideKeyboard: () => [action('keyboard', ['dismiss'])], - pressKey: ({ value }) => [convertPressKey(value)], - back: () => [action('back')], - waitForAnimationToEnd: ({ value }) => [ - action(MAESTRO_RUNTIME_COMMAND.waitForAnimationToEnd, [String(readTimeoutMs(value, 15000))]), - ], - stopApp: ({ value, config, context }) => [convertStopApp(value, config, context)], - runScript: ({ value, context }) => [convertRunScript(value, context)], - runFlow: ({ value, config, context, deps }) => - convertRunFlow(value, config, context, deps, convertCommandList), - repeat: ({ value, config, context, deps }) => - convertRepeat(value, config, context, deps, convertCommandList), - retry: ({ value, config, context, deps }) => - convertRetry(value, config, context, deps, convertCommandList), -}; - -const SCALAR_COMMAND_HANDLERS: Record< - string, - (config: MaestroFlowConfig, context: MaestroParseContext) => SessionAction[] -> = { - launchApp: (config, context) => [convertLaunchApp(undefined, config, context)], - scroll: () => [action('scroll', ['down'])], - hideKeyboard: () => [action('keyboard', ['dismiss'])], - eraseText: () => [convertEraseText(undefined)], - back: () => [action('back')], - waitForAnimationToEnd: () => [action(MAESTRO_RUNTIME_COMMAND.waitForAnimationToEnd, ['15000'])], - stopApp: (config, context) => [convertStopApp(undefined, config, context)], -}; - -export function convertMaestroCommandWithLine( - command: MaestroCommand, - config: MaestroFlowConfig, - line: number, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, -): MaestroConvertedActions { - try { - return convertMaestroCommand(command, config, context, deps); - } catch (error) { - if (error instanceof AppError && !/\bline \d+\b/.test(error.message)) { - throw new AppError(error.code, `${error.message} (line ${line})`, error.details); - } - throw error; - } -} - -function convertMaestroCommand( - command: MaestroCommand, - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, -): MaestroConvertedActions { - if (typeof command === 'string') { - return toConvertedActions(convertScalarCommand(command, config, context)); - } - - const entries = Object.entries(command); - if (entries.length !== 1) { - throw new AppError('INVALID_ARGS', 'Maestro command maps must contain exactly one command.'); - } - - const [name, value] = entries[0] as [string, unknown]; - const handler = MAP_COMMAND_HANDLERS[name]; - if (!handler) return toConvertedActions(unsupportedCommand(name)); - return toConvertedActions(handler({ value, config, context, deps, name })); -} - -function convertScalarCommand( - command: string, - config: MaestroFlowConfig, - context: MaestroParseContext, -): SessionAction[] { - const handler = SCALAR_COMMAND_HANDLERS[command]; - if (!handler) return unsupportedCommand(command); - return handler(config, context); -} - -function convertOpenLink( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, - name: string, -): SessionAction { - const rawLink = readOpenLink(value, name); - const url = resolveMaestroString(rawLink, context); - if ((context.platform === 'ios' || context.platform === 'android') && config.appId) { - return action( - 'open', - [resolveMaestroString(requireAppId(config, name), context), url], - context.platform === 'ios' ? { maestro: { prewarmRunnerBeforeOpen: true } } : undefined, - ); - } - return action('open', [url]); -} - -function readOpenLink(value: unknown, name: string): string { - if (typeof value === 'string') return value; - if (!isPlainRecord(value)) return requireStringValue(name, value); - assertOnlyKeys(value, name, ['link']); - return requireStringValue(`${name}.link`, value.link); -} - -function convertCommandList( - commands: MaestroCommand[], - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, -): MaestroConvertedActions { - const converted = commands.map((command, index) => - convertMaestroCommandWithLine(command, config, index + 1, context, deps), - ); - return { - actions: converted.flatMap((entry) => entry.actions), - sources: converted.flatMap((entry) => entry.sources), - }; -} diff --git a/src/compat/maestro/daemon-runtime-port-observation.ts b/src/compat/maestro/daemon-runtime-port-observation.ts new file mode 100644 index 000000000..d84037526 --- /dev/null +++ b/src/compat/maestro/daemon-runtime-port-observation.ts @@ -0,0 +1,456 @@ +import { createRequestCanceledError } from '../../request/cancel.ts'; +import { + getSnapshotReferenceFrame, + type TouchReferenceFrame, +} from '../../daemon/touch-reference-frame.ts'; +import { AppError } from '../../kernel/errors.ts'; +import type { Rect, SnapshotState } from '../../kernel/snapshot.ts'; +import type { MaestroObservation, MaestroObservationCondition } from './engine-types.ts'; +import type { MaestroPlatform, MaestroSelector } from './program-ir.ts'; +import { + resolveMaestroTargetFromSnapshot, + type MaestroMatchResolutionOptions, + type MaestroPreferredContext, + type MaestroTargetQuery as SnapshotTargetQuery, +} from './runtime-targets.ts'; +import type { + MaestroDispatchSelector, + MaestroRuntimeOperationContext, + MaestroTargetMatch, + MaestroTargetQuery, +} from './runtime-port-types.ts'; + +export const MAESTRO_OBSERVATION_POLL_MS = 250; +// Snapshot transport readiness is distinct from selector matching. A launched app may own the +// foreground activity before it exposes an accessibility surface, especially during a cold start. +export const MAESTRO_INITIAL_SNAPSHOT_READY_TIMEOUT_MS = 30_000; +const MAESTRO_STABILITY_POLL_MS = 200; + +export type DaemonMaestroRuntimeDependencies = { + readonly now: () => number; + readonly sleep: (milliseconds: number, signal?: AbortSignal) => Promise; +}; + +export type MaestroSnapshotReader = ( + context: MaestroRuntimeOperationContext, +) => Promise; + +export type MaestroSnapshotSource = { + readonly capture: MaestroSnapshotReader; + readonly bindObservation: (observation: MaestroObservation) => void; + readonly reuseObservation: (context: MaestroRuntimeOperationContext) => SnapshotState | undefined; + readonly invalidate: () => void; +}; + +export type MaestroTargetResolutionMode = 'tap' | 'swipe' | 'observe'; + +export async function captureRetriableMaestroSnapshot( + params: { + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: MaestroSnapshotReader; + readonly dependencies: DaemonMaestroRuntimeDependencies; + }, + deadline: number, +): Promise { + while (true) { + throwIfAborted(params.context.signal); + try { + return await params.snapshot(params.context); + } catch (error) { + if (!isRetriableSnapshotError(error)) throw error; + throwIfAborted(params.context.signal); + const remaining = deadline - params.dependencies.now(); + if (remaining <= 0) throw error; + await sleepWithinBudget( + params.dependencies, + Math.min(MAESTRO_OBSERVATION_POLL_MS, remaining), + params.context.signal, + ); + } + } +} + +export async function resolveTypedMaestroTarget(params: { + readonly query: MaestroTargetQuery; + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: SnapshotState; + readonly platform: Extract; + readonly preferredContext?: MaestroPreferredContext; +}): Promise { + return resolveTargetFromSnapshot({ + ...params, + mode: params.query.purpose === 'swipe' ? 'swipe' : 'tap', + }); +} + +export function resolveTypedMaestroPreferredContext(params: { + readonly selector: MaestroSelector; + readonly snapshot: SnapshotState; + readonly platform: Extract; +}): MaestroPreferredContext | undefined { + const resolution = resolveMaestroTargetFromSnapshot( + params.snapshot, + { selector: params.selector }, + params.platform, + getSnapshotReferenceFrame(params.snapshot), + resolutionOptions('observe', undefined), + ); + return resolution.ok ? { node: resolution.node, rect: resolution.rect } : undefined; +} + +function resolveTargetFromSnapshot(params: { + readonly query: SnapshotTargetQuery & { readonly allowAtomicSelectorDispatch?: boolean }; + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: SnapshotState; + readonly platform: Extract; + readonly mode: MaestroTargetResolutionMode; + readonly preferredContext?: MaestroPreferredContext; +}): MaestroTargetMatch { + const frame = getSnapshotReferenceFrame(params.snapshot); + const resolution = resolveMaestroTargetFromSnapshot( + params.snapshot, + params.query, + params.platform, + frame, + resolutionOptions(params.mode, params.preferredContext), + ); + return targetMatchFromResolution( + resolution, + params.context.generation, + frame, + params.query, + params.snapshot, + params.platform, + params.mode, + ); +} + +export async function observeTypedMaestroCondition(params: { + readonly condition: MaestroObservationCondition; + readonly timeoutMs: number; + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: MaestroSnapshotReader; + readonly dependencies: DaemonMaestroRuntimeDependencies; + readonly platform: Extract; +}): Promise { + validateTimeout(params.timeoutMs, 'observation'); + let lastMatch: MaestroTargetMatch | undefined; + const initialSnapshotDeadline = + params.dependencies.now() + MAESTRO_INITIAL_SNAPSHOT_READY_TIMEOUT_MS; + let conditionDeadline: number | undefined; + + while (true) { + throwIfAborted(params.context.signal); + const captureStartedAt = params.dependencies.now(); + const snapshot = await captureRetriableMaestroSnapshot( + params, + conditionDeadline ?? initialSnapshotDeadline, + ); + conditionDeadline ??= params.dependencies.now() + params.timeoutMs; + const match = resolveTargetFromSnapshot({ + query: { selector: params.condition.selector }, + context: params.context, + snapshot, + platform: params.platform, + mode: 'observe', + }); + lastMatch = match; + if (conditionMatches(params.condition, match)) return match; + if (captureStartedAt >= conditionDeadline) break; + + const remaining = conditionDeadline - params.dependencies.now(); + if (remaining > 0) { + await sleepWithinBudget( + params.dependencies, + Math.min(MAESTRO_OBSERVATION_POLL_MS, remaining), + params.context.signal, + ); + } + } + + throwIfAborted(params.context.signal); + return lastMatch ?? unreachableObservationResult(params.context.generation); +} + +export async function scrollUntilTypedMaestroTarget(params: { + readonly selector: MaestroSelector; + readonly direction: string; + readonly timeoutMs: number; + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: MaestroSnapshotReader; + readonly scroll: () => Promise; + readonly dependencies: DaemonMaestroRuntimeDependencies; + readonly platform: Extract; +}): Promise { + validateTimeout(params.timeoutMs, 'scrollUntilVisible'); + const deadline = params.dependencies.now() + params.timeoutMs; + let lastMatch: MaestroTargetMatch | undefined; + + while (true) { + throwIfAborted(params.context.signal); + const captureStartedAt = params.dependencies.now(); + const snapshot = await captureRetriableMaestroSnapshot(params, deadline); + lastMatch = resolveTargetFromSnapshot({ + query: { selector: params.selector }, + context: params.context, + snapshot, + platform: params.platform, + mode: 'observe', + }); + if (lastMatch.matched && lastMatch.visible) return lastMatch; + if (captureStartedAt >= deadline) break; + + const remaining = deadline - params.dependencies.now(); + if (remaining > 0) { + await params.scroll(); + const afterScroll = deadline - params.dependencies.now(); + if (afterScroll > 0) { + await sleepWithinBudget( + params.dependencies, + Math.min(MAESTRO_OBSERVATION_POLL_MS, afterScroll), + params.context.signal, + ); + } + } + } + + throwIfAborted(params.context.signal); + return lastMatch ?? unreachableObservationResult(params.context.generation); +} + +export async function waitForTypedSnapshotStability(params: { + readonly timeoutMs: number; + readonly context: MaestroRuntimeOperationContext; + readonly snapshot: MaestroSnapshotReader; + readonly dependencies: DaemonMaestroRuntimeDependencies; +}): Promise { + validateTimeout(params.timeoutMs, 'waitForAnimationToEnd'); + const deadline = params.dependencies.now() + params.timeoutMs; + let previousSignature: string | undefined; + + while (true) { + throwIfAborted(params.context.signal); + const captureStartedAt = params.dependencies.now(); + const snapshot = await captureRetriableMaestroSnapshot(params, deadline); + const signature = snapshotStabilitySignature(snapshot); + if (signature === previousSignature) return; + const hadPreviousSignature = previousSignature !== undefined; + previousSignature = signature; + if (captureStartedAt >= deadline) return; + + const remaining = deadline - params.dependencies.now(); + if (hadPreviousSignature && remaining > 0) { + await sleepWithinBudget( + params.dependencies, + Math.min(MAESTRO_STABILITY_POLL_MS, remaining), + params.context.signal, + ); + } + } +} + +export function snapshotViewportRect(frame: TouchReferenceFrame | undefined): Rect | undefined { + return frame + ? { x: 0, y: 0, width: frame.referenceWidth, height: frame.referenceHeight } + : undefined; +} + +function resolutionOptions( + mode: MaestroTargetResolutionMode, + preferredContext: MaestroPreferredContext | undefined, +): MaestroMatchResolutionOptions { + return { + promoteTapTarget: mode === 'tap', + requireOnScreen: true, + ...(preferredContext ? { preferredContext } : {}), + }; +} + +function targetMatchFromResolution( + resolution: ReturnType, + generation: number, + frame: TouchReferenceFrame | undefined, + query: SnapshotTargetQuery & { allowAtomicSelectorDispatch?: boolean }, + snapshot: SnapshotState, + platform: Extract, + mode: MaestroTargetResolutionMode, +): MaestroTargetMatch { + if (!resolution.ok) { + return { + generation, + matched: resolution.evidence.matched, + visible: resolution.evidence.visible, + candidateCount: resolution.evidence.candidateCount, + ...(resolution.evidence.ref ? { ref: resolution.evidence.ref } : {}), + ...(snapshotViewportRect(frame) ? { viewport: snapshotViewportRect(frame) } : {}), + }; + } + const dispatchSelector = resolveAtomicIosDispatchSelector({ + query, + resolution, + snapshot, + platform, + mode, + }); + return { + generation, + matched: true, + visible: true, + candidateCount: resolution.evidence.candidateCount, + rect: resolution.rect, + ...(resolution.evidence.ref ? { ref: resolution.evidence.ref } : {}), + ...(snapshotViewportRect(frame) ? { viewport: snapshotViewportRect(frame) } : {}), + ...(dispatchSelector ? { dispatchSelector } : {}), + }; +} + +function resolveAtomicIosDispatchSelector(params: { + query: SnapshotTargetQuery & { allowAtomicSelectorDispatch?: boolean }; + resolution: Extract, { ok: true }>; + snapshot: SnapshotState; + platform: Extract; + mode: MaestroTargetResolutionMode; +}): MaestroDispatchSelector | undefined { + const { query, resolution, snapshot, platform, mode } = params; + if (!allowsAtomicIosDispatch(query, platform, mode)) return undefined; + const selector = singleExactDispatchSelector(query.selector); + if (!selector) return undefined; + + const exactMatches = snapshot.nodes.filter((node) => + runnerSelectorValues(node, selector.key).some((candidate) => + equalIgnoringCase(candidate, selector.value), + ), + ); + if (exactMatches.length !== 1 || exactMatches[0]?.index !== resolution.node.index) { + return undefined; + } + return selector; +} + +function allowsAtomicIosDispatch( + query: SnapshotTargetQuery & { allowAtomicSelectorDispatch?: boolean }, + platform: Extract, + mode: MaestroTargetResolutionMode, +): boolean { + return ( + platform === 'ios' && + mode === 'tap' && + query.allowAtomicSelectorDispatch === true && + query.index === undefined && + query.childOf === undefined + ); +} + +function singleExactDispatchSelector( + selector: MaestroSelector, +): MaestroDispatchSelector | undefined { + const entries = Object.entries(selector).filter(([, value]) => value !== undefined); + if (entries.length !== 1) return undefined; + const [key, rawValue] = entries[0]!; + if (!isDispatchSelectorKey(key) || typeof rawValue !== 'string') return undefined; + const value = rawValue.trim(); + return value ? { key, value } : undefined; +} + +function isDispatchSelectorKey(key: string): key is MaestroDispatchSelector['key'] { + return key === 'id' || key === 'label' || key === 'text'; +} + +function runnerSelectorValues( + node: SnapshotState['nodes'][number], + key: MaestroDispatchSelector['key'], +): string[] { + if (key === 'id') return typeof node.identifier === 'string' ? [node.identifier] : []; + if (key === 'label') return typeof node.label === 'string' ? [node.label] : []; + return [node.label, node.identifier, node.value].filter( + (value): value is string => typeof value === 'string', + ); +} + +function equalIgnoringCase(left: string, right: string): boolean { + return left.toLocaleLowerCase() === right.toLocaleLowerCase(); +} + +function conditionMatches( + condition: MaestroObservationCondition, + match: MaestroTargetMatch, +): boolean { + return condition.kind === 'visible' + ? match.matched && match.visible + : !match.matched || !match.visible; +} + +async function sleepWithinBudget( + dependencies: DaemonMaestroRuntimeDependencies, + milliseconds: number, + signal: AbortSignal | undefined, +): Promise { + try { + await dependencies.sleep(milliseconds, signal); + } catch (error) { + if (signal?.aborted) throw createRequestCanceledError(); + throw error; + } + throwIfAborted(signal); +} + +function throwIfAborted(signal: AbortSignal | undefined): void { + if (signal?.aborted) throw createRequestCanceledError(); +} + +function validateTimeout(timeoutMs: number, command: string): void { + if (!Number.isFinite(timeoutMs) || timeoutMs < 0) { + throw new AppError('INVALID_ARGS', `${command} timeout must be a non-negative number.`); + } +} + +function isRetriableSnapshotError(error: unknown): boolean { + return error instanceof AppError && error.details?.retriable === true; +} + +function snapshotStabilitySignature(snapshot: SnapshotState): string { + return JSON.stringify( + snapshot.nodes.map((node) => ({ + index: node.index, + parentIndex: node.parentIndex, + depth: node.depth, + type: node.type, + role: node.role, + subrole: node.subrole, + identifier: node.identifier, + label: node.label, + value: node.value, + enabled: node.enabled, + selected: node.selected, + focused: node.focused, + visibleToUser: node.visibleToUser, + hittable: node.hittable, + pid: node.pid, + bundleId: node.bundleId, + appName: node.appName, + windowTitle: node.windowTitle, + surface: node.surface, + hiddenContentAbove: node.hiddenContentAbove, + hiddenContentBelow: node.hiddenContentBelow, + interactionBlocked: node.interactionBlocked, + presentationHints: node.presentationHints, + rect: node.rect + ? { + x: Math.round(node.rect.x), + y: Math.round(node.rect.y), + width: Math.round(node.rect.width), + height: Math.round(node.rect.height), + } + : undefined, + })), + ); +} + +function unreachableObservationResult(generation: number): MaestroTargetMatch { + return { + generation, + matched: false, + visible: false, + candidateCount: 0, + }; +} diff --git a/src/compat/maestro/daemon-runtime-port-support.ts b/src/compat/maestro/daemon-runtime-port-support.ts new file mode 100644 index 000000000..f166f648d --- /dev/null +++ b/src/compat/maestro/daemon-runtime-port-support.ts @@ -0,0 +1,243 @@ +import path from 'node:path'; +import type { CommandFlags } from '../../core/dispatch.ts'; +import type { + DaemonInvokeFn, + DaemonRequest, + DaemonResponse, + DaemonResponseData, +} from '../../daemon/types.ts'; +import { AppError } from '../../kernel/errors.ts'; +import type { MaestroPlatform } from './program-ir.ts'; +import type { MaestroObservation } from './engine-types.ts'; +import type { + MaestroRuntimeOperationContext, + MaestroSinglePointerGestureInput, + MaestroTargetMatch, + MaestroTargetQuery, +} from './runtime-port-types.ts'; +import type { DaemonMaestroRuntimeDependencies } from './daemon-runtime-port-observation.ts'; + +export type DaemonMaestroRuntimeBaseRequest = Omit; + +export type CreateDaemonMaestroRuntimeOperationsOptions = { + readonly baseReq: DaemonMaestroRuntimeBaseRequest; + readonly invoke: DaemonInvokeFn; + readonly dependencies: DaemonMaestroRuntimeDependencies; + readonly sourcePath?: string; + readonly platform: Extract; +}; + +export async function invokeMaestroPublicCommand( + options: CreateDaemonMaestroRuntimeOperationsOptions, + command: string, + positionals: string[], + requestOptions: { + input?: Record; + flags?: CommandFlags; + internal?: DaemonRequest['internal']; + } = {}, +): Promise { + const { input, flags, internal: requestInternal } = requestOptions; + const { + input: _baseInput, + flags: baseFlags, + internal: baseInternal, + ...baseReq + } = options.baseReq; + const effectiveFlags = flags ?? stripReplayControlFlags(baseFlags); + const effectiveInternal = + requestInternal === undefined ? baseInternal : { ...baseInternal, ...requestInternal }; + const response = await options.invoke({ + ...baseReq, + command, + positionals, + ...(input === undefined ? {} : { input }), + ...(effectiveFlags === undefined ? {} : { flags: effectiveFlags }), + ...(effectiveInternal === undefined ? {} : { internal: effectiveInternal }), + }); + if (!response.ok) throw daemonResponseError(response); + return response.data; +} + +export function flagsWith( + base: CommandFlags | undefined, + extra: Partial, +): CommandFlags | undefined { + const flags = { ...stripReplayControlFlags(base), ...extra }; + return Object.keys(flags).length > 0 ? flags : undefined; +} + +function stripReplayControlFlags(flags: CommandFlags | undefined): CommandFlags | undefined { + if (!flags) return undefined; + const nested = { ...flags }; + delete nested.saveScript; + delete nested.replayUpdate; + delete nested.replayBackend; + delete nested.replayEnv; + delete nested.replayShellEnv; + delete nested.replayFrom; + delete nested.replayPlanDigest; + delete nested.failFast; + delete nested.timeoutMs; + delete nested.retries; + delete nested.recordVideo; + delete nested.shardAll; + delete nested.shardSplit; + delete nested.shardCount; + delete nested.shardIndex; + delete nested.maestro; + return Object.keys(nested).length > 0 ? nested : undefined; +} + +export function launchArgumentValues( + value: + | { kind: 'scalar'; value: string | number | boolean } + | { kind: 'list'; values: Array } + | { kind: 'map'; values: Record } + | undefined, +): string[] { + if (!value) return []; + if (value.kind === 'scalar') return [String(value.value)]; + if (value.kind === 'list') return value.values.map(String); + return Object.entries(value.values).flatMap(([key, entry]) => [key, String(entry)]); +} + +export function publicGestureRequest( + input: MaestroSinglePointerGestureInput, + context: MaestroRuntimeOperationContext, +): PublicGestureRequest { + const endpoints = endpointGesture(input); + if (endpoints && shouldPreserveEndpoints(input, context)) { + return { command: 'swipe', input: endpoints }; + } + return input.intent === 'fling' ? publicFlingRequest(input) : publicPanRequest(input); +} + +type PublicGestureRequest = { command: 'gesture' | 'swipe'; input: Record }; + +function shouldPreserveEndpoints( + input: MaestroSinglePointerGestureInput, + context: MaestroRuntimeOperationContext, +): boolean { + const authoredKind = context.authoredSwipe?.kind; + return authoredKind === 'coordinates' || authoredKind === 'target' || input.intent === 'fling'; +} + +function publicFlingRequest( + input: Extract, +): PublicGestureRequest { + if ('preset' in input) { + return { command: 'gesture', input: { kind: 'swipe', preset: input.preset } }; + } + if ('from' in input) return { command: 'swipe', input: { from: input.from, to: input.to } }; + return { + command: 'gesture', + input: { + kind: 'fling', + direction: input.direction, + origin: input.origin, + ...(input.distance === undefined ? {} : { distance: input.distance }), + }, + }; +} + +function publicPanRequest( + input: Extract, +): PublicGestureRequest { + if ('preset' in input) { + return { + command: 'gesture', + input: { kind: 'swipe', preset: input.preset, durationMs: input.durationMs }, + }; + } + return { + command: 'gesture', + input: { + kind: 'pan', + origin: input.origin, + delta: input.delta, + ...(input.durationMs === undefined ? {} : { durationMs: input.durationMs }), + }, + }; +} + +export function observationFromMatch( + selector: MaestroTargetQuery['selector'], + match: MaestroTargetMatch, +): MaestroObservation { + return { + generation: match.generation, + matched: match.matched && match.visible, + candidateCount: match.candidateCount, + evidence: { + kind: 'selector', + selector, + visible: match.visible, + ...(match.rect ? { frame: match.rect } : {}), + candidateCount: match.candidateCount, + ...(match.ref ? { ref: match.ref } : {}), + }, + }; +} + +export function artifactPathsFromData(data: DaemonResponseData | undefined): string[] { + if (!data) return []; + const paths: string[] = []; + if (typeof data.path === 'string') paths.push(data.path); + if (Array.isArray(data.artifactPaths)) { + paths.push(...data.artifactPaths.filter((value): value is string => typeof value === 'string')); + } + if (Array.isArray(data.artifacts)) { + for (const artifact of data.artifacts) { + if (typeof artifact.localPath === 'string') paths.push(artifact.localPath); + else if (typeof artifact.path === 'string') paths.push(artifact.path); + } + } + return [...new Set(paths)]; +} + +export function resolveScriptPath( + file: string, + context: MaestroRuntimeOperationContext, + sourcePath: string | undefined, +): string { + if (path.isAbsolute(file)) return file; + const parent = context.source?.path ?? sourcePath; + if (!parent) { + throw new AppError('INVALID_ARGS', 'Maestro runScript file paths require a source path.'); + } + return path.resolve(path.dirname(parent), file); +} + +export function stringifyEnvironment( + env: Record, +): Record { + return Object.fromEntries(Object.entries(env).map(([key, value]) => [key, String(value)])); +} + +function daemonResponseError(response: Extract): AppError { + const error = response.error; + const details = { + ...(error.details ?? {}), + ...(error.hint === undefined ? {} : { hint: error.hint }), + ...(error.diagnosticId === undefined ? {} : { diagnosticId: error.diagnosticId }), + ...(error.logPath === undefined ? {} : { logPath: error.logPath }), + ...(error.retriable === undefined ? {} : { retriable: error.retriable }), + ...(error.supportedOn === undefined ? {} : { supportedOn: error.supportedOn }), + }; + return new AppError(error.code, error.message, Object.keys(details).length ? details : undefined); +} + +function endpointGesture( + input: MaestroSinglePointerGestureInput, +): Record | undefined { + if (input.intent === 'fling' && 'from' in input) return { from: input.from, to: input.to }; + if (input.intent === 'pan' && !('preset' in input)) { + return { + from: input.origin, + to: { x: input.origin.x + input.delta.x, y: input.origin.y + input.delta.y }, + ...(input.durationMs === undefined ? {} : { durationMs: input.durationMs }), + }; + } + return undefined; +} diff --git a/src/compat/maestro/daemon-runtime-port.ts b/src/compat/maestro/daemon-runtime-port.ts new file mode 100644 index 000000000..92bfc3b2d --- /dev/null +++ b/src/compat/maestro/daemon-runtime-port.ts @@ -0,0 +1,418 @@ +import type { CommandFlags } from '../../core/dispatch.ts'; +import { getSnapshotReferenceFrame } from '../../daemon/touch-reference-frame.ts'; +import { AppError, asAppError } from '../../kernel/errors.ts'; +import type { Rect, SnapshotState } from '../../kernel/snapshot.ts'; +import { executeRunScriptFile } from './run-script-execution.ts'; +import { executeMaestroRuntimeCommand } from './runtime-port-commands.ts'; +import { observeMaestroCondition } from './runtime-port-observation.ts'; +import type { + MaestroObservation, + MaestroRuntimePort, + MaestroRuntimeRequest, + MaestroRuntimeResult, +} from './engine-types.ts'; +import type { + MaestroDispatchSelector, + MaestroRuntimeOperationContext, + MaestroRuntimeOperations, + MaestroTargetMatch, + MaestroTargetQuery, +} from './runtime-port-types.ts'; +import { pointInsideRect } from '../../utils/rect-center.ts'; +import { + MAESTRO_OBSERVATION_POLL_MS, + captureRetriableMaestroSnapshot, + observeTypedMaestroCondition, + resolveTypedMaestroPreferredContext, + resolveTypedMaestroTarget, + scrollUntilTypedMaestroTarget, + snapshotViewportRect, + waitForTypedSnapshotStability, + type MaestroSnapshotReader, + type MaestroSnapshotSource, +} from './daemon-runtime-port-observation.ts'; +import { + artifactPathsFromData, + flagsWith, + invokeMaestroPublicCommand, + launchArgumentValues, + observationFromMatch, + publicGestureRequest, + resolveScriptPath, + stringifyEnvironment, + type CreateDaemonMaestroRuntimeOperationsOptions, +} from './daemon-runtime-port-support.ts'; + +// Maestro's default waitForAppToSettle loop is bounded to ten 200 ms polls. +const MAESTRO_TEXT_SETTLE_TIMEOUT_MS = 2_000; + +export type { DaemonMaestroRuntimeDependencies } from './daemon-runtime-port-observation.ts'; +export type { + CreateDaemonMaestroRuntimeOperationsOptions, + DaemonMaestroRuntimeBaseRequest, +} from './daemon-runtime-port-support.ts'; + +function createDaemonMaestroRuntimeParts(options: CreateDaemonMaestroRuntimeOperationsOptions): { + operations: MaestroRuntimeOperations; + snapshots: MaestroSnapshotSource; +} { + const snapshots = createSnapshotSource(options); + const platform = options.platform; + const invoke = ( + command: string, + positionals: string[], + requestOptions: Parameters[3] = {}, + ) => invokeMaestroPublicCommand(options, command, positionals, requestOptions); + const invokeMutation = ( + command: string, + positionals: string[], + requestOptions: Parameters[3] = {}, + ) => { + snapshots.invalidate(); + return invoke(command, positionals, requestOptions); + }; + const typeTextAndSettle = async ( + text: string, + context: MaestroRuntimeOperationContext, + ): Promise => { + await invokeMutation('type', [text]); + await waitForTypedSnapshotStability({ + timeoutMs: MAESTRO_TEXT_SETTLE_TIMEOUT_MS, + context, + snapshot: snapshots.capture, + dependencies: options.dependencies, + }); + }; + + const operations: MaestroRuntimeOperations = { + resolveTarget: async (input, context) => + await resolveDaemonMaestroTarget({ input, context, snapshots, options }), + observe: async (input, context) => + await observeTypedMaestroCondition({ + condition: input.condition, + timeoutMs: input.timeoutMs, + context, + snapshot: snapshots.capture, + dependencies: options.dependencies, + platform, + }), + resolveGestureViewport: async (context) => { + const frame = getSnapshotReferenceFrame(await snapshots.capture(context)); + if (!frame) + throw new AppError('COMMAND_FAILED', 'Unable to resolve Maestro gesture viewport.'); + return snapshotViewportRect(frame) as Rect; + }, + + launchApp: async (input, context) => { + const appId = input.appId ?? context.appId; + const launchArgs = [ + ...launchArgumentValues(input.arguments), + ...launchArgumentValues(input.launchArguments), + ]; + const clearState = input.clearState === true; + const relaunch = !clearState && input.stopApp !== false; + await invokeMutation('open', appId ? [appId] : [], { + flags: flagsWith(options.baseReq.flags, { + ...(relaunch ? { relaunch: true } : {}), + ...(clearState ? { clearAppState: true } : {}), + ...(launchArgs.length ? { launchArgs } : {}), + }), + }); + }, + stopApp: async (input, context) => { + const appId = input.appId ?? context.appId; + await invokeMutation('close', appId ? [appId] : []); + }, + openLink: async (input, context) => { + const positionals = context.appId ? [context.appId, input.link] : [input.link]; + await invokeMutation('open', positionals, { + flags: flagsWith(options.baseReq.flags, { + ...(platform === 'ios' ? { maestro: { prewarmRunnerBeforeOpen: true } } : {}), + }), + }); + }, + + tapOn: async (input, context) => + await tapTarget(options, snapshots, input.target, context, { + count: input.repeat, + intervalMs: input.delay, + postGestureStabilization: true, + }), + doubleTapOn: async (input) => { + snapshots.invalidate(); + await clickTarget(options, input.target.point, { + doubleTap: true, + ...(input.delay === undefined ? {} : { intervalMs: input.delay }), + postGestureStabilization: true, + }); + }, + longPressOn: async (input) => { + snapshots.invalidate(); + await clickTarget(options, input.target.point, { + holdMs: 3_000, + postGestureStabilization: true, + }); + }, + gesture: async (input, context) => { + const request = publicGestureRequest(input, context); + await invokeMutation(request.command, [], { + input: request.input, + ...(context.gestureViewport + ? { internal: { gestureViewport: context.gestureViewport } } + : {}), + }); + }, + inputText: async (input, context) => await typeTextAndSettle(input.text, context), + eraseText: async (input, context) => + await typeTextAndSettle('\b'.repeat(input.charactersToErase ?? 50), context), + pasteText: async (input, context) => await typeTextAndSettle(input.text, context), + scroll: async (input) => { + await invokeMutation('scroll', [input.direction]); + }, + scrollUntilVisible: async (input, context) => { + const match = await scrollUntilTypedMaestroTarget({ + selector: input.selector, + direction: input.direction, + timeoutMs: input.timeoutMs, + context, + snapshot: snapshots.capture, + dependencies: options.dependencies, + platform, + scroll: async () => { + await invokeMutation('scroll', [input.direction]); + }, + }); + if (!match.matched || !match.visible) { + throw new AppError( + 'COMMAND_FAILED', + 'Maestro scrollUntilVisible target did not become visible.', + { selector: input.selector, timeoutMs: input.timeoutMs }, + ); + } + return { observation: observationFromMatch(input.selector, match) }; + }, + pressKey: async (input) => { + if (input.key === 'back' || input.key === 'home') { + await invokeMutation(input.key, []); + return; + } + try { + await invokeMutation('keyboard', [input.key]); + } catch (error) { + if (input.key !== 'enter' && input.key !== 'return') throw error; + await invokeMutation('type', ['\n']); + } + }, + back: async () => { + await invokeMutation('back', []); + }, + hideKeyboard: async () => { + await invokeMutation('keyboard', ['dismiss']); + }, + waitForAnimationToEnd: async (input, context) => + await waitForTypedSnapshotStability({ + timeoutMs: input.timeoutMs ?? 15_000, + context, + snapshot: snapshots.capture, + dependencies: options.dependencies, + }), + takeScreenshot: async (input) => ({ + artifactPaths: artifactPathsFromData(await invoke('screenshot', [input.path])), + }), + runScript: async (input, context) => ({ + outputEnv: executeRunScriptFile({ + scriptPath: resolveScriptPath(input.file, context, options.sourcePath), + env: { + ...context.env, + ...(input.env ? stringifyEnvironment(input.env) : {}), + ...(options.baseReq.flags?.maestro?.runScriptEnv ?? {}), + }, + }), + }), + }; + return { operations, snapshots }; +} + +async function resolveDaemonMaestroTarget(params: { + input: MaestroTargetQuery & { timeoutMs: number }; + context: MaestroRuntimeOperationContext; + snapshots: MaestroSnapshotSource; + options: CreateDaemonMaestroRuntimeOperationsOptions; + allowObservationReuse?: boolean; +}): Promise { + const { input, context, snapshots, options } = params; + const deadline = options.dependencies.now() + input.timeoutMs; + let currentSnapshot = + params.allowObservationReuse === false ? undefined : snapshots.reuseObservation(context); + while (true) { + const captureStartedAt = options.dependencies.now(); + const reusedObservation = currentSnapshot !== undefined; + currentSnapshot ??= await captureRetriableMaestroSnapshot( + { context, snapshot: snapshots.capture, dependencies: options.dependencies }, + deadline, + ); + const match = await resolveTypedMaestroTarget({ + query: input, + context, + snapshot: currentSnapshot, + platform: options.platform, + preferredContext: preferredContextForObservation(context, currentSnapshot, options.platform), + }); + if ( + canUseResolvedTarget(match, options.platform, reusedObservation) || + captureStartedAt >= deadline + ) { + return match; + } + currentSnapshot = undefined; + if (!reusedObservation) await sleepBeforeTargetPoll(options, deadline, context.signal); + } +} + +function preferredContextForObservation( + context: MaestroRuntimeOperationContext, + snapshot: SnapshotState, + platform: CreateDaemonMaestroRuntimeOperationsOptions['platform'], +) { + const evidence = context.cachedObservation?.evidence; + if (!evidence?.visible) return undefined; + return resolveTypedMaestroPreferredContext({ selector: evidence.selector, snapshot, platform }); +} + +function isActionableTarget(match: MaestroTargetMatch): boolean { + return match.matched && match.visible && match.rect !== undefined; +} + +function canUseResolvedTarget( + match: MaestroTargetMatch, + platform: CreateDaemonMaestroRuntimeOperationsOptions['platform'], + reusedObservation: boolean, +): boolean { + if (!isActionableTarget(match)) return false; + return !reusedObservation || platform === 'android' || match.dispatchSelector !== undefined; +} + +async function sleepBeforeTargetPoll( + options: CreateDaemonMaestroRuntimeOperationsOptions, + deadline: number, + signal: AbortSignal | undefined, +): Promise { + const remaining = deadline - options.dependencies.now(); + if (remaining <= 0) return; + await options.dependencies.sleep(Math.min(MAESTRO_OBSERVATION_POLL_MS, remaining), signal); +} + +export function createDaemonMaestroRuntimePort( + options: CreateDaemonMaestroRuntimeOperationsOptions, +): MaestroRuntimePort { + const { operations, snapshots } = createDaemonMaestroRuntimeParts(options); + return { + execute: async (request: MaestroRuntimeRequest): Promise => + await executeMaestroRuntimeCommand(request, operations), + observe: async (request) => { + const observation = await observeMaestroCondition(request, operations); + snapshots.bindObservation(observation); + return observation; + }, + }; +} + +function createSnapshotSource( + options: CreateDaemonMaestroRuntimeOperationsOptions, +): MaestroSnapshotSource { + let cached: + | { generation: number; snapshot: SnapshotState; observation?: MaestroObservation } + | undefined; + const capture: MaestroSnapshotReader = async (context) => { + const data = await invokeMaestroPublicCommand(options, 'snapshot', [], { + flags: flagsWith(options.baseReq.flags, { noRecord: true }), + }); + if (!data || !Array.isArray(data.nodes)) { + throw new AppError('COMMAND_FAILED', 'Maestro snapshot did not return node data.'); + } + const snapshot = data as SnapshotState; + cached = { generation: context.generation, snapshot }; + return snapshot; + }; + return { + capture, + bindObservation: (observation) => { + if (cached?.generation === observation.generation) cached.observation = observation; + }, + reuseObservation: (context) => { + if (context.cachedObservation?.generation !== context.generation) return undefined; + if (cached?.generation !== context.generation) return undefined; + return cached.observation === context.cachedObservation ? cached.snapshot : undefined; + }, + invalidate: () => { + cached = undefined; + }, + }; +} + +async function tapTarget( + options: CreateDaemonMaestroRuntimeOperationsOptions, + snapshots: MaestroSnapshotSource, + target: Parameters[0]['target'], + context: MaestroRuntimeOperationContext, + flags: Partial, +): Promise { + const dispatchSelector = target.resolution?.dispatchSelector; + if (dispatchSelector) { + const resolution = target.resolution!; + snapshots.invalidate(); + try { + await clickSelector(options, dispatchSelector, flags); + return; + } catch (error) { + if (!isAtomicSelectorFallbackError(error)) throw error; + const refreshed = await resolveDaemonMaestroTarget({ + input: resolution.query, + context, + snapshots, + options, + allowObservationReuse: false, + }); + if (!isActionableTarget(refreshed)) throw error; + snapshots.invalidate(); + await clickTarget(options, pointInsideRect(refreshed.rect!), flags); + return; + } + } + snapshots.invalidate(); + await clickTarget(options, target.point, flags); +} + +async function clickSelector( + options: CreateDaemonMaestroRuntimeOperationsOptions, + selector: MaestroDispatchSelector, + flags: Partial, +): Promise { + await invokeMaestroPublicCommand( + options, + 'click', + [`${selector.key}=${JSON.stringify(selector.value)}`], + { + flags: flagsWith(options.baseReq.flags, { + ...flags, + maestro: { allowNonHittableCoordinateFallback: true }, + }), + }, + ); +} + +function isAtomicSelectorFallbackError(error: unknown): boolean { + const code = asAppError(error).code; + return code === 'AMBIGUOUS_MATCH' || code === 'ELEMENT_NOT_FOUND' || code === 'ELEMENT_OFFSCREEN'; +} + +async function clickTarget( + options: CreateDaemonMaestroRuntimeOperationsOptions, + point: { x: number; y: number } | undefined, + flags: Partial, +): Promise { + if (!point) throw new AppError('COMMAND_FAILED', 'Maestro target did not resolve to a point.'); + await invokeMaestroPublicCommand(options, 'click', [String(point.x), String(point.y)], { + flags: flagsWith(options.baseReq.flags, flags), + }); +} diff --git a/src/compat/maestro/device-actions.ts b/src/compat/maestro/device-actions.ts deleted file mode 100644 index d5d9de66e..000000000 --- a/src/compat/maestro/device-actions.ts +++ /dev/null @@ -1,98 +0,0 @@ -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { - action, - assertOnlyKeys, - isPlainRecord, - requireAppId, - resolveMaestroString, - unsupportedMaestroSyntax, -} from './support.ts'; -import type { MaestroFlowConfig, MaestroParseContext } from './types.ts'; - -export function convertLaunchApp( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, -): SessionAction { - if (value === null || value === undefined) { - return action('open', [resolveMaestroString(requireAppId(config, 'launchApp'), context)], { - relaunch: true, - }); - } - if (typeof value === 'string') { - return action('open', [resolveMaestroString(value, context)], { relaunch: true }); - } - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'launchApp expects a string or map.'); - } - assertOnlyKeys(value, 'launchApp', [ - 'appId', - 'stopApp', - 'clearState', - 'clearKeychain', - 'arguments', - 'permissions', - 'launchArguments', - ]); - rejectUnsupportedLaunchOption(value, 'permissions'); - rejectUnsupportedLaunchOption(value, 'clearKeychain'); - const appId = resolveMaestroString( - typeof value.appId === 'string' ? value.appId : requireAppId(config, 'launchApp'), - context, - ); - const launchArgs = readLaunchArgs(value, context); - const shouldClearState = value.clearState === true; - const shouldRelaunch = !shouldClearState && value.stopApp !== false; - return action('open', [appId], { - ...(shouldRelaunch ? { relaunch: true } : {}), - ...(shouldClearState ? { clearAppState: true } : {}), - ...(launchArgs.length > 0 ? { launchArgs } : {}), - }); -} - -export function convertStopApp( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, -): SessionAction { - if (value === null || value === undefined) { - return action('close', [resolveMaestroString(requireAppId(config, 'stopApp'), context)]); - } - if (typeof value === 'string') return action('close', [resolveMaestroString(value, context)]); - throw new AppError('INVALID_ARGS', 'stopApp expects a string appId or no value.'); -} - -function readLaunchArgs(value: Record, context: MaestroParseContext): string[] { - return [ - ...readLaunchArgValue(value.arguments, 'launchApp.arguments', context), - ...readLaunchArgValue(value.launchArguments, 'launchApp.launchArguments', context), - ]; -} - -function readLaunchArgValue(value: unknown, name: string, context: MaestroParseContext): string[] { - if (value === undefined || value === null) return []; - if (typeof value === 'string') return [resolveMaestroString(value, context)]; - if (Array.isArray(value)) { - return value.map((entry, index) => readLaunchArgScalar(entry, `${name}[${index}]`, context)); - } - if (isPlainRecord(value)) { - return Object.entries(value).flatMap(([key, entry]) => [ - resolveMaestroString(key, context), - readLaunchArgScalar(entry, `${name}.${key}`, context), - ]); - } - throw new AppError('INVALID_ARGS', `${name} expects a string, list, or map.`); -} - -function readLaunchArgScalar(value: unknown, name: string, context: MaestroParseContext): string { - if (typeof value === 'string') return resolveMaestroString(value, context); - if (typeof value === 'number' || typeof value === 'boolean') return String(value); - throw new AppError('INVALID_ARGS', `${name} must be a string, number, or boolean.`); -} - -function rejectUnsupportedLaunchOption(value: Record, key: string): void { - if (value[key] !== undefined) { - throw unsupportedMaestroSyntax(`Maestro launchApp field "${key}" is not supported yet.`); - } -} diff --git a/src/compat/maestro/engine-context.ts b/src/compat/maestro/engine-context.ts new file mode 100644 index 000000000..7dbfe140b --- /dev/null +++ b/src/compat/maestro/engine-context.ts @@ -0,0 +1,111 @@ +import { AppError } from '../../kernel/errors.ts'; +import type { MaestroObservation } from './engine-types.ts'; + +export type MaestroExecutionContext = ReturnType; + +export function createMaestroExecutionContext( + defaults: Record = {}, + runtimeOverrides: Record = {}, +) { + const overrides = { ...runtimeOverrides }; + // Flow config and runFlow env values are stack-scoped; script output variables persist. + let persistentValues = stringifyValues(defaults); + const scopes: Record[] = []; + const expandedNames = new Set(); + let generation = 0; + let observation: MaestroObservation | undefined; + + return { + get values(): Readonly> { + return currentValues(); + }, + get generation(): number { + return generation; + }, + get observation(): MaestroObservation | undefined { + return observation?.generation === generation ? observation : undefined; + }, + get expandedVariables(): Readonly> { + const values = currentValues(); + return Object.fromEntries( + [...expandedNames] + .filter((name) => Object.hasOwn(values, name)) + .map((name) => [name, values[name]!] as const), + ); + }, + enter(scopedValues: Record = {}): () => void { + const resolved = resolveScopedValues(scopedValues); + scopes.push(resolved); + return () => { + const current = scopes.pop(); + if (current !== resolved) { + throw new AppError( + 'COMMAND_FAILED', + 'Maestro environment scopes were left out of order.', + ); + } + }; + }, + merge(output: Record): void { + persistentValues = { ...persistentValues, ...output }; + }, + recordObservation(next: MaestroObservation): void { + if (next.generation !== generation) { + throw new AppError( + 'COMMAND_FAILED', + `Maestro observation generation ${next.generation} does not match ${generation}.`, + ); + } + observation = next; + }, + recordMutation(): void { + generation += 1; + observation = undefined; + }, + resolve(value: string): string { + return resolveValue(value, currentValues(), (name) => expandedNames.add(name)); + }, + }; + + function currentValues(): Record { + const scoped = scopes.reduce((values, scope) => ({ ...values, ...scope }), { + ...persistentValues, + }); + return { ...scoped, ...overrides }; + } + + function resolveScopedValues( + scopedValues: Record, + ): Record { + const rawValues = stringifyValues(scopedValues); + const resolved: Record = {}; + for (const [key, value] of Object.entries(rawValues)) { + resolved[key] = resolveValue(value, { + ...currentValues(), + ...rawValues, + ...resolved, + ...overrides, + }); + } + return resolved; + } +} + +function stringifyValues( + values: Record, +): Record { + return Object.fromEntries(Object.entries(values).map(([key, value]) => [key, String(value)])); +} + +function resolveValue( + value: string, + values: Readonly>, + onExpanded?: (name: string) => void, + resolving = new Set(), +): string { + return value.replace(/\$\{([A-Za-z_][A-Za-z0-9_.]*)\}/g, (match, key: string) => { + if (!Object.hasOwn(values, key) || resolving.has(key)) return match; + onExpanded?.(key); + return resolveValue(values[key]!, values, onExpanded, new Set([...resolving, key])); + }); +} diff --git a/src/compat/maestro/engine-expression.ts b/src/compat/maestro/engine-expression.ts new file mode 100644 index 000000000..07cb647f2 --- /dev/null +++ b/src/compat/maestro/engine-expression.ts @@ -0,0 +1,207 @@ +import { AppError } from '../../kernel/errors.ts'; +import type { MaestroExecutionContext } from './engine-context.ts'; +import type { MaestroPlatform } from './program-ir.ts'; + +export function evaluateMaestroBooleanExpression( + value: string, + context: MaestroExecutionContext, + platform: MaestroPlatform | undefined, +): boolean { + const resolved = unwrapMaestroExpression(context.resolve(value)); + return new MaestroBooleanExpressionParser( + tokenizeMaestroBooleanExpression(resolved), + platform, + ).parse(); +} + +function unwrapMaestroExpression(value: string): string { + const trimmed = value.trim(); + return trimmed.startsWith('${') && trimmed.endsWith('}') ? trimmed.slice(2, -1).trim() : trimmed; +} + +type MaestroBooleanToken = + | { type: 'platform' } + | { type: 'operator'; value: '==' | '!=' | '&&' | '||' } + | { type: 'paren'; value: '(' | ')' } + | { type: 'string'; value: string } + | { type: 'boolean'; value: boolean }; + +function tokenizeMaestroBooleanExpression(expression: string): MaestroBooleanToken[] { + const tokens: MaestroBooleanToken[] = []; + let index = 0; + while (index < expression.length) { + const remaining = expression.slice(index); + const whitespace = /^\s+/.exec(remaining)?.[0].length ?? 0; + if (whitespace > 0) { + index += whitespace; + continue; + } + const token = readMaestroBooleanToken(remaining); + if (!token) { + throw new AppError( + 'INVALID_ARGS', + `Unsupported runFlow.when.true expression near "${remaining.slice(0, 24)}".`, + ); + } + tokens.push(token.value); + index += token.length; + } + return tokens; +} + +function readMaestroBooleanToken( + remaining: string, +): { value: MaestroBooleanToken; length: number } | null { + return ( + readPlatformToken(remaining) ?? + readOperatorToken(remaining) ?? + readParenToken(remaining) ?? + readQuotedToken(remaining) ?? + readBooleanToken(remaining) + ); +} + +type ReadToken = { value: MaestroBooleanToken; length: number }; + +function readPlatformToken(remaining: string): ReadToken | null { + const platform = 'maestro.platform'; + return remaining.startsWith(platform) + ? { value: { type: 'platform' }, length: platform.length } + : null; +} + +function readOperatorToken(remaining: string): ReadToken | null { + const operator = /^(==|!=|&&|\|\|)/.exec(remaining)?.[1]; + if (!operator) return null; + return { + value: { + type: 'operator', + value: operator as Extract['value'], + }, + length: operator.length, + }; +} + +function readParenToken(remaining: string): ReadToken | null { + const paren = remaining[0]; + return paren === '(' || paren === ')' + ? { value: { type: 'paren', value: paren }, length: 1 } + : null; +} + +function readQuotedToken(remaining: string): ReadToken | null { + const quoted = /^(['"])(.*?)\1/.exec(remaining); + return quoted + ? { value: { type: 'string', value: quoted[2] ?? '' }, length: quoted[0].length } + : null; +} + +function readBooleanToken(remaining: string): ReadToken | null { + const boolean = /^(true|false)\b/.exec(remaining)?.[1]; + return boolean + ? { value: { type: 'boolean', value: boolean === 'true' }, length: boolean.length } + : null; +} + +class MaestroBooleanExpressionParser { + private index = 0; + private readonly tokens: MaestroBooleanToken[]; + private readonly platform: MaestroPlatform | undefined; + + constructor(tokens: MaestroBooleanToken[], platform: MaestroPlatform | undefined) { + this.tokens = tokens; + this.platform = platform; + } + + parse(): boolean { + const result = this.parseOr(); + if (this.peek()) { + throw new AppError('INVALID_ARGS', 'Unsupported trailing runFlow.when.true expression.'); + } + return result; + } + + private parseOr(): boolean { + let result = this.parseAnd(); + while (this.consumeOperator('||')) result = this.parseAnd() || result; + return result; + } + + private parseAnd(): boolean { + let result = this.parsePrimary(); + while (this.consumeOperator('&&')) result = this.parsePrimary() && result; + return result; + } + + private parsePrimary(): boolean { + const token = this.peek(); + if (!token) throw new AppError('INVALID_ARGS', 'Incomplete runFlow.when.true expression.'); + if (token.type === 'boolean') { + this.index += 1; + return token.value; + } + if (token.type === 'paren' && token.value === '(') { + this.index += 1; + const result = this.parseOr(); + if (!this.consumeParen(')')) { + throw new AppError('INVALID_ARGS', 'Unclosed runFlow.when.true parenthesis.'); + } + return result; + } + return this.parsePlatformComparison(); + } + + private parsePlatformComparison(): boolean { + this.consumePlatformToken(); + const operator = this.consumeComparisonOperator(); + const expectedPlatform = this.consumeStringLiteral(); + const matches = this.platform === expectedPlatform.toLowerCase(); + return operator.value === '==' ? matches : !matches; + } + + private consumePlatformToken(): void { + if (this.peek()?.type !== 'platform') { + throw new AppError( + 'INVALID_ARGS', + 'runFlow.when.true supports maestro.platform comparisons.', + ); + } + this.index += 1; + } + + private consumeComparisonOperator(): Extract { + const token = this.peek(); + if (token?.type !== 'operator' || (token.value !== '==' && token.value !== '!=')) { + throw new AppError('INVALID_ARGS', 'runFlow.when.true comparison requires == or !=.'); + } + this.index += 1; + return token; + } + + private consumeStringLiteral(): string { + const token = this.peek(); + if (token?.type !== 'string') { + throw new AppError('INVALID_ARGS', 'runFlow.when.true comparison requires a string literal.'); + } + this.index += 1; + return token.value; + } + + private consumeOperator(value: '&&' | '||'): boolean { + const token = this.peek(); + if (token?.type !== 'operator' || token.value !== value) return false; + this.index += 1; + return true; + } + + private consumeParen(value: '(' | ')'): boolean { + const token = this.peek(); + if (token?.type !== 'paren' || token.value !== value) return false; + this.index += 1; + return true; + } + + private peek(): MaestroBooleanToken | undefined { + return this.tokens[this.index]; + } +} diff --git a/src/compat/maestro/engine-flow.ts b/src/compat/maestro/engine-flow.ts new file mode 100644 index 000000000..05a0669b6 --- /dev/null +++ b/src/compat/maestro/engine-flow.ts @@ -0,0 +1,170 @@ +import path from 'node:path'; +import { AppError } from '../../kernel/errors.ts'; +import { createRequestCanceledError } from '../../request/cancel.ts'; +import type { + MaestroCommand, + MaestroProgram, + MaestroRunFlowCommand, + MaestroRunFlowCondition, +} from './program-ir.ts'; +import type { MaestroExecutionContext } from './engine-context.ts'; +import { evaluateMaestroBooleanExpression } from './engine-expression.ts'; +import { + DEFAULT_MAESTRO_COMPATIBILITY_TIMING_POLICY, + type MaestroCompatibilityTimingPolicy, + type MaestroEngineOptions, + type MaestroObservationCondition, +} from './engine-types.ts'; + +const MAX_MAESTRO_REPEAT_EXPANSIONS = 1000; + +export function resolveCommand( + command: T, + context: MaestroExecutionContext, +): T { + return { + ...resolveValue(command, context), + source: command.source, + }; +} + +export function readIterationCount( + value: number | string | undefined, + fallback: number, + context: MaestroExecutionContext, + name: string, +): number { + const resolved = value === undefined ? fallback : Number(context.resolve(String(value))); + if (!Number.isInteger(resolved) || resolved < 0) { + throw new AppError('INVALID_ARGS', `Maestro ${name} must resolve to a non-negative integer.`); + } + return resolved; +} + +export function assertMaestroRepeatExpansionLimit(times: number): void { + if (times > MAX_MAESTRO_REPEAT_EXPANSIONS) { + throw new AppError( + 'INVALID_ARGS', + `repeat.times must be <= ${MAX_MAESTRO_REPEAT_EXPANSIONS} for deterministic replay expansion.`, + ); + } +} + +export function resolveMaestroTimingPolicy( + overrides: Partial = {}, +): MaestroCompatibilityTimingPolicy { + const policy = { ...DEFAULT_MAESTRO_COMPATIBILITY_TIMING_POLICY, ...overrides }; + for (const [name, value] of Object.entries(policy)) { + if (!Number.isInteger(value) || value < 0) { + throw new AppError( + 'INVALID_ARGS', + `Maestro timing policy ${name} must be a non-negative integer.`, + ); + } + } + return policy; +} + +export function checkpointMaestroCancellation(signal: AbortSignal | undefined): void { + if (signal?.aborted) throw createRequestCanceledError(); +} + +export async function readIncludedProgram( + command: MaestroRunFlowCommand, + options: MaestroEngineOptions, +): Promise { + if (command.include.kind === 'commands') { + return { + kind: 'program', + source: command.source, + config: {}, + commands: command.include.commands, + }; + } + if (!options.loadProgram) { + throw new AppError('INVALID_ARGS', 'Maestro file runFlow requires a program loader.'); + } + checkpointMaestroCancellation(options.signal); + if (options.signal) { + return await options.loadProgram(command.include.path, command.source.path, options.signal); + } + return await options.loadProgram(command.include.path, command.source.path); +} + +function includePathKey( + command: MaestroRunFlowCommand, + parentSource: string | undefined, +): string | undefined { + if (command.include.kind !== 'file') return undefined; + return path.resolve( + parentSource ? path.dirname(parentSource) : process.cwd(), + command.include.path, + ); +} + +export function sourcePathKey(source: string | undefined): string | undefined { + return source === undefined ? undefined : path.resolve(source); +} + +export function assertIncludePathAvailable( + command: MaestroRunFlowCommand, + activePaths: ReadonlySet, +): string | undefined { + const requestedPath = includePathKey(command, command.source.path); + if (requestedPath && activePaths.has(requestedPath)) { + throw new AppError('INVALID_ARGS', `Maestro runFlow cycle detected at ${requestedPath}.`); + } + return requestedPath; +} + +export function registerIncludedProgramPaths( + command: MaestroRunFlowCommand, + program: MaestroProgram, + requestedPath: string | undefined, + activePaths: Set, +): Set { + const loadedPath = + command.include.kind === 'file' ? sourcePathKey(program.source.path) : undefined; + const includedPaths = new Set( + [requestedPath, loadedPath].filter((value): value is string => value !== undefined), + ); + const repeatedPath = [...includedPaths].find((value) => activePaths.has(value)); + if (repeatedPath) { + throw new AppError('INVALID_ARGS', `Maestro runFlow cycle detected at ${repeatedPath}.`); + } + includedPaths.forEach((value) => activePaths.add(value)); + return includedPaths; +} + +export function staticConditionMatches( + condition: MaestroRunFlowCondition, + context: MaestroExecutionContext, + options: MaestroEngineOptions, +): boolean { + if (condition.platform && condition.platform !== options.platform) return false; + if (condition.true === undefined) return true; + if (typeof condition.true === 'boolean') return condition.true; + return evaluateMaestroBooleanExpression(condition.true, context, options.platform); +} + +export function observationConditions( + condition: MaestroRunFlowCondition, +): MaestroObservationCondition[] { + return [ + ...(condition.visible ? [{ kind: 'visible' as const, selector: condition.visible }] : []), + ...(condition.notVisible + ? [{ kind: 'notVisible' as const, selector: condition.notVisible }] + : []), + ]; +} + +function resolveValue(value: T, context: MaestroExecutionContext): T { + if (typeof value === 'string') return context.resolve(value) as T; + if (Array.isArray(value)) return value.map((entry) => resolveValue(entry, context)) as T; + if (value && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, entry]) => [key, resolveValue(entry, context)]), + ) as T; + } + return value; +} diff --git a/src/compat/maestro/engine-types.ts b/src/compat/maestro/engine-types.ts new file mode 100644 index 000000000..df082c159 --- /dev/null +++ b/src/compat/maestro/engine-types.ts @@ -0,0 +1,136 @@ +import type { + MaestroCommand, + MaestroPlatform, + MaestroProgram, + MaestroSelector, + MaestroSourceLocation, +} from './program-ir.ts'; + +export type MaestroControlCommand = Extract< + MaestroCommand, + { kind: 'runFlow' | 'repeat' | 'retry' } +>; + +export type MaestroRuntimeCommand = Exclude; + +export type MaestroObservationFrame = { + x: number; + y: number; + width: number; + height: number; +}; + +export type MaestroObservationEvidence = { + kind: 'selector'; + selector: MaestroSelector; + visible: boolean; + frame?: MaestroObservationFrame; + candidateCount: number; + ref?: string; +}; + +export type MaestroObservation = { + generation: number; + matched: boolean; + candidateCount?: number; + evidence?: MaestroObservationEvidence; +}; + +export type MaestroObservationCondition = + | { kind: 'visible'; selector: MaestroSelector } + | { kind: 'notVisible'; selector: MaestroSelector }; + +export type MaestroRuntimeRequest = { + command: MaestroRuntimeCommand; + env: Readonly>; + appId?: string; + generation: number; + cachedObservation?: MaestroObservation; + signal?: AbortSignal; +}; + +export type MaestroRuntimeResult = { + mutated: boolean; + observation?: MaestroObservation; + outputEnv?: Record; + artifactPaths?: string[]; +}; + +export type MaestroRuntimePort = { + execute(request: MaestroRuntimeRequest): Promise; + observe(request: { + condition: MaestroObservationCondition; + timeoutMs: number; + generation: number; + env: Readonly>; + cachedObservation?: MaestroObservation; + signal?: AbortSignal; + }): Promise; +}; + +export type MaestroCompatibilityTimingPolicy = { + assertVisibleTimeoutMs: number; + assertNotVisibleTimeoutMs: number; + extendedWaitUntilTimeoutMs: number; + runFlowConditionTimeoutMs: number; +}; + +export const DEFAULT_MAESTRO_COMPATIBILITY_TIMING_POLICY = { + assertVisibleTimeoutMs: 17_000, + assertNotVisibleTimeoutMs: 3_000, + extendedWaitUntilTimeoutMs: 10_000, + runFlowConditionTimeoutMs: 3_000, +} as const satisfies MaestroCompatibilityTimingPolicy; + +export type MaestroEngineEvent = { + command: MaestroCommand; + source: MaestroSourceLocation; + generation: number; + stepIndex: number; + stepTotal: number; +}; + +export type MaestroEngineObserver = { + commandStarted?(event: MaestroEngineEvent): void; + commandCompleted?(event: MaestroEngineEvent & { durationMs: number }): void; + commandFailed?( + event: MaestroEngineEvent & { + durationMs: number; + error: unknown; + artifactPaths: readonly string[]; + expandedVariables: Readonly>; + }, + ): void; +}; + +export type MaestroEngineOptions = { + /** Highest-precedence invocation values, normally CLI over shell. */ + env?: Readonly>; + /** Lowest-precedence defaults, normally replay built-ins. */ + defaults?: Readonly>; + /** Explicit alias for callers passing replay built-ins. */ + builtins?: Readonly>; + platform?: MaestroPlatform; + target?: string; + /** Internal zero-based plan offset. Prefer from/planDigest for resume callers. */ + startIndex?: number; + /** One-based safe resume request and its plan digest. */ + from?: number; + planDigest?: string; + loadProgram?: ( + path: string, + parentSource?: string, + signal?: AbortSignal, + ) => Promise; + timing?: Partial; + signal?: AbortSignal; + observer?: MaestroEngineObserver; + now?: () => number; +}; + +export type MaestroEngineResult = { + executed: number; + skipped: number; + generation: number; + artifactPaths: string[]; +}; diff --git a/src/compat/maestro/engine.ts b/src/compat/maestro/engine.ts new file mode 100644 index 000000000..b28d7acfc --- /dev/null +++ b/src/compat/maestro/engine.ts @@ -0,0 +1,44 @@ +import type { MaestroProgram } from './program-ir.ts'; +import { + assertMaestroReplayStartIndex, + compileMaestroReplayPlan, + resolveMaestroReplayStartIndex, +} from './replay-plan.ts'; +import { executeMaestroReplayPlan } from './replay-plan-execution.ts'; +import type { + MaestroEngineOptions, + MaestroEngineResult, + MaestroRuntimePort, +} from './engine-types.ts'; +import type { MaestroReplayPlan } from './replay-plan-types.ts'; + +export async function executeMaestroProgram( + program: MaestroProgram, + port: MaestroRuntimePort, + options: MaestroEngineOptions = {}, +): Promise { + const plan = await compileMaestroReplayPlan(program, options); + const startIndex = resolveExecutionStartIndex(plan, options); + return await executeMaestroReplayPlan(plan, port, { ...options, startIndex }); +} + +export async function executeMaestroPlan( + plan: MaestroReplayPlan, + port: MaestroRuntimePort, + options: MaestroEngineOptions = {}, +): Promise { + const startIndex = resolveExecutionStartIndex(plan, options); + return await executeMaestroReplayPlan(plan, port, { ...options, startIndex }); +} + +function resolveExecutionStartIndex( + plan: MaestroReplayPlan, + options: MaestroEngineOptions, +): number { + return options.from !== undefined || options.planDigest !== undefined + ? resolveMaestroReplayStartIndex(plan, { + from: options.from, + planDigest: options.planDigest, + }) + : assertMaestroReplayStartIndex(plan, options.startIndex ?? 0); +} diff --git a/src/compat/maestro/export-flow.ts b/src/compat/maestro/export-flow.ts index 72ef3459c..d9ed1ea3d 100644 --- a/src/compat/maestro/export-flow.ts +++ b/src/compat/maestro/export-flow.ts @@ -7,9 +7,9 @@ import { readReplayScriptMetadata, type ReplayScriptMetadata, } from '../../replay/script.ts'; -import { stringifyMaestroYamlDocuments } from './flow-yaml.ts'; -import { formatMaestroPoint } from './points.ts'; -import type { MaestroCommand, MaestroFlowConfig } from './types.ts'; +import { formatMaestroPoint } from './export-points.ts'; +import type { MaestroExportCommand, MaestroExportConfig } from './export-types.ts'; +import { stringifyMaestroYamlDocuments } from './export-yaml.ts'; export type MaestroExportWarning = { line: number; @@ -22,8 +22,6 @@ export type MaestroExportResult = { warnings: MaestroExportWarning[]; }; -type MaestroExportConfig = Pick; - type ExportContext = { config: MaestroExportConfig; warnings: MaestroExportWarning[]; @@ -31,8 +29,8 @@ type ExportContext = { }; type ConvertedAction = - | { kind: 'commands'; commands: MaestroCommand[]; warnings?: string[] } - | { kind: 'config'; appId: string; commands: MaestroCommand[]; warnings?: string[] } + | { kind: 'commands'; commands: MaestroExportCommand[]; warnings?: string[] } + | { kind: 'config'; appId: string; commands: MaestroExportCommand[]; warnings?: string[] } | { kind: 'unsupported'; message: string }; type ActionConverter = (action: SessionAction) => ConvertedAction; @@ -67,7 +65,7 @@ function exportReplayActionsToMaestro( warnings: [], unsupported: [], }; - const commands: MaestroCommand[] = []; + const commands: MaestroExportCommand[] = []; for (const [index, action] of actions.entries()) { const line = options.actionLines?.[index] ?? index + 1; @@ -155,7 +153,7 @@ function convertOpenAction(action: SessionAction): ConvertedAction { return { kind: 'config', appId: first, commands: [launchApp] }; } -function buildLaunchAppCommand(action: SessionAction, appId: string): MaestroCommand { +function buildLaunchAppCommand(action: SessionAction, appId: string): MaestroExportCommand { const options = buildLaunchAppOptions(action); return options ? { launchApp: { appId, ...options } } : 'launchApp'; } @@ -448,7 +446,7 @@ function readUnsupportedRepeatedTapOption(action: SessionAction): string | undef return undefined; } -function withTapOptions(target: unknown, options: Record): MaestroCommand { +function withTapOptions(target: unknown, options: Record): MaestroExportCommand { if (Object.keys(options).length === 0) return { tapOn: target }; if (typeof target === 'string') return { tapOn: { text: target, ...options } }; if (target && typeof target === 'object' && !Array.isArray(target)) { @@ -503,7 +501,7 @@ function readTimeout(action: SessionAction, fallback: number): number { return candidate && isNumber(candidate) ? Number(candidate) : fallback; } -function formatMaestroYaml(config: MaestroExportConfig, commands: MaestroCommand[]): string { +function formatMaestroYaml(config: MaestroExportConfig, commands: MaestroExportCommand[]): string { const hasConfig = Object.keys(config).length > 0; const docs: unknown[] = hasConfig ? [config, commands] : [commands]; return stringifyMaestroYamlDocuments(docs); diff --git a/src/compat/maestro/export-points.ts b/src/compat/maestro/export-points.ts new file mode 100644 index 000000000..f46b78e60 --- /dev/null +++ b/src/compat/maestro/export-points.ts @@ -0,0 +1,3 @@ +export function formatMaestroPoint(x: number | string, y: number | string): string { + return `${x},${y}`; +} diff --git a/src/compat/maestro/export-types.ts b/src/compat/maestro/export-types.ts new file mode 100644 index 000000000..3d46400f3 --- /dev/null +++ b/src/compat/maestro/export-types.ts @@ -0,0 +1,6 @@ +export type MaestroExportConfig = { + appId?: string; + env?: Record; +}; + +export type MaestroExportCommand = string | Record; diff --git a/src/compat/maestro/export-yaml.ts b/src/compat/maestro/export-yaml.ts new file mode 100644 index 000000000..d72990917 --- /dev/null +++ b/src/compat/maestro/export-yaml.ts @@ -0,0 +1,5 @@ +import { stringify } from 'yaml'; + +export function stringifyMaestroYamlDocuments(documents: readonly unknown[]): string { + return `${documents.map((document) => stringify(document).trimEnd()).join('\n---\n')}\n`; +} diff --git a/src/compat/maestro/file-execution.ts b/src/compat/maestro/file-execution.ts new file mode 100644 index 000000000..530f9a66a --- /dev/null +++ b/src/compat/maestro/file-execution.ts @@ -0,0 +1,47 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { executeMaestroProgram } from './engine.ts'; +import type { + MaestroEngineObserver, + MaestroEngineResult, + MaestroCompatibilityTimingPolicy, + MaestroRuntimePort, +} from './engine-types.ts'; +import type { MaestroPlatform } from './program-ir.ts'; +import { parseMaestroProgram } from './program-ir-parser.ts'; +import { createMaestroProgramLoader } from './program-loader.ts'; + +export async function executeMaestroFile(params: { + filePath: string; + port: MaestroRuntimePort; + env?: Readonly>; + defaults?: Readonly>; + builtins?: Readonly>; + platform?: MaestroPlatform; + target?: string; + startIndex?: number; + from?: number; + planDigest?: string; + timing?: Partial; + signal?: AbortSignal; + observer?: MaestroEngineObserver; + now?: () => number; +}): Promise { + const filePath = path.resolve(params.filePath); + const program = parseMaestroProgram(fs.readFileSync(filePath, 'utf8'), { sourcePath: filePath }); + return await executeMaestroProgram(program, params.port, { + loadProgram: createMaestroProgramLoader(path.dirname(filePath)), + ...(params.env ? { env: params.env } : {}), + ...(params.defaults ? { defaults: params.defaults } : {}), + ...(params.builtins ? { builtins: params.builtins } : {}), + ...(params.platform ? { platform: params.platform } : {}), + ...(params.target ? { target: params.target } : {}), + ...(params.startIndex === undefined ? {} : { startIndex: params.startIndex }), + ...(params.from === undefined ? {} : { from: params.from }), + ...(params.planDigest ? { planDigest: params.planDigest } : {}), + ...(params.timing ? { timing: params.timing } : {}), + ...(params.signal ? { signal: params.signal } : {}), + ...(params.observer ? { observer: params.observer } : {}), + ...(params.now ? { now: params.now } : {}), + }); +} diff --git a/src/compat/maestro/flow-control.ts b/src/compat/maestro/flow-control.ts deleted file mode 100644 index d5eadd7ab..000000000 --- a/src/compat/maestro/flow-control.ts +++ /dev/null @@ -1,509 +0,0 @@ -import type { ReplayControlActionSource, SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { maestroSelector } from './interactions.ts'; -import { - action, - assertOnlyKeys, - isPlainRecord, - normalizeCommandList, - readEnvMap, - resolveMaestroString, - unsupportedMaestroSyntax, -} from './support.ts'; -import type { - MaestroCommand, - MaestroCommandMapperDeps, - MaestroConvertedActions, - MaestroFlowConfig, - MaestroParseContext, - MaestroReplayFlow, -} from './types.ts'; - -// repeat.times is expanded at parse time for deterministic replay traces. Keep -// a guardrail until repeat can execute as a runtime loop without materializing -// every child action. -const MAX_REPEAT_EXPANSIONS = 1000; -type MaestroConditionPlatform = 'android' | 'ios' | 'web'; - -type ConvertCommandList = ( - commands: MaestroCommand[], - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, -) => MaestroConvertedActions; - -const NO_CONVERTED_ACTIONS: MaestroConvertedActions = { actions: [], sources: [] }; - -/** A parsed `runFlow` include's actions, paired with their own file+line. */ -function includeConvertedActions(flow: MaestroReplayFlow): MaestroConvertedActions { - return { - actions: flow.actions, - sources: flow.actions.map((_, index) => { - const path = flow.actionSourcePaths?.[index]; - return path ? { path, line: flow.actionLines[index] ?? 1 } : undefined; - }), - }; -} - -export function convertRunFlow( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, - convertCommandList: ConvertCommandList, -): MaestroConvertedActions { - if (typeof value === 'string') { - return includeConvertedActions( - deps.parseRunFlowFile(resolveMaestroString(value, context), context), - ); - } - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'runFlow expects a file path string or map.'); - } - assertOnlyKeys(value, 'runFlow', ['file', 'commands', 'env', 'when', 'label']); - const condition = readRunFlowCondition(value.when, context); - if (!condition.shouldRun) return NO_CONVERTED_ACTIONS; - - const runContext = { - ...context, - env: { ...context.env, ...readEnvMap(value.env, 'runFlow.env'), ...context.envOverrides }, - }; - const converted = readRunFlowActions(value, config, runContext, deps, convertCommandList); - return wrapRunFlowCondition(converted, condition); -} - -export function convertRepeat( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, - convertCommandList: ConvertCommandList, -): MaestroConvertedActions { - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'repeat expects a map.'); - } - assertOnlyKeys(value, 'repeat', ['times', 'commands', 'while']); - if (value.while !== undefined) { - throw unsupportedMaestroSyntax( - 'Maestro repeat.while is not supported yet. Only deterministic repeat.times is supported.', - ); - } - const times = readRepeatTimes(value.times, context); - if (!Array.isArray(value.commands)) { - throw new AppError('INVALID_ARGS', 'repeat requires a commands list.'); - } - if (times > MAX_REPEAT_EXPANSIONS) { - throw new AppError( - 'INVALID_ARGS', - `repeat.times must be <= ${MAX_REPEAT_EXPANSIONS} for deterministic replay expansion.`, - ); - } - const commands = normalizeCommandList(value.commands); - const iterations = Array.from({ length: times }, () => - convertCommandList(commands, config, context, deps), - ); - return { - actions: iterations.flatMap((entry) => entry.actions), - sources: iterations.flatMap((entry) => entry.sources), - }; -} - -export function convertRetry( - value: unknown, - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, - convertCommandList: ConvertCommandList, -): MaestroConvertedActions { - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'retry expects a map.'); - } - assertOnlyKeys(value, 'retry', ['maxRetries', 'commands']); - if (!Array.isArray(value.commands)) { - throw new AppError('INVALID_ARGS', 'retry requires a commands list.'); - } - const maxRetries = readRetryMaxRetries(value.maxRetries, context); - const commands = normalizeCommandList(value.commands); - const converted = convertCommandList(commands, config, context, deps); - return controlWrapperActions( - replayControlAction('retry', [String(maxRetries)], { - kind: 'retry', - maxRetries, - actions: converted.actions, - ...actionSourcesEntry(converted.sources), - }), - ); -} - -function readRunFlowActions( - value: Record, - config: MaestroFlowConfig, - context: MaestroParseContext, - deps: MaestroCommandMapperDeps, - convertCommandList: ConvertCommandList, -): MaestroConvertedActions { - if (typeof value.file === 'string') { - return includeConvertedActions( - deps.parseRunFlowFile(resolveMaestroString(value.file, context), context), - ); - } - if (Array.isArray(value.commands)) { - return convertCommandList(normalizeCommandList(value.commands), config, context, deps); - } - throw new AppError('INVALID_ARGS', 'runFlow map requires either file or commands.'); -} - -/** A control wrapper is itself a same-file action; its nested sources live in the control. */ -function controlWrapperActions(wrapper: SessionAction): MaestroConvertedActions { - return { actions: [wrapper], sources: [undefined] }; -} - -function actionSourcesEntry(sources: (ReplayControlActionSource | undefined)[]): { - actionSources?: (ReplayControlActionSource | undefined)[]; -} { - return sources.some((entry) => entry !== undefined) ? { actionSources: sources } : {}; -} - -type RunFlowCondition = { - shouldRun: boolean; - visibleSelector?: string; - notVisibleSelector?: string; -}; - -function readRunFlowCondition(value: unknown, context: MaestroParseContext): RunFlowCondition { - if (value === undefined || value === null) return { shouldRun: true }; - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'runFlow.when expects a map.'); - } - assertOnlyKeys(value, 'runFlow.when', ['platform', 'visible', 'notVisible', 'true']); - if (!matchesRunFlowStaticCondition(value, context)) return { shouldRun: false }; - return { - shouldRun: true, - ...readRunFlowVisibilityCondition(value, context), - }; -} - -function matchesRunFlowStaticCondition( - value: Record, - context: MaestroParseContext, -): boolean { - if (value.true !== undefined && !evaluateRunFlowTrueCondition(value.true, context)) return false; - if (value.platform === undefined) return true; - const platform = normalizeRunFlowPlatform(value.platform, 'runFlow.when.platform'); - if (!context.platform) { - throw new AppError( - 'INVALID_ARGS', - 'Maestro runFlow.when.platform requires replay to be run with --platform ios|android.', - ); - } - return platform === context.platform; -} - -function readRunFlowVisibilityCondition( - value: Record, - context: MaestroParseContext, -): Pick { - return { - ...(value.visible !== undefined - ? { visibleSelector: maestroSelector(value.visible, 'runFlow.when.visible', [], context) } - : {}), - ...(value.notVisible !== undefined - ? { - notVisibleSelector: maestroSelector( - value.notVisible, - 'runFlow.when.notVisible', - [], - context, - ), - } - : {}), - }; -} - -function normalizeRunFlowPlatform(value: unknown, name: string): MaestroConditionPlatform { - if (typeof value !== 'string') { - throw new AppError('INVALID_ARGS', `${name} expects Android, iOS, or Web.`); - } - const normalized = value.trim().toLowerCase(); - if (normalized === 'android' || normalized === 'ios' || normalized === 'web') { - return normalized; - } - throw new AppError('INVALID_ARGS', `${name} expects Android, iOS, or Web.`); -} - -function evaluateRunFlowTrueCondition(value: unknown, context: MaestroParseContext): boolean { - if (typeof value === 'boolean') return value; - if (typeof value !== 'string') { - throw new AppError('INVALID_ARGS', 'runFlow.when.true expects a boolean or expression string.'); - } - const expression = unwrapMaestroExpression(resolveMaestroString(value, context)); - const parser = new MaestroBooleanExpressionParser(tokenizeMaestroBooleanExpression(expression), { - platform: context.platform, - }); - return parser.parse(); -} - -function unwrapMaestroExpression(value: string): string { - const trimmed = value.trim(); - return trimmed.startsWith('${') && trimmed.endsWith('}') ? trimmed.slice(2, -1).trim() : trimmed; -} - -type MaestroBooleanToken = - | { type: 'platform' } - | MaestroBooleanOperatorToken - | { type: 'paren'; value: '(' | ')' } - | { type: 'string'; value: string } - | { type: 'boolean'; value: boolean }; - -type MaestroBooleanOperatorToken = { type: 'operator'; value: '==' | '!=' | '&&' | '||' }; - -type MaestroBooleanTokenMatch = { - token: MaestroBooleanToken; - length: number; -}; - -function tokenizeMaestroBooleanExpression(expression: string): MaestroBooleanToken[] { - const tokens: MaestroBooleanToken[] = []; - let index = 0; - while (index < expression.length) { - const remaining = expression.slice(index); - const skipped = whitespaceLength(remaining); - if (skipped > 0) { - index += skipped; - continue; - } - const next = readMaestroBooleanToken(remaining); - if (next) { - tokens.push(next.token); - index += next.length; - continue; - } - throw new AppError( - 'INVALID_ARGS', - `Unsupported runFlow.when.true expression near "${remaining.slice(0, 24)}".`, - ); - } - return tokens; -} - -function whitespaceLength(value: string): number { - return /^\s+/.exec(value)?.[0].length ?? 0; -} - -function readMaestroBooleanToken(remaining: string): MaestroBooleanTokenMatch | null { - return ( - readPlatformToken(remaining) ?? - readOperatorToken(remaining) ?? - readParenToken(remaining) ?? - readStringToken(remaining) ?? - readBooleanToken(remaining) - ); -} - -function readPlatformToken(remaining: string): MaestroBooleanTokenMatch | null { - const name = 'maestro.platform'; - return remaining.startsWith(name) ? { token: { type: 'platform' }, length: name.length } : null; -} - -function readOperatorToken(remaining: string): MaestroBooleanTokenMatch | null { - const operator = /^(==|!=|&&|\|\|)/.exec(remaining)?.[1]; - return operator - ? { - token: { type: 'operator', value: operator as MaestroBooleanOperatorToken['value'] }, - length: operator.length, - } - : null; -} - -function readParenToken(remaining: string): MaestroBooleanTokenMatch | null { - const value = remaining[0]; - return value === '(' || value === ')' ? { token: { type: 'paren', value }, length: 1 } : null; -} - -function readStringToken(remaining: string): MaestroBooleanTokenMatch | null { - const quoted = /^(['"])(.*?)\1/.exec(remaining); - return quoted - ? { token: { type: 'string', value: quoted[2] ?? '' }, length: quoted[0].length } - : null; -} - -function readBooleanToken(remaining: string): MaestroBooleanTokenMatch | null { - const value = /^(true|false)\b/.exec(remaining)?.[1]; - return value - ? { token: { type: 'boolean', value: value === 'true' }, length: value.length } - : null; -} - -class MaestroBooleanExpressionParser { - private index = 0; - private readonly tokens: MaestroBooleanToken[]; - private readonly context: { platform?: 'android' | 'ios' }; - - constructor(tokens: MaestroBooleanToken[], context: { platform?: 'android' | 'ios' }) { - this.tokens = tokens; - this.context = context; - } - - parse(): boolean { - const result = this.parseOr(); - if (this.peek()) { - throw new AppError('INVALID_ARGS', 'Unsupported trailing runFlow.when.true expression.'); - } - return result; - } - - private parseOr(): boolean { - let result = this.parseAnd(); - while (this.consumeOperator('||')) { - result = this.parseAnd() || result; - } - return result; - } - - private parseAnd(): boolean { - let result = this.parsePrimary(); - while (this.consumeOperator('&&')) { - result = this.parsePrimary() && result; - } - return result; - } - - private parsePrimary(): boolean { - const token = this.peek(); - if (!token) { - throw new AppError('INVALID_ARGS', 'Incomplete runFlow.when.true expression.'); - } - if (token.type === 'boolean') { - this.index += 1; - return token.value; - } - if (token.type === 'paren' && token.value === '(') { - this.index += 1; - const result = this.parseOr(); - if (!this.consumeParen(')')) { - throw new AppError('INVALID_ARGS', 'Unclosed runFlow.when.true parenthesis.'); - } - return result; - } - return this.parsePlatformComparison(); - } - - private parsePlatformComparison(): boolean { - this.expectPlatform(); - const operator = this.expectEqualityOperator(); - const value = this.expectString().toLowerCase(); - const platform = this.context.platform; - return operator === '==' ? platform === value : platform !== value; - } - - private expectPlatform(): void { - if (this.peek()?.type !== 'platform') { - throw new AppError( - 'INVALID_ARGS', - 'runFlow.when.true supports maestro.platform comparisons.', - ); - } - this.index += 1; - } - - private expectEqualityOperator(): '==' | '!=' { - const token = this.peek(); - if (token?.type === 'operator' && (token.value === '==' || token.value === '!=')) { - this.index += 1; - return token.value; - } - throw new AppError('INVALID_ARGS', 'runFlow.when.true comparison requires == or !=.'); - } - - private expectString(): string { - const token = this.peek(); - if (token?.type === 'string') { - this.index += 1; - return token.value; - } - throw new AppError('INVALID_ARGS', 'runFlow.when.true comparison requires a string literal.'); - } - - private consumeOperator(value: '&&' | '||'): boolean { - const token = this.peek(); - if (token?.type !== 'operator' || token.value !== value) return false; - this.index += 1; - return true; - } - - private consumeParen(value: '(' | ')'): boolean { - const token = this.peek(); - if (token?.type !== 'paren' || token.value !== value) return false; - this.index += 1; - return true; - } - - private peek(): MaestroBooleanToken | undefined { - return this.tokens[this.index]; - } -} - -function wrapRunFlowCondition( - converted: MaestroConvertedActions, - condition: RunFlowCondition, -): MaestroConvertedActions { - const { visibleSelector, notVisibleSelector } = condition; - if (!visibleSelector && !notVisibleSelector) return converted; - if (visibleSelector && notVisibleSelector) { - throw unsupportedMaestroSyntax( - 'Maestro runFlow.when cannot combine visible and notVisible yet.', - ); - } - const mode = visibleSelector ? 'visible' : 'notVisible'; - const selector = visibleSelector ?? notVisibleSelector ?? ''; - return controlWrapperActions( - replayControlAction('runFlow.when', [mode, selector], { - kind: 'maestroRunFlowWhen', - mode, - selector, - actions: converted.actions, - ...actionSourcesEntry(converted.sources), - }), - ); -} - -function replayControlAction( - command: string, - positionals: string[], - replayControl: NonNullable, -): SessionAction { - return { - ...action(command, positionals), - replayControl, - }; -} - -function readRepeatTimes(value: unknown, context: MaestroParseContext): number { - return readMaestroNonNegativeInteger(value, context, 'repeat.times'); -} - -function readRetryMaxRetries(value: unknown, context: MaestroParseContext): number { - if (value === undefined) return 1; - return readMaestroNonNegativeInteger(value, context, 'retry.maxRetries'); -} - -function readMaestroNonNegativeInteger( - value: unknown, - context: MaestroParseContext, - name: string, -): number { - const resolved = typeof value === 'string' ? resolveMaestroString(value, context) : value; - const numeric = - typeof resolved === 'number' - ? resolved - : typeof resolved === 'string' && /^\d+$/.test(resolved) - ? Number(resolved) - : undefined; - if (numeric === undefined || !Number.isInteger(numeric) || numeric < 0) { - throw new AppError( - 'INVALID_ARGS', - `${name} must be a non-negative integer or \${VAR} resolving to one.`, - ); - } - return numeric; -} diff --git a/src/compat/maestro/flow-yaml.ts b/src/compat/maestro/flow-yaml.ts deleted file mode 100644 index 604dc5c59..000000000 --- a/src/compat/maestro/flow-yaml.ts +++ /dev/null @@ -1,19 +0,0 @@ -import { parseAllDocuments, stringify } from 'yaml'; -import { AppError } from '../../kernel/errors.ts'; - -export function parseMaestroYamlDocuments(script: string): unknown[] { - const documents = parseAllDocuments(script); - for (const document of documents) { - if (document.errors.length > 0) { - const message = document.errors[0]?.message ?? 'Invalid Maestro YAML flow.'; - throw new AppError('INVALID_ARGS', `Invalid Maestro YAML flow: ${message}`); - } - } - return documents - .map((document) => document.toJSON() as unknown) - .filter((value) => value !== null); -} - -export function stringifyMaestroYamlDocuments(documents: readonly unknown[]): string { - return `${documents.map((document) => stringify(document).trimEnd()).join('\n---\n')}\n`; -} diff --git a/src/compat/maestro/interactions.ts b/src/compat/maestro/interactions.ts deleted file mode 100644 index 952fa611b..000000000 --- a/src/compat/maestro/interactions.ts +++ /dev/null @@ -1,408 +0,0 @@ -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { - action, - assertOnlyKeys, - isPlainRecord, - readTimeoutMs, - requireStringValue, - resolveMaestroString, - unsupportedMaestroSyntax, -} from './support.ts'; -import { parseAbsolutePoint, parseMaestroPoint } from './points.ts'; -import { MAESTRO_RUNTIME_COMMAND } from './runtime-commands.ts'; -import type { MaestroParseContext } from './types.ts'; -import type { ScrollDirection } from '../../contracts/scroll-gesture.ts'; - -export function convertTapOn(value: unknown, context: MaestroParseContext): SessionAction { - if (typeof value === 'string') { - return action( - MAESTRO_RUNTIME_COMMAND.tapOn, - [visibleTextSelector(resolveMaestroString(value, context))], - maestroTapOnFlags(value), - ); - } - if (isPlainRecord(value) && typeof value.point === 'string') { - assertOnlyKeys(value, 'tapOn', ['point', 'repeat', 'delay', 'optional', 'label']); - const point = parseMaestroPoint(value.point); - if (point.kind === 'percent') { - return action( - MAESTRO_RUNTIME_COMMAND.tapPointPercent, - [String(point.x), String(point.y)], - tapFlags(value), - ); - } - return action('click', [String(point.x), String(point.y)], tapFlags(value)); - } - if (isPlainRecord(value)) { - assertOnlyKeys(value, 'tapOn', [ - 'id', - 'text', - 'childOf', - 'enabled', - 'index', - 'selected', - 'repeat', - 'delay', - 'optional', - 'label', - ]); - } - const flags = maestroTapOnFlags(value); - return action( - MAESTRO_RUNTIME_COMMAND.tapOn, - [ - maestroSelector( - value, - 'tapOn', - ['repeat', 'delay', 'optional', 'label', 'index', 'childOf'], - context, - ), - ...maestroTapOnRuntimeOptions(value, context), - ], - flags, - ); -} - -export function convertDoubleTapOn(value: unknown, context: MaestroParseContext): SessionAction { - if (isPlainRecord(value) && typeof value.point === 'string') { - assertOnlyKeys(value, 'doubleTapOn', ['point', 'delay']); - const point = parseAbsolutePoint(value.point); - return action('click', [String(point.x), String(point.y)], doubleTapFlags(value)); - } - if (isPlainRecord(value)) { - assertOnlyKeys(value, 'doubleTapOn', ['id', 'text', 'enabled', 'selected', 'delay']); - } - return action( - 'click', - [maestroSelector(value, 'doubleTapOn', ['delay'], context)], - doubleTapFlags(value), - ); -} - -export function convertLongPressOn(value: unknown, context: MaestroParseContext): SessionAction { - if (isPlainRecord(value) && typeof value.point === 'string') { - assertOnlyKeys(value, 'longPressOn', ['point']); - const point = parseAbsolutePoint(value.point); - return action('longpress', [String(point.x), String(point.y), '3000']); - } - if (isPlainRecord(value)) { - assertOnlyKeys(value, 'longPressOn', ['id', 'text', 'enabled', 'selected']); - } - return action('click', [maestroSelector(value, 'longPressOn', [], context)], { holdMs: 3000 }); -} - -export function readInputText(value: unknown): string { - if (typeof value === 'string') return value; - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'inputText expects a string or map.'); - } - assertOnlyKeys(value, 'inputText', ['text', 'label']); - if (typeof value.text !== 'string') { - throw new AppError('INVALID_ARGS', 'inputText map requires a string text field.'); - } - return value.text; -} - -export function convertEraseText(value: unknown): SessionAction { - if (value === null || value === undefined) return action('type', ['\b'.repeat(50)]); - if (typeof value === 'number' && Number.isInteger(value) && value > 0) { - return action('type', ['\b'.repeat(value)]); - } - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'eraseText expects empty, a positive count, or a map.'); - } - assertOnlyKeys(value, 'eraseText', ['charactersToErase']); - if (value.charactersToErase === undefined) return action('type', ['\b'.repeat(50)]); - if ( - typeof value.charactersToErase !== 'number' || - !Number.isInteger(value.charactersToErase) || - value.charactersToErase <= 0 - ) { - throw new AppError('INVALID_ARGS', 'eraseText.charactersToErase must be a positive integer.'); - } - return action('type', ['\b'.repeat(value.charactersToErase)]); -} - -export function convertExtendedWaitUntil( - value: unknown, - context: MaestroParseContext, -): SessionAction[] { - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'extendedWaitUntil expects a map.'); - } - assertOnlyKeys(value, 'extendedWaitUntil', ['visible', 'notVisible', 'timeout']); - const target = value.visible ?? value.notVisible; - if (target === undefined) { - throw unsupportedMaestroSyntax( - 'Only Maestro extendedWaitUntil.visible/notVisible is supported.', - ); - } - const selector = maestroSelector(target, 'extendedWaitUntil', [], context); - const timeoutMs = String(readTimeoutMs(value, 17000)); - if (value.notVisible !== undefined) { - return [action(MAESTRO_RUNTIME_COMMAND.assertNotVisible, [selector, timeoutMs])]; - } - return [ - action(MAESTRO_RUNTIME_COMMAND.assertVisible, [selector, timeoutMs], { - maestro: { allowAlreadyPastLoading: true }, - }), - ]; -} - -export function convertScroll(value: unknown): SessionAction { - if (value !== null && value !== undefined) { - throw unsupportedMaestroSyntax('Maestro scroll options are not supported yet.'); - } - return action('scroll', ['down']); -} - -export function convertScrollUntilVisible( - value: unknown, - context: MaestroParseContext, -): SessionAction[] { - if (typeof value === 'string') { - return [ - action(MAESTRO_RUNTIME_COMMAND.scrollUntilVisible, [ - visibleTextSelector(resolveMaestroString(value, context)), - '5000', - 'down', - ]), - ]; - } - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'scrollUntilVisible expects a string or map.'); - } - assertOnlyKeys(value, 'scrollUntilVisible', ['element', 'direction', 'timeout']); - const selector = maestroSelector(value.element, 'scrollUntilVisible.element', [], context); - const direction = - typeof value.direction === 'string' - ? readMaestroDirection(value.direction, 'scrollUntilVisible.direction') - : 'down'; - const timeoutMs = String(readTimeoutMs(value, 5000)); - return [action(MAESTRO_RUNTIME_COMMAND.scrollUntilVisible, [selector, timeoutMs, direction])]; -} - -export function convertSwipe(value: unknown, context: MaestroParseContext): SessionAction { - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'swipe expects a map.'); - } - assertOnlyKeys(value, 'swipe', ['start', 'end', 'direction', 'duration', 'from', 'label']); - // Maestro classifies the command as coordinate-based when either endpoint key is present. - if (Object.hasOwn(value, 'start') || Object.hasOwn(value, 'end')) { - if (value.direction !== undefined) { - throw unsupportedMaestroSyntax( - 'Maestro swipe cannot combine direction with start/end coordinates.', - ); - } - return convertCoordinateSwipe(value); - } - const from = value.from ?? (typeof value.label === 'string' ? value.label : undefined); - if (from !== undefined) { - return convertTargetedSwipe(value, from, context); - } - if (typeof value.direction === 'string') { - return action(MAESTRO_RUNTIME_COMMAND.swipeScreen, [ - 'direction', - readSwipeDirection(value.direction), - ...swipeDurationPositionals(value), - ]); - } - return convertCoordinateSwipe(value); -} - -function convertTargetedSwipe( - value: Record, - from: unknown, - context: MaestroParseContext, -): SessionAction { - const direction = readSwipeDirection( - typeof value.direction === 'string' ? value.direction : 'up', - ); - return action(MAESTRO_RUNTIME_COMMAND.swipeOn, [ - maestroSelector(from, 'swipe.from', [], context), - direction, - ...swipeDurationPositionals(value), - ]); -} - -function convertCoordinateSwipe(value: Record): SessionAction { - const { start, end } = readCoordinateSwipePoints(value); - const durationMs = readSwipeDurationMs(value.duration); - return convertCoordinateSwipePoints(start, end, durationMs); -} - -function readCoordinateSwipePoints(value: Record): { - start: ReturnType; - end: ReturnType; -} { - if (typeof value.start === 'string' && typeof value.end === 'string') { - return { start: parseMaestroPoint(value.start), end: parseMaestroPoint(value.end) }; - } - throw unsupportedMaestroSyntax('Maestro swipe requires both start and end coordinates.'); -} - -function readSwipeDurationMs(duration: unknown): string | undefined { - return typeof duration === 'number' && Number.isFinite(duration) - ? String(Math.max(16, Math.floor(duration))) - : undefined; -} - -function convertCoordinateSwipePoints( - start: ReturnType, - end: ReturnType, - durationMs: string | undefined, -): SessionAction { - if (start.kind === 'absolute' && end.kind === 'absolute') { - return action('swipe', [ - String(start.x), - String(start.y), - String(end.x), - String(end.y), - ...(durationMs ? [durationMs] : []), - ]); - } - if (start.kind === 'percent' && end.kind === 'percent') { - return action(MAESTRO_RUNTIME_COMMAND.swipeScreen, [ - 'percent', - String(start.x), - String(start.y), - String(end.x), - String(end.y), - ...(durationMs ? [durationMs] : []), - ]); - } - throw unsupportedMaestroSyntax( - 'Maestro swipe start/end must both be absolute pixels or both be percentages.', - ); -} - -function readMaestroDirection(direction: string, name: string): ScrollDirection { - const normalized = direction.toLowerCase(); - switch (normalized) { - case 'up': - case 'down': - case 'left': - case 'right': - return normalized; - default: - throw unsupportedMaestroSyntax(`Maestro ${name} must be UP, DOWN, LEFT, or RIGHT.`); - } -} - -function readSwipeDirection(direction: string): ScrollDirection { - return readMaestroDirection(direction, 'swipe direction'); -} - -export function convertPressKey(value: unknown): SessionAction { - const key = requireStringValue('pressKey', value).toLowerCase(); - if (key === 'back') return action('back'); - if (key === 'enter' || key === 'return') return action(MAESTRO_RUNTIME_COMMAND.pressEnter); - if (key === 'home') return action('home'); - throw unsupportedMaestroSyntax(`Maestro pressKey "${key}" is not supported yet.`); -} - -export function maestroSelector( - value: unknown, - command: string, - allowedExtraKeys: readonly string[] = [], - context: MaestroParseContext, -): string { - if (typeof value === 'string') return visibleTextSelector(resolveMaestroString(value, context)); - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', `${command} expects a string or selector map.`); - } - assertOnlyKeys(value, command, ['id', 'text', 'enabled', 'selected', ...allowedExtraKeys]); - - const terms: string[] = []; - const stateTerms: string[] = []; - if (typeof value.enabled === 'boolean') - stateTerms.push(selectorTerm('enabled', String(value.enabled))); - if (typeof value.selected === 'boolean') - stateTerms.push(selectorTerm('selected', String(value.selected))); - if (typeof value.id === 'string') - terms.push(selectorTerm('id', resolveMaestroString(value.id, context)), ...stateTerms); - if (typeof value.text === 'string' && terms.length === 0) { - return visibleTextSelector(resolveMaestroString(value.text, context), stateTerms); - } - if (typeof value.label === 'string' && terms.length === 0) - terms.push(selectorTerm('label', resolveMaestroString(value.label, context)), ...stateTerms); - if (terms.length === 0 && stateTerms.length > 0) terms.push(...stateTerms); - if (terms.length === 0) { - throw new AppError( - 'INVALID_ARGS', - `${command} selector map must include one of id, text, label, enabled, or selected.`, - ); - } - return terms.join(' '); -} - -function visibleTextSelector(value: string, extraTerms: readonly string[] = []): string { - return [ - [selectorTerm('label', value), ...extraTerms].join(' '), - [selectorTerm('text', value), ...extraTerms].join(' '), - [selectorTerm('id', value), ...extraTerms].join(' '), - ].join(' || '); -} - -function maestroTapOnRuntimeOptions(value: unknown, context: MaestroParseContext): string[] { - if (!isPlainRecord(value)) return []; - const options: { index?: number; childOf?: string } = {}; - if (value.index !== undefined) { - if (typeof value.index !== 'number' || !Number.isInteger(value.index) || value.index < 0) { - throw new AppError('INVALID_ARGS', 'tapOn.index must be a non-negative integer.'); - } - options.index = value.index; - } - if (value.childOf !== undefined) { - options.childOf = maestroSelector(value.childOf, 'tapOn.childOf', [], context); - } - return Object.keys(options).length > 0 ? [JSON.stringify(options)] : []; -} - -function swipeDurationPositionals(value: Record): string[] { - const durationMs = readSwipeDurationMs(value.duration); - return durationMs ? [durationMs] : []; -} - -function selectorTerm(key: string, value: string): string { - return `${key}=${JSON.stringify(value)}`; -} - -function tapFlags(value: unknown): SessionAction['flags'] | undefined { - if (!isPlainRecord(value)) return undefined; - const flags: SessionAction['flags'] = {}; - const repeat = positiveInteger(value.repeat); - const delay = nonNegativeInteger(value.delay); - if (repeat && repeat > 1) flags.count = repeat; - if (delay !== undefined) flags.intervalMs = delay; - if (value.optional === true) flags.maestro = { optional: true }; - return Object.keys(flags).length > 0 ? flags : undefined; -} - -function maestroTapOnFlags(value: unknown): SessionAction['flags'] { - const flags = tapFlags(value) ?? {}; - return { - ...flags, - maestro: { - ...(flags.maestro ?? {}), - allowNonHittableCoordinateFallback: true, - }, - }; -} - -function doubleTapFlags(value: unknown): SessionAction['flags'] { - const flags: SessionAction['flags'] = { doubleTap: true }; - if (isPlainRecord(value) && typeof value.delay === 'number' && Number.isInteger(value.delay)) { - flags.intervalMs = Math.max(0, value.delay); - } - return flags; -} - -function positiveInteger(value: unknown): number | undefined { - return typeof value === 'number' && Number.isInteger(value) && value > 0 ? value : undefined; -} - -function nonNegativeInteger(value: unknown): number | undefined { - return typeof value === 'number' && Number.isInteger(value) && value >= 0 ? value : undefined; -} diff --git a/src/compat/maestro/points.ts b/src/compat/maestro/points.ts deleted file mode 100644 index 4b01fe6f5..000000000 --- a/src/compat/maestro/points.ts +++ /dev/null @@ -1,41 +0,0 @@ -import { unsupportedMaestroSyntax } from './support.ts'; - -export type MaestroPoint = - | { - kind: 'absolute'; - x: number; - y: number; - } - | { - kind: 'percent'; - x: number; - y: number; - }; - -export function formatMaestroPoint(x: number | string, y: number | string): string { - return `${x},${y}`; -} - -export function parseAbsolutePoint(value: string): { x: number; y: number } { - const match = value.match(/^(\d+),(\d+)$/); - if (!match) { - throw unsupportedMaestroSyntax( - 'Only absolute Maestro point selectors like "100,200" are supported.', - ); - } - return { x: Number(match[1]), y: Number(match[2]) }; -} - -export function parseMaestroPoint(value: string): MaestroPoint { - const absolute = value.match(/^\s*(\d+)\s*,\s*(\d+)\s*$/); - if (absolute) { - return { kind: 'absolute', x: Number(absolute[1]), y: Number(absolute[2]) }; - } - const percent = value.match(/^\s*(\d+(?:\.\d+)?)%\s*,\s*(\d+(?:\.\d+)?)%\s*$/); - if (percent) { - return { kind: 'percent', x: Number(percent[1]), y: Number(percent[2]) }; - } - throw unsupportedMaestroSyntax( - 'Only Maestro swipe coordinates like "100,200" or "50%,75%" are supported.', - ); -} diff --git a/src/compat/maestro/program-ir-command-parser.ts b/src/compat/maestro/program-ir-command-parser.ts new file mode 100644 index 000000000..fce1acf48 --- /dev/null +++ b/src/compat/maestro/program-ir-command-parser.ts @@ -0,0 +1,468 @@ +import { isMap, isScalar, isSeq, type Node } from 'yaml'; +import type { + MaestroAssertNotVisibleCommand, + MaestroAssertVisibleCommand, + MaestroBackCommand, + MaestroCommand, + MaestroEraseTextCommand, + MaestroExtendedWaitUntilCommand, + MaestroHideKeyboardCommand, + MaestroInputTextCommand, + MaestroLaunchAppCommand, + MaestroLaunchArguments, + MaestroOpenLinkCommand, + MaestroPasteTextCommand, + MaestroPressKeyCommand, + MaestroScrollCommand, + MaestroScrollUntilVisibleCommand, + MaestroStopAppCommand, + MaestroTakeScreenshotCommand, + MaestroWaitForAnimationToEndCommand, +} from './program-ir.ts'; +import { + parseMaestroDirection, + parseMaestroDoubleTapOnCommand, + parseMaestroLongPressOnCommand, + parseMaestroSelector, + parseMaestroSwipeCommand, + parseMaestroTapOnCommand, +} from './program-ir-gesture-parser.ts'; +import { + parseMaestroRepeatCommand, + parseMaestroRetryCommand, + parseMaestroRunFlowCommand, + parseMaestroRunScriptCommand, +} from './program-ir-flow-parser.ts'; +import { + assertOnlyKeys, + entryValue, + hasEntry, + invalidAt, + isNullNode, + readMapEntries, + readOptionalBoolean, + readOptionalEntry, + readOptionalNonNegativeInteger, + readOptionalNumber, + readOptionalString, + readRequiredPositiveInteger, + readRequiredString, + readScalarMap, + readScalarValue, + readSequenceItems, + sourceAt, + type MaestroProgramParseContext, +} from './program-ir-values.ts'; + +export function parseMaestroCommandList( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroCommand[] { + return readSequenceItems(node, name, context).map((item) => parseMaestroCommand(item, context)); +} + +function parseMaestroCommand( + node: Node | null | undefined, + context: MaestroProgramParseContext, +): MaestroCommand { + if (isScalar(node)) { + if (typeof node.value !== 'string') + invalidAt('Maestro command names must be strings.', node, context); + return parseScalarCommand(node.value, node, context); + } + if (!isMap(node)) invalidAt('Maestro commands must be a scalar or one-key map.', node, context); + const entries = readMapEntries(node, 'command', context); + if (entries.length !== 1) + invalidAt('Maestro command maps must contain exactly one command.', node, context); + const entry = entries[0]!; + return parseCommandValue(entry.key, entry.value, node, context); +} + +function parseScalarCommand( + name: string, + node: Node, + context: MaestroProgramParseContext, +): MaestroCommand { + const source = sourceAt(node, context); + switch (name) { + case 'launchApp': + return { kind: 'launchApp', source }; + case 'scroll': + return { kind: 'scroll', source }; + case 'eraseText': + return { kind: 'eraseText', source }; + case 'hideKeyboard': + return { kind: 'hideKeyboard', source }; + case 'back': + return { kind: 'back', source }; + case 'waitForAnimationToEnd': + return { kind: 'waitForAnimationToEnd', source }; + case 'stopApp': + return { kind: 'stopApp', source }; + default: + invalidAt(`Maestro command "${name}" is not supported.`, node, context); + } +} + +type CommandValueParser = ( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +) => MaestroCommand; + +const COMMAND_VALUE_PARSERS: Readonly> = { + launchApp: parseLaunchApp, + tapOn: parseMaestroTapOnCommand, + doubleTapOn: parseMaestroDoubleTapOnCommand, + longPressOn: parseMaestroLongPressOnCommand, + inputText: parseInputText, + eraseText: parseEraseText, + pasteText: parsePasteText, + openLink: parseOpenLink, + assertVisible: (value, node, context) => parseAssertion('assertVisible', value, node, context), + assertNotVisible: (value, node, context) => + parseAssertion('assertNotVisible', value, node, context), + extendedWaitUntil: parseExtendedWaitUntil, + takeScreenshot: parseTakeScreenshot, + scroll: parseScroll, + scrollUntilVisible: parseScrollUntilVisible, + swipe: parseMaestroSwipeCommand, + hideKeyboard: parseHideKeyboard, + pressKey: parsePressKey, + back: parseBack, + waitForAnimationToEnd: parseWaitForAnimationToEnd, + stopApp: parseStopApp, + runScript: parseMaestroRunScriptCommand, + runFlow: (value, node, context) => + parseMaestroRunFlowCommand(value, node, context, parseMaestroCommandList), + repeat: (value, node, context) => + parseMaestroRepeatCommand(value, node, context, parseMaestroCommandList), + retry: (value, node, context) => + parseMaestroRetryCommand(value, node, context, parseMaestroCommandList), +}; + +function parseCommandValue( + name: string, + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroCommand { + const parser = COMMAND_VALUE_PARSERS[name]; + if (!parser) invalidAt(`Maestro command "${name}" is not supported.`, commandNode, context); + return parser(value, commandNode, context); +} + +function parseLaunchApp( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroLaunchAppCommand { + const source = sourceAt(commandNode, context); + if (isNullNode(value)) return { kind: 'launchApp', source }; + if (isScalar(value)) + return { kind: 'launchApp', source, appId: readRequiredString(value, 'launchApp', context) }; + const entries = readMapEntries(value, 'launchApp', context); + assertOnlyKeys( + entries, + 'launchApp', + ['appId', 'stopApp', 'clearState', 'arguments', 'launchArguments'], + context, + ); + const appId = readOptionalEntry(entries, 'appId', (entry) => + readOptionalString(entry, 'launchApp.appId', context), + ); + const stopApp = readOptionalEntry(entries, 'stopApp', (entry) => + readOptionalBoolean(entry, 'launchApp.stopApp', context), + ); + const clearState = readOptionalEntry(entries, 'clearState', (entry) => + readOptionalBoolean(entry, 'launchApp.clearState', context), + ); + const args = readOptionalEntry(entries, 'arguments', (entry) => + parseLaunchArguments(entry, 'launchApp.arguments', context), + ); + const launchArguments = readOptionalEntry(entries, 'launchArguments', (entry) => + parseLaunchArguments(entry, 'launchApp.launchArguments', context), + ); + return { + kind: 'launchApp', + source, + ...(appId === undefined ? {} : { appId }), + ...(stopApp === undefined ? {} : { stopApp }), + ...(clearState === undefined ? {} : { clearState }), + ...(args === undefined ? {} : { arguments: args }), + ...(launchArguments === undefined ? {} : { launchArguments }), + }; +} + +function parseInputText( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroInputTextCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) + return { kind: 'inputText', source, text: readRequiredString(value, 'inputText', context) }; + const entries = readMapEntries(value, 'inputText', context); + assertOnlyKeys(entries, 'inputText', ['text', 'label'], context); + if (!hasEntry(entries, 'text')) + invalidAt('Maestro inputText requires text.', commandNode, context); + const text = readRequiredString(entryValue(entries, 'text'), 'inputText.text', context); + const label = hasEntry(entries, 'label') + ? readOptionalString(entryValue(entries, 'label'), 'inputText.label', context) + : undefined; + return { kind: 'inputText', source, text, ...(label === undefined ? {} : { label }) }; +} + +function parseEraseText( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroEraseTextCommand { + const source = sourceAt(commandNode, context); + if (isNullNode(value)) return { kind: 'eraseText', source }; + if (isScalar(value)) + return { + kind: 'eraseText', + source, + charactersToErase: readRequiredPositiveInteger(value, 'eraseText', context), + }; + const entries = readMapEntries(value, 'eraseText', context); + assertOnlyKeys(entries, 'eraseText', ['charactersToErase'], context); + const charactersToErase = hasEntry(entries, 'charactersToErase') + ? readOptionalPositiveInteger( + entryValue(entries, 'charactersToErase'), + 'eraseText.charactersToErase', + context, + ) + : undefined; + return { + kind: 'eraseText', + source, + ...(charactersToErase === undefined ? {} : { charactersToErase }), + }; +} + +function parsePasteText( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroPasteTextCommand { + return { + kind: 'pasteText', + source: sourceAt(commandNode, context), + text: readRequiredString(value, 'pasteText', context), + }; +} + +function parseOpenLink( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroOpenLinkCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) + return { kind: 'openLink', source, link: readRequiredString(value, 'openLink', context) }; + const entries = readMapEntries(value, 'openLink', context); + assertOnlyKeys(entries, 'openLink', ['link'], context); + return { + kind: 'openLink', + source, + link: readRequiredString(entryValue(entries, 'link'), 'openLink.link', context), + }; +} + +function parseAssertion( + kind: 'assertVisible' | 'assertNotVisible', + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroAssertVisibleCommand | MaestroAssertNotVisibleCommand { + return { + kind, + source: sourceAt(commandNode, context), + target: parseMaestroSelector(value, kind, context), + }; +} + +function parseExtendedWaitUntil( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroExtendedWaitUntilCommand { + const entries = readMapEntries(value, 'extendedWaitUntil', context); + assertOnlyKeys(entries, 'extendedWaitUntil', ['visible', 'notVisible', 'timeout'], context); + const visible = hasEntry(entries, 'visible') + ? parseMaestroSelector(entryValue(entries, 'visible'), 'extendedWaitUntil.visible', context) + : undefined; + const notVisible = hasEntry(entries, 'notVisible') + ? parseMaestroSelector( + entryValue(entries, 'notVisible'), + 'extendedWaitUntil.notVisible', + context, + ) + : undefined; + if (visible === undefined && notVisible === undefined) + invalidAt('Maestro extendedWaitUntil requires visible or notVisible.', commandNode, context); + const timeout = hasEntry(entries, 'timeout') + ? readOptionalNumber(entryValue(entries, 'timeout'), 'extendedWaitUntil.timeout', context) + : undefined; + return { + kind: 'extendedWaitUntil', + source: sourceAt(commandNode, context), + ...(visible === undefined ? {} : { visible }), + ...(notVisible === undefined ? {} : { notVisible }), + ...(timeout === undefined ? {} : { timeout }), + }; +} + +function parseTakeScreenshot( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroTakeScreenshotCommand { + return { + kind: 'takeScreenshot', + source: sourceAt(commandNode, context), + path: readRequiredString(value, 'takeScreenshot', context), + }; +} + +function parseScroll( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroScrollCommand { + if (!isNullNode(value)) + invalidAt('Maestro scroll does not accept options yet.', commandNode, context); + return { kind: 'scroll', source: sourceAt(commandNode, context) }; +} + +function parseScrollUntilVisible( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroScrollUntilVisibleCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) + return { + kind: 'scrollUntilVisible', + source, + element: parseMaestroSelector(value, 'scrollUntilVisible.element', context), + }; + const entries = readMapEntries(value, 'scrollUntilVisible', context); + assertOnlyKeys(entries, 'scrollUntilVisible', ['element', 'direction', 'timeout'], context); + if (!hasEntry(entries, 'element')) + invalidAt('Maestro scrollUntilVisible requires element.', commandNode, context); + const element = parseMaestroSelector( + entryValue(entries, 'element'), + 'scrollUntilVisible.element', + context, + ); + const direction = hasEntry(entries, 'direction') + ? parseMaestroDirection( + entryValue(entries, 'direction'), + 'scrollUntilVisible.direction', + context, + ) + : undefined; + const timeout = hasEntry(entries, 'timeout') + ? readOptionalNumber(entryValue(entries, 'timeout'), 'scrollUntilVisible.timeout', context) + : undefined; + return { + kind: 'scrollUntilVisible', + source, + element, + ...(direction === undefined ? {} : { direction }), + ...(timeout === undefined ? {} : { timeout }), + }; +} + +function parseHideKeyboard( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroHideKeyboardCommand { + if (!isNullNode(value)) + invalidAt('Maestro hideKeyboard does not accept options.', commandNode, context); + return { kind: 'hideKeyboard', source: sourceAt(commandNode, context) }; +} + +function parsePressKey( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroPressKeyCommand { + const key = readRequiredString(value, 'pressKey', context).toLowerCase(); + if (key !== 'back' && key !== 'enter' && key !== 'return' && key !== 'home') + invalidAt(`Maestro pressKey "${key}" is not supported.`, value, context); + return { kind: 'pressKey', source: sourceAt(commandNode, context), key }; +} + +function parseBack( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroBackCommand { + if (!isNullNode(value)) invalidAt('Maestro back does not accept options.', commandNode, context); + return { kind: 'back', source: sourceAt(commandNode, context) }; +} + +function parseWaitForAnimationToEnd( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroWaitForAnimationToEndCommand { + const source = sourceAt(commandNode, context); + if (isNullNode(value)) return { kind: 'waitForAnimationToEnd', source }; + if (isScalar(value)) { + const timeout = readOptionalNumber(value, 'waitForAnimationToEnd', context); + return { kind: 'waitForAnimationToEnd', source, ...(timeout === undefined ? {} : { timeout }) }; + } + const entries = readMapEntries(value, 'waitForAnimationToEnd', context); + assertOnlyKeys(entries, 'waitForAnimationToEnd', ['timeout'], context); + const timeout = hasEntry(entries, 'timeout') + ? readOptionalNumber(entryValue(entries, 'timeout'), 'waitForAnimationToEnd.timeout', context) + : undefined; + return { kind: 'waitForAnimationToEnd', source, ...(timeout === undefined ? {} : { timeout }) }; +} + +function parseStopApp( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroStopAppCommand { + const source = sourceAt(commandNode, context); + if (isNullNode(value)) return { kind: 'stopApp', source }; + return { kind: 'stopApp', source, appId: readRequiredString(value, 'stopApp', context) }; +} + +function parseLaunchArguments( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroLaunchArguments { + if (isSeq(node)) { + const values = readSequenceItems(node, name, context).map((item, index) => { + const value = readScalarValue(item, `${name}[${index}]`, context); + if (value === null) invalidAt(`${name}[${index}] expects a scalar value.`, item, context); + return value; + }); + return { kind: 'list', values }; + } + if (isMap(node)) return { kind: 'map', values: readScalarMap(node, name, context) }; + const value = readScalarValue(node, name, context); + if (value === null) invalidAt(`${name} expects a scalar, list, or map.`, node, context); + return { kind: 'scalar', value }; +} + +function readOptionalPositiveInteger( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): number | undefined { + const value = readOptionalNonNegativeInteger(node, name, context); + if (value !== undefined && value === 0) + invalidAt(`Maestro ${name} expects a positive integer.`, node, context); + return value; +} diff --git a/src/compat/maestro/program-ir-flow-parser.ts b/src/compat/maestro/program-ir-flow-parser.ts new file mode 100644 index 000000000..a25c88efe --- /dev/null +++ b/src/compat/maestro/program-ir-flow-parser.ts @@ -0,0 +1,206 @@ +import { isScalar, type Node } from 'yaml'; +import type { + MaestroCommand, + MaestroPlatform, + MaestroRepeatCommand, + MaestroRetryCommand, + MaestroRunFlowCommand, + MaestroRunFlowCondition, + MaestroRunScriptCommand, +} from './program-ir.ts'; +import { + assertOnlyKeys, + entryValue, + hasEntry, + invalidAt, + readIntegerValue, + readMapEntries, + readOptionalString, + readOptionalEntry, + readRequiredString, + readScalarMap, + readScalarValue, + sourceAt, + type MaestroProgramParseContext, +} from './program-ir-values.ts'; +import { parseMaestroSelector } from './program-ir-gesture-parser.ts'; + +export type MaestroCommandListParser = ( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +) => MaestroCommand[]; + +export function parseMaestroRunScriptCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroRunScriptCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) { + return { + kind: 'runScript', + source, + file: readRequiredString(value, 'runScript', context), + }; + } + const entries = readMapEntries(value, 'runScript', context); + assertOnlyKeys(entries, 'runScript', ['file', 'env'], context); + if (!hasEntry(entries, 'file')) + invalidAt('Maestro runScript requires a file.', commandNode, context); + const file = readRequiredString(entryValue(entries, 'file'), 'runScript.file', context); + const env = hasEntry(entries, 'env') + ? readScalarMap(entryValue(entries, 'env'), 'runScript.env', context) + : undefined; + return { + kind: 'runScript', + source, + file, + ...(env === undefined ? {} : { env }), + }; +} + +export function parseMaestroRunFlowCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, + parseCommands: MaestroCommandListParser, +): MaestroRunFlowCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) { + return { + kind: 'runFlow', + source, + include: { + kind: 'file', + path: readRequiredString(value, 'runFlow', context), + }, + }; + } + + const entries = readMapEntries(value, 'runFlow', context); + assertOnlyKeys(entries, 'runFlow', ['file', 'commands', 'env', 'when', 'label'], context); + const hasFile = hasEntry(entries, 'file'); + const hasCommands = hasEntry(entries, 'commands'); + if (hasFile === hasCommands) { + invalidAt('Maestro runFlow requires exactly one of file or commands.', commandNode, context); + } + const include = hasFile + ? { + kind: 'file' as const, + path: readRequiredString(entryValue(entries, 'file'), 'runFlow.file', context), + } + : { + kind: 'commands' as const, + commands: parseCommands(entryValue(entries, 'commands'), 'runFlow.commands', context), + }; + const env = hasEntry(entries, 'env') + ? readScalarMap(entryValue(entries, 'env'), 'runFlow.env', context) + : undefined; + const when = hasEntry(entries, 'when') + ? parseMaestroRunFlowCondition(entryValue(entries, 'when'), context) + : undefined; + const label = hasEntry(entries, 'label') + ? readOptionalString(entryValue(entries, 'label'), 'runFlow.label', context) + : undefined; + return { + kind: 'runFlow', + source, + include, + ...(env === undefined ? {} : { env }), + ...(when === undefined ? {} : { when }), + ...(label === undefined ? {} : { label }), + }; +} + +export function parseMaestroRepeatCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, + parseCommands: MaestroCommandListParser, +): MaestroRepeatCommand { + const source = sourceAt(commandNode, context); + const entries = readMapEntries(value, 'repeat', context); + assertOnlyKeys(entries, 'repeat', ['times', 'commands', 'while'], context); + if (hasEntry(entries, 'while')) { + invalidAt( + 'Maestro repeat.while is not supported; use repeat.times.', + entryValue(entries, 'while'), + context, + ); + } + if (!hasEntry(entries, 'times')) + invalidAt('Maestro repeat requires times.', commandNode, context); + if (!hasEntry(entries, 'commands')) + invalidAt('Maestro repeat requires commands.', commandNode, context); + return { + kind: 'repeat', + source, + times: readIntegerValue(entryValue(entries, 'times'), 'repeat.times', context), + commands: parseCommands(entryValue(entries, 'commands'), 'repeat.commands', context), + }; +} + +export function parseMaestroRetryCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, + parseCommands: MaestroCommandListParser, +): MaestroRetryCommand { + const source = sourceAt(commandNode, context); + const entries = readMapEntries(value, 'retry', context); + assertOnlyKeys(entries, 'retry', ['maxRetries', 'commands'], context); + if (!hasEntry(entries, 'commands')) + invalidAt('Maestro retry requires commands.', commandNode, context); + const maxRetries = hasEntry(entries, 'maxRetries') + ? readIntegerValue(entryValue(entries, 'maxRetries'), 'retry.maxRetries', context) + : undefined; + return { + kind: 'retry', + source, + commands: parseCommands(entryValue(entries, 'commands'), 'retry.commands', context), + ...(maxRetries === undefined ? {} : { maxRetries }), + }; +} + +function parseMaestroRunFlowCondition( + node: Node | null | undefined, + context: MaestroProgramParseContext, +): MaestroRunFlowCondition { + const entries = readMapEntries(node, 'runFlow.when', context); + assertOnlyKeys(entries, 'runFlow.when', ['platform', 'visible', 'notVisible', 'true'], context); + if (entries.length === 0) invalidAt('Maestro runFlow.when cannot be empty.', node, context); + + const platform = readOptionalEntry(entries, 'platform', (entry) => parsePlatform(entry, context)); + const visible = readOptionalEntry(entries, 'visible', (entry) => + parseMaestroSelector(entry, 'runFlow.when.visible', context), + ); + const notVisible = readOptionalEntry(entries, 'notVisible', (entry) => + parseMaestroSelector(entry, 'runFlow.when.notVisible', context), + ); + const truth = readOptionalEntry(entries, 'true', (entry) => readConditionTruth(entry, context)); + return { + ...(platform === undefined ? {} : { platform }), + ...(visible === undefined ? {} : { visible }), + ...(notVisible === undefined ? {} : { notVisible }), + ...(truth === undefined ? {} : { true: truth as boolean | string }), + }; +} + +function readConditionTruth( + node: Node | null | undefined, + context: MaestroProgramParseContext, +): boolean | string { + const value = readScalarValue(node, 'runFlow.when.true', context); + if (typeof value === 'boolean' || typeof value === 'string') return value; + invalidAt('Maestro runFlow.when.true expects a boolean or expression string.', node, context); +} + +function parsePlatform( + node: Node | null | undefined, + context: MaestroProgramParseContext, +): MaestroPlatform { + const value = readRequiredString(node, 'runFlow.when.platform', context).toLowerCase(); + if (value === 'android' || value === 'ios' || value === 'web') return value; + invalidAt('Maestro runFlow.when.platform expects Android, iOS, or Web.', node, context); +} diff --git a/src/compat/maestro/program-ir-gesture-parser.ts b/src/compat/maestro/program-ir-gesture-parser.ts new file mode 100644 index 000000000..18b333b6e --- /dev/null +++ b/src/compat/maestro/program-ir-gesture-parser.ts @@ -0,0 +1,432 @@ +import { isScalar, type Node } from 'yaml'; +import type { + MaestroCoordinate, + MaestroDirection, + MaestroDoubleTapOnCommand, + MaestroGestureTarget, + MaestroLongPressOnCommand, + MaestroSelector, + MaestroSelectorMap, + MaestroSourceLocation, + MaestroSwipeCommand, + MaestroTapOnCommand, +} from './program-ir.ts'; +import { + assertOnlyKeys, + entryValue, + hasEntry, + invalidAt, + readMapEntries, + readOptionalBoolean, + readOptionalEntry, + readOptionalNonNegativeInteger, + readOptionalNumber, + readOptionalString, + readRequiredPositiveInteger, + readRequiredString, + sourceAt, + type MaestroMapEntry, + type MaestroProgramParseContext, +} from './program-ir-values.ts'; + +const BASE_SELECTOR_KEYS = ['id', 'text', 'enabled', 'selected'] as const; +const TAP_SELECTOR_KEYS = [...BASE_SELECTOR_KEYS, 'label'] as const; +type SelectorKey = keyof MaestroSelectorMap; + +type SelectorFieldReader = ( + selector: MaestroSelectorMap, + entry: MaestroMapEntry, + name: string, + context: MaestroProgramParseContext, +) => void; + +const SELECTOR_FIELD_READERS: Readonly> = { + id: (selector, entry, name, context) => + assignStringSelector(selector, 'id', entry, name, context), + text: (selector, entry, name, context) => + assignStringSelector(selector, 'text', entry, name, context), + label: (selector, entry, name, context) => + assignStringSelector(selector, 'label', entry, name, context), + enabled: (selector, entry, name, context) => + assignBooleanSelector(selector, 'enabled', entry, name, context), + selected: (selector, entry, name, context) => + assignBooleanSelector(selector, 'selected', entry, name, context), +}; + +export function parseMaestroSelector( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, + selectorKeys: readonly SelectorKey[] = BASE_SELECTOR_KEYS, +): MaestroSelector { + if (isScalar(node)) return { text: readRequiredString(node, name, context) }; + const entries = readMapEntries(node, name, context); + assertOnlyKeys(entries, name, selectorKeys, context); + return parseSelectorEntries(entries, name, context); +} + +export function parseMaestroTapOnCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroTapOnCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) { + return { + kind: 'tapOn', + source, + target: selectorTarget(parseMaestroSelector(value, 'tapOn', context, TAP_SELECTOR_KEYS)), + }; + } + + const entries = readMapEntries(value, 'tapOn', context); + if (hasEntry(entries, 'point')) { + assertOnlyKeys(entries, 'tapOn', ['point', 'repeat', 'delay', 'optional', 'label'], context); + return { + kind: 'tapOn', + source, + target: parsePoint(entryValue(entries, 'point'), 'tapOn.point', context), + ...tapOptions(entries, context, true), + }; + } + + assertOnlyKeys( + entries, + 'tapOn', + [...TAP_SELECTOR_KEYS, 'repeat', 'delay', 'optional', 'index', 'childOf'], + context, + ); + const selectorEntries = entries.filter((entry) => isSelectorKey(entry.key, TAP_SELECTOR_KEYS)); + const childOf = hasEntry(entries, 'childOf') + ? parseMaestroSelector(entryValue(entries, 'childOf'), 'tapOn.childOf', context) + : undefined; + const options = tapOptions(entries, context, false); + const index = hasEntry(entries, 'index') + ? readOptionalNonNegativeInteger(entryValue(entries, 'index'), 'tapOn.index', context) + : undefined; + return { + kind: 'tapOn', + source, + target: selectorTarget(parseSelectorEntries(selectorEntries, 'tapOn', context)), + ...options, + ...(index === undefined ? {} : { index }), + ...(childOf === undefined ? {} : { childOf }), + }; +} + +export function parseMaestroDoubleTapOnCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroDoubleTapOnCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) { + return { + kind: 'doubleTapOn', + source, + target: selectorTarget(parseMaestroSelector(value, 'doubleTapOn', context)), + }; + } + const entries = readMapEntries(value, 'doubleTapOn', context); + assertOnlyKeys(entries, 'doubleTapOn', ['point', ...BASE_SELECTOR_KEYS, 'delay'], context); + const delay = hasEntry(entries, 'delay') + ? readOptionalNonNegativeInteger(entryValue(entries, 'delay'), 'doubleTapOn.delay', context) + : undefined; + if (hasEntry(entries, 'point')) { + if (entries.some((entry) => isSelectorKey(entry.key, BASE_SELECTOR_KEYS))) { + invalidAt( + 'Maestro doubleTapOn.point cannot be combined with a selector.', + commandNode, + context, + ); + } + return { + kind: 'doubleTapOn', + source, + target: parseAbsolutePoint(entryValue(entries, 'point'), 'doubleTapOn.point', context), + ...(delay === undefined ? {} : { delay }), + }; + } + return { + kind: 'doubleTapOn', + source, + target: selectorTarget( + parseSelectorEntries( + entries.filter((entry) => isSelectorKey(entry.key, BASE_SELECTOR_KEYS)), + 'doubleTapOn', + context, + ), + ), + ...(delay === undefined ? {} : { delay }), + }; +} + +export function parseMaestroLongPressOnCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroLongPressOnCommand { + const source = sourceAt(commandNode, context); + if (isScalar(value)) { + return { + kind: 'longPressOn', + source, + target: selectorTarget(parseMaestroSelector(value, 'longPressOn', context)), + }; + } + const entries = readMapEntries(value, 'longPressOn', context); + assertOnlyKeys(entries, 'longPressOn', ['point', ...BASE_SELECTOR_KEYS], context); + if (hasEntry(entries, 'point')) { + if (entries.some((entry) => isSelectorKey(entry.key, BASE_SELECTOR_KEYS))) { + invalidAt( + 'Maestro longPressOn.point cannot be combined with a selector.', + commandNode, + context, + ); + } + return { + kind: 'longPressOn', + source, + target: parseAbsolutePoint(entryValue(entries, 'point'), 'longPressOn.point', context), + }; + } + return { + kind: 'longPressOn', + source, + target: selectorTarget( + parseSelectorEntries( + entries.filter((entry) => isSelectorKey(entry.key, BASE_SELECTOR_KEYS)), + 'longPressOn', + context, + ), + ), + }; +} + +export function parseMaestroSwipeCommand( + value: Node | null, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroSwipeCommand { + const source = sourceAt(commandNode, context); + const entries = readMapEntries(value, 'swipe', context); + assertOnlyKeys( + entries, + 'swipe', + ['start', 'end', 'direction', 'duration', 'from', 'label'], + context, + ); + const duration = hasEntry(entries, 'duration') + ? readOptionalNumber(entryValue(entries, 'duration'), 'swipe.duration', context) + : undefined; + if (hasEntry(entries, 'start') || hasEntry(entries, 'end')) { + return parseCoordinateSwipe(entries, source, duration, commandNode, context); + } + const direction = hasEntry(entries, 'direction') + ? parseMaestroDirection(entryValue(entries, 'direction'), 'swipe.direction', context) + : undefined; + if (hasEntry(entries, 'from') || hasEntry(entries, 'label')) { + return parseTargetSwipe(entries, source, direction, duration, context); + } + return parseScreenSwipe(source, direction, duration, commandNode, context); +} + +function parseCoordinateSwipe( + entries: readonly MaestroMapEntry[], + source: MaestroSourceLocation, + duration: number | undefined, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroSwipeCommand { + if (hasEntry(entries, 'direction')) { + invalidAt( + 'Maestro swipe cannot combine direction with start/end coordinates.', + commandNode, + context, + ); + } + if (!hasEntry(entries, 'start') || !hasEntry(entries, 'end')) { + invalidAt('Maestro swipe requires both start and end coordinates.', commandNode, context); + } + const start = parsePoint(entryValue(entries, 'start'), 'swipe.start', context); + const end = parsePoint(entryValue(entries, 'end'), 'swipe.end', context); + if (start.space !== end.space) { + invalidAt('Maestro swipe start/end must use the same coordinate space.', commandNode, context); + } + return { + kind: 'swipe', + source, + gesture: { + kind: 'coordinates', + start, + end, + ...(duration === undefined ? {} : { duration }), + }, + }; +} + +function parseTargetSwipe( + entries: readonly MaestroMapEntry[], + source: MaestroSourceLocation, + direction: MaestroDirection | undefined, + duration: number | undefined, + context: MaestroProgramParseContext, +): MaestroSwipeCommand { + const label = hasEntry(entries, 'label') + ? readOptionalString(entryValue(entries, 'label'), 'swipe.label', context) + : undefined; + const from = hasEntry(entries, 'from') + ? parseMaestroSelector(entryValue(entries, 'from'), 'swipe.from', context) + : { text: label ?? '' }; + return { + kind: 'swipe', + source, + gesture: { + kind: 'target', + from, + ...(direction === undefined ? {} : { direction }), + ...(duration === undefined ? {} : { duration }), + ...(label === undefined ? {} : { label }), + }, + }; +} + +function parseScreenSwipe( + source: MaestroSourceLocation, + direction: MaestroDirection | undefined, + duration: number | undefined, + commandNode: Node, + context: MaestroProgramParseContext, +): MaestroSwipeCommand { + if (direction === undefined) { + invalidAt( + 'Maestro swipe requires direction, target, or start/end coordinates.', + commandNode, + context, + ); + } + return { + kind: 'swipe', + source, + gesture: { kind: 'screen', direction, ...(duration === undefined ? {} : { duration }) }, + }; +} + +function tapOptions( + entries: readonly MaestroMapEntry[], + context: MaestroProgramParseContext, + includeLabel: boolean, +): Pick { + const repeat = readOptionalEntry(entries, 'repeat', (entry) => + readRequiredPositiveInteger(entry, 'tapOn.repeat', context), + ); + const delay = readOptionalEntry(entries, 'delay', (entry) => + readOptionalNonNegativeInteger(entry, 'tapOn.delay', context), + ); + const optional = readOptionalEntry(entries, 'optional', (entry) => + readOptionalBoolean(entry, 'tapOn.optional', context), + ); + const label = includeLabel + ? readOptionalEntry(entries, 'label', (entry) => + readOptionalString(entry, 'tapOn.label', context), + ) + : undefined; + return { + ...(repeat === undefined ? {} : { repeat }), + ...(delay === undefined ? {} : { delay }), + ...(optional === undefined ? {} : { optional }), + ...(label === undefined ? {} : { label }), + }; +} + +function parseSelectorEntries( + entries: readonly MaestroMapEntry[], + name: string, + context: MaestroProgramParseContext, +): MaestroSelectorMap { + if (entries.length === 0) invalidAt(`Maestro ${name} selector is empty.`, undefined, context); + const selector: MaestroSelectorMap = {}; + for (const entry of entries) { + const read = SELECTOR_FIELD_READERS[entry.key]; + if (!read) { + invalidAt( + `Maestro ${name} selector field "${entry.key}" is not supported.`, + entry.keyNode, + context, + ); + } + read(selector, entry, name, context); + } + if (Object.keys(selector).length === 0) { + invalidAt( + `Maestro ${name} selector must contain a selector value.`, + entries[0]?.keyNode, + context, + ); + } + return selector; +} + +function assignStringSelector( + selector: MaestroSelectorMap, + key: 'id' | 'text' | 'label', + entry: MaestroMapEntry, + name: string, + context: MaestroProgramParseContext, +): void { + const value = readOptionalString(entry.value, `${name}.${key}`, context); + if (value !== undefined) selector[key] = value; +} + +function assignBooleanSelector( + selector: MaestroSelectorMap, + key: 'enabled' | 'selected', + entry: MaestroMapEntry, + name: string, + context: MaestroProgramParseContext, +): void { + const value = readOptionalBoolean(entry.value, `${name}.${key}`, context); + if (value !== undefined) selector[key] = value; +} + +function isSelectorKey(key: string, selectorKeys: readonly SelectorKey[]): key is SelectorKey { + return selectorKeys.includes(key as SelectorKey); +} + +function parsePoint( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroCoordinate { + const value = readRequiredString(node, name, context); + const absolute = /^\s*(\d+)\s*,\s*(\d+)\s*$/.exec(value); + if (absolute) return { space: 'absolute', x: Number(absolute[1]), y: Number(absolute[2]) }; + const percent = /^\s*(\d+(?:\.\d+)?)%\s*,\s*(\d+(?:\.\d+)?)%\s*$/.exec(value); + if (percent) return { space: 'percent', x: Number(percent[1]), y: Number(percent[2]) }; + invalidAt(`Maestro ${name} expects absolute or percentage coordinates.`, node, context); +} + +function parseAbsolutePoint( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroCoordinate { + const point = parsePoint(node, name, context); + if (point.space !== 'absolute') + invalidAt(`Maestro ${name} only supports absolute coordinates.`, node, context); + return point; +} + +export function parseMaestroDirection( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroDirection { + const value = readRequiredString(node, name, context).toLowerCase(); + if (value === 'up' || value === 'down' || value === 'left' || value === 'right') return value; + invalidAt(`Maestro ${name} must be UP, DOWN, LEFT, or RIGHT.`, node, context); +} + +function selectorTarget(selector: MaestroSelector): MaestroGestureTarget { + return { space: 'target', selector }; +} diff --git a/src/compat/maestro/program-ir-parser.ts b/src/compat/maestro/program-ir-parser.ts new file mode 100644 index 000000000..39082ede5 --- /dev/null +++ b/src/compat/maestro/program-ir-parser.ts @@ -0,0 +1,97 @@ +import { isMap, isSeq, LineCounter, parseAllDocuments, type Node } from 'yaml'; +import { AppError } from '../../kernel/errors.ts'; +import type { + MaestroProgram, + MaestroProgramConfig, + MaestroProgramParseOptions, +} from './program-ir.ts'; +import { parseMaestroCommandList } from './program-ir-command-parser.ts'; +import { + assertOnlyKeys, + invalidAt, + readMapEntries, + readOptionalString, + readOptionalEntry, + readScalarMap, + readStringSequence, + sourceAt, + type MaestroProgramParseContext, +} from './program-ir-values.ts'; + +export function parseMaestroProgram( + script: string, + options: MaestroProgramParseOptions = {}, +): MaestroProgram { + const lineCounter = new LineCounter(); + const documents = parseAllDocuments(script, { lineCounter }); + for (const document of documents) { + if (document.errors.length > 0) { + const message = document.errors[0]?.message ?? 'Invalid Maestro YAML flow.'; + throw new AppError('INVALID_ARGS', `Invalid Maestro YAML flow: ${message}`); + } + } + + const context: MaestroProgramParseContext = { + lineCounter, + ...(options.sourcePath === undefined ? {} : { sourcePath: options.sourcePath }), + }; + const contents = documents.map((document) => document.contents).filter((value) => value !== null); + if (contents.length === 0) throw new AppError('INVALID_ARGS', 'Maestro flow is empty.'); + + let configNode: Node | undefined; + let commandsNode: Node | undefined; + if (contents.length === 1 && isSeq(contents[0])) { + commandsNode = contents[0]; + } else if (contents.length === 2 && isMap(contents[0]) && isSeq(contents[1])) { + configNode = contents[0]; + commandsNode = contents[1]; + } else { + invalidAt( + 'Maestro flow must contain a command list, optionally preceded by one config document.', + contents[0], + context, + ); + } + + const config = configNode ? parseProgramConfig(configNode, context) : {}; + return { + kind: 'program', + source: sourceAt(configNode ?? commandsNode, context), + config, + commands: parseMaestroCommandList(commandsNode, 'commands', context), + }; +} + +function parseProgramConfig(node: Node, context: MaestroProgramParseContext): MaestroProgramConfig { + const entries = readMapEntries(node, 'flow config', context); + assertOnlyKeys( + entries, + 'flow config', + ['name', 'appId', 'tags', 'env', 'onFlowStart', 'onFlowComplete'], + context, + ); + const name = readOptionalEntry(entries, 'name', (entry) => + readOptionalString(entry, 'name', context), + ); + const appId = readOptionalEntry(entries, 'appId', (entry) => + readOptionalString(entry, 'appId', context), + ); + const tags = readOptionalEntry(entries, 'tags', (entry) => + readStringSequence(entry, 'tags', context), + ); + const env = readOptionalEntry(entries, 'env', (entry) => readScalarMap(entry, 'env', context)); + const onFlowStart = readOptionalEntry(entries, 'onFlowStart', (entry) => + parseMaestroCommandList(entry, 'onFlowStart', context), + ); + const onFlowComplete = readOptionalEntry(entries, 'onFlowComplete', (entry) => + parseMaestroCommandList(entry, 'onFlowComplete', context), + ); + return { + ...(name === undefined ? {} : { name }), + ...(appId === undefined ? {} : { appId }), + ...(tags === undefined ? {} : { tags }), + ...(env === undefined ? {} : { env }), + ...(onFlowStart === undefined ? {} : { onFlowStart }), + ...(onFlowComplete === undefined ? {} : { onFlowComplete }), + }; +} diff --git a/src/compat/maestro/program-ir-values.ts b/src/compat/maestro/program-ir-values.ts new file mode 100644 index 000000000..537e0164b --- /dev/null +++ b/src/compat/maestro/program-ir-values.ts @@ -0,0 +1,259 @@ +import { isMap, isNode, isScalar, isSeq, LineCounter, type Node } from 'yaml'; +import { AppError } from '../../kernel/errors.ts'; +import type { MaestroScalar, MaestroSourceLocation } from './program-ir.ts'; + +export type MaestroProgramParseContext = { + lineCounter: LineCounter; + sourcePath?: string; +}; + +export type MaestroMapEntry = { + key: string; + keyNode: Node; + value: Node | null; +}; + +export function sourceAt( + node: Node | null | undefined, + context: MaestroProgramParseContext, +): MaestroSourceLocation { + const offset = node?.range?.[0] ?? 0; + const line = context.lineCounter.linePos(offset).line; + return context.sourcePath === undefined ? { line } : { path: context.sourcePath, line }; +} + +export function invalidAt( + message: string, + node: Node | null | undefined, + context: MaestroProgramParseContext, +): never { + throw new AppError('INVALID_ARGS', `${message} (line ${sourceAt(node, context).line})`); +} + +export function readMapEntries( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroMapEntry[] { + if (!isMap(node)) invalidAt(`Maestro ${name} expects a map.`, node, context); + const entries: MaestroMapEntry[] = []; + for (const pair of node.items) { + const keyNode = pair.key; + if (!isScalar(keyNode)) { + invalidAt(`Maestro ${name} map keys must be strings.`, undefined, context); + } + const keyValue = keyNode.value; + if ( + typeof keyValue !== 'string' && + typeof keyValue !== 'number' && + typeof keyValue !== 'boolean' + ) { + invalidAt(`Maestro ${name} map keys must be strings.`, keyNode, context); + } + if (pair.value !== null && !isNode(pair.value)) { + invalidAt(`Maestro ${name} values must be YAML nodes.`, keyNode, context); + } + entries.push({ + key: String(keyValue), + keyNode, + value: pair.value as Node | null, + }); + } + return entries; +} + +export function readSequenceItems( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): Node[] { + if (!isSeq(node)) invalidAt(`Maestro ${name} expects a list.`, node, context); + return node.items as Node[]; +} + +export function readStringSequence( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): string[] { + return readSequenceItems(node, name, context).map((item, index) => + readRequiredString(item, `${name}[${index}]`, context), + ); +} + +export function assertOnlyKeys( + entries: readonly MaestroMapEntry[], + name: string, + supportedKeys: readonly string[], + context: MaestroProgramParseContext, +): void { + const supported = new Set(supportedKeys); + const seen = new Set(); + for (const entry of entries) { + if (seen.has(entry.key)) { + invalidAt(`Maestro ${name} contains duplicate field "${entry.key}".`, entry.keyNode, context); + } + seen.add(entry.key); + if (!supported.has(entry.key)) { + invalidAt(`Maestro ${name} field "${entry.key}" is not supported.`, entry.keyNode, context); + } + } +} + +export function hasEntry(entries: readonly MaestroMapEntry[], key: string): boolean { + return entries.some((entry) => entry.key === key); +} + +export function entryValue( + entries: readonly MaestroMapEntry[], + key: string, +): Node | null | undefined { + return entries.find((entry) => entry.key === key)?.value; +} + +export function readOptionalEntry( + entries: readonly MaestroMapEntry[], + key: string, + read: (value: Node | null | undefined) => T, +): T | undefined { + return hasEntry(entries, key) ? read(entryValue(entries, key)) : undefined; +} + +export function isNullNode(node: Node | null | undefined): boolean { + return node === null || (isScalar(node) && node.value === null); +} + +export function readScalarValue( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): MaestroScalar | null { + if (node === null || node === undefined) return null; + if (!isScalar(node)) invalidAt(`Maestro ${name} expects a scalar value.`, node, context); + const value = node.value; + if ( + value === null || + typeof value === 'string' || + typeof value === 'number' || + typeof value === 'boolean' + ) { + return value; + } + invalidAt(`Maestro ${name} contains an unsupported scalar value.`, node, context); +} + +export function readRequiredString( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): string { + const value = readScalarValue(node, name, context); + if (typeof value !== 'string') invalidAt(`Maestro ${name} expects a string.`, node, context); + return value; +} + +export function readOptionalString( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): string | undefined { + const value = readScalarValue(node, name, context); + if (value === null) return undefined; + if (typeof value !== 'string') invalidAt(`Maestro ${name} expects a string.`, node, context); + return value; +} + +export function readOptionalBoolean( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): boolean | undefined { + const value = readScalarValue(node, name, context); + if (value === null) return undefined; + if (typeof value !== 'boolean') invalidAt(`Maestro ${name} expects a boolean.`, node, context); + return value; +} + +export function readOptionalNumber( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): number | undefined { + const value = readScalarValue(node, name, context); + if (value === null) return undefined; + if (typeof value !== 'number' || !Number.isFinite(value)) { + invalidAt(`Maestro ${name} expects a finite number.`, node, context); + } + return value; +} + +export function readRequiredPositiveInteger( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): number { + const value = readScalarValue(node, name, context); + if (typeof value !== 'number' || !Number.isInteger(value) || value <= 0) { + invalidAt(`Maestro ${name} expects a positive integer.`, node, context); + } + return value; +} + +export function readOptionalNonNegativeInteger( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): number | undefined { + const value = readScalarValue(node, name, context); + if (value === null) return undefined; + if (typeof value !== 'number' || !Number.isInteger(value) || value < 0) { + invalidAt(`Maestro ${name} expects a non-negative integer.`, node, context); + } + return value; +} + +export function readIntegerValue( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): number | string { + const value = readScalarValue(node, name, context); + if (typeof value === 'number') { + if (!Number.isInteger(value) || value < 0) { + invalidAt(`Maestro ${name} expects a non-negative integer.`, node, context); + } + return value; + } + if ( + typeof value === 'string' && + (/^\d+$/.test(value) || /^\$\{[A-Za-z_][A-Za-z0-9_.]*\}$/.test(value)) + ) { + return value; + } + invalidAt( + `Maestro ${name} expects a non-negative integer or a variable expression.`, + node, + context, + ); +} + +export function readScalarMap( + node: Node | null | undefined, + name: string, + context: MaestroProgramParseContext, +): Record { + const entries = readMapEntries(node, name, context); + const values: Record = {}; + for (const entry of entries) { + const value = readScalarValue(entry.value, `${name}.${entry.key}`, context); + if (value === null) { + invalidAt( + `Maestro ${name}.${entry.key} expects a string, number, or boolean.`, + entry.value, + context, + ); + } + values[entry.key] = value; + } + return values; +} diff --git a/src/compat/maestro/program-ir.ts b/src/compat/maestro/program-ir.ts new file mode 100644 index 000000000..b9f1144f0 --- /dev/null +++ b/src/compat/maestro/program-ir.ts @@ -0,0 +1,273 @@ +export type MaestroScalar = string | number | boolean | null; + +export type MaestroSourceLocation = { + path?: string; + line: number; +}; + +export type MaestroSource = MaestroSourceLocation; + +export type MaestroPlatform = 'android' | 'ios' | 'web'; + +export type MaestroDirection = 'up' | 'down' | 'left' | 'right'; + +export type MaestroCoordinate = + | { space: 'absolute'; x: number; y: number } + | { space: 'percent'; x: number; y: number }; + +export type MaestroPoint = MaestroCoordinate; + +export type MaestroSelectorMap = { + text?: string; + id?: string; + label?: string; + enabled?: boolean; + selected?: boolean; +}; + +export type MaestroSelector = MaestroSelectorMap; + +export type MaestroGestureTarget = + | MaestroCoordinate + | { space: 'target'; selector: MaestroSelector }; + +export type MaestroLaunchArguments = + | { kind: 'scalar'; value: string | number | boolean } + | { kind: 'list'; values: Array } + | { kind: 'map'; values: Record }; + +export type MaestroLaunchAppCommand = { + kind: 'launchApp'; + source: MaestroSourceLocation; + appId?: string; + stopApp?: boolean; + clearState?: boolean; + arguments?: MaestroLaunchArguments; + launchArguments?: MaestroLaunchArguments; +}; + +export type MaestroTapOnCommand = { + kind: 'tapOn'; + source: MaestroSourceLocation; + target: MaestroGestureTarget; + repeat?: number; + delay?: number; + optional?: boolean; + label?: string; + index?: number; + childOf?: MaestroSelector; +}; + +export type MaestroDoubleTapOnCommand = { + kind: 'doubleTapOn'; + source: MaestroSourceLocation; + target: MaestroGestureTarget; + delay?: number; +}; + +export type MaestroLongPressOnCommand = { + kind: 'longPressOn'; + source: MaestroSourceLocation; + target: MaestroGestureTarget; +}; + +export type MaestroSwipeGesture = + | { + kind: 'coordinates'; + start: MaestroCoordinate; + end: MaestroCoordinate; + duration?: number; + } + | { + kind: 'screen'; + direction: MaestroDirection; + duration?: number; + } + | { + kind: 'target'; + from: MaestroSelector; + direction?: MaestroDirection; + duration?: number; + label?: string; + }; + +export type MaestroSwipeCommand = { + kind: 'swipe'; + source: MaestroSourceLocation; + gesture: MaestroSwipeGesture; +}; + +export type MaestroInputTextCommand = { + kind: 'inputText'; + source: MaestroSourceLocation; + text: string; + label?: string; +}; + +export type MaestroEraseTextCommand = { + kind: 'eraseText'; + source: MaestroSourceLocation; + charactersToErase?: number; +}; + +export type MaestroPasteTextCommand = { + kind: 'pasteText'; + source: MaestroSourceLocation; + text: string; +}; + +export type MaestroOpenLinkCommand = { + kind: 'openLink'; + source: MaestroSourceLocation; + link: string; +}; + +export type MaestroAssertVisibleCommand = { + kind: 'assertVisible'; + source: MaestroSourceLocation; + target: MaestroSelector; +}; + +export type MaestroAssertNotVisibleCommand = { + kind: 'assertNotVisible'; + source: MaestroSourceLocation; + target: MaestroSelector; +}; + +export type MaestroExtendedWaitUntilCommand = { + kind: 'extendedWaitUntil'; + source: MaestroSourceLocation; + visible?: MaestroSelector; + notVisible?: MaestroSelector; + timeout?: number; +}; + +export type MaestroTakeScreenshotCommand = { + kind: 'takeScreenshot'; + source: MaestroSourceLocation; + path: string; +}; + +export type MaestroScrollCommand = { + kind: 'scroll'; + source: MaestroSourceLocation; +}; + +export type MaestroScrollUntilVisibleCommand = { + kind: 'scrollUntilVisible'; + source: MaestroSourceLocation; + element: MaestroSelector; + direction?: MaestroDirection; + timeout?: number; +}; + +export type MaestroHideKeyboardCommand = { + kind: 'hideKeyboard'; + source: MaestroSourceLocation; +}; + +export type MaestroPressKeyCommand = { + kind: 'pressKey'; + source: MaestroSourceLocation; + key: 'back' | 'enter' | 'return' | 'home'; +}; + +export type MaestroBackCommand = { + kind: 'back'; + source: MaestroSourceLocation; +}; + +export type MaestroWaitForAnimationToEndCommand = { + kind: 'waitForAnimationToEnd'; + source: MaestroSourceLocation; + timeout?: number; +}; + +export type MaestroStopAppCommand = { + kind: 'stopApp'; + source: MaestroSourceLocation; + appId?: string; +}; + +export type MaestroRunScriptCommand = { + kind: 'runScript'; + source: MaestroSourceLocation; + file: string; + env?: Record; +}; + +export type MaestroRunFlowCondition = { + platform?: MaestroPlatform; + visible?: MaestroSelector; + notVisible?: MaestroSelector; + true?: boolean | string; +}; + +export type MaestroRunFlowCommand = { + kind: 'runFlow'; + source: MaestroSourceLocation; + include: { kind: 'file'; path: string } | { kind: 'commands'; commands: MaestroCommand[] }; + when?: MaestroRunFlowCondition; + env?: Record; + label?: string; +}; + +export type MaestroRepeatCommand = { + kind: 'repeat'; + source: MaestroSourceLocation; + times: number | string; + commands: MaestroCommand[]; +}; + +export type MaestroRetryCommand = { + kind: 'retry'; + source: MaestroSourceLocation; + maxRetries?: number | string; + commands: MaestroCommand[]; +}; + +export type MaestroCommand = + | MaestroLaunchAppCommand + | MaestroTapOnCommand + | MaestroDoubleTapOnCommand + | MaestroLongPressOnCommand + | MaestroSwipeCommand + | MaestroInputTextCommand + | MaestroEraseTextCommand + | MaestroPasteTextCommand + | MaestroOpenLinkCommand + | MaestroAssertVisibleCommand + | MaestroAssertNotVisibleCommand + | MaestroExtendedWaitUntilCommand + | MaestroTakeScreenshotCommand + | MaestroScrollCommand + | MaestroScrollUntilVisibleCommand + | MaestroHideKeyboardCommand + | MaestroPressKeyCommand + | MaestroBackCommand + | MaestroWaitForAnimationToEndCommand + | MaestroStopAppCommand + | MaestroRunScriptCommand + | MaestroRunFlowCommand + | MaestroRepeatCommand + | MaestroRetryCommand; + +export type MaestroProgramConfig = { + name?: string; + appId?: string; + tags?: string[]; + env?: Record; + onFlowStart?: MaestroCommand[]; + onFlowComplete?: MaestroCommand[]; +}; + +export type MaestroProgram = { + kind: 'program'; + source: MaestroSourceLocation; + config: MaestroProgramConfig; + commands: MaestroCommand[]; +}; + +export type MaestroProgramParseOptions = { + sourcePath?: string; +}; diff --git a/src/compat/maestro/program-loader.ts b/src/compat/maestro/program-loader.ts new file mode 100644 index 000000000..d042fd1a2 --- /dev/null +++ b/src/compat/maestro/program-loader.ts @@ -0,0 +1,35 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { createRequestCanceledError } from '../../request/cancel.ts'; +import type { MaestroProgram } from './program-ir.ts'; +import { parseMaestroProgram } from './program-ir-parser.ts'; + +export type MaestroProgramLoader = ( + includePath: string, + parentSource?: string, + signal?: AbortSignal, +) => Promise; + +export function createMaestroProgramLoader(cwd: string): MaestroProgramLoader { + const programs = new Map(); + return async (includePath, parentSource, signal) => { + if (signal?.aborted) throw createRequestCanceledError(); + const resolvedPath = resolveMaestroIncludePath(includePath, parentSource, cwd); + const cached = programs.get(resolvedPath); + if (cached) return cached; + const program = parseMaestroProgram(fs.readFileSync(resolvedPath, 'utf8'), { + sourcePath: resolvedPath, + }); + programs.set(resolvedPath, program); + return program; + }; +} + +export function resolveMaestroIncludePath( + includePath: string, + parentSource: string | undefined, + cwd: string, +): string { + const basePath = parentSource ? path.dirname(parentSource) : cwd; + return path.resolve(basePath, includePath); +} diff --git a/src/compat/maestro/progress.ts b/src/compat/maestro/progress.ts new file mode 100644 index 000000000..209bfe256 --- /dev/null +++ b/src/compat/maestro/progress.ts @@ -0,0 +1,127 @@ +import type { MaestroCommand, MaestroGestureTarget, MaestroSelector } from './program-ir.ts'; + +export type MaestroCommandProgress = { + command: string; + value?: string; +}; + +export function formatMaestroCommandProgress(command: MaestroCommand): MaestroCommandProgress { + return { + command: command.kind, + ...progressValue(command), + }; +} + +function progressValue(command: MaestroCommand): Pick { + if (isGestureTargetCommand(command)) return valueOf(formatGestureTarget(command.target)); + if (isSelectorProgressCommand(command)) return selectorProgressValue(command); + if (command.kind === 'inputText' || command.kind === 'pasteText') return valueOf(''); + return commandDetailProgressValue(command); +} + +type GestureTargetCommand = Extract< + MaestroCommand, + { kind: 'tapOn' | 'doubleTapOn' | 'longPressOn' } +>; + +function isGestureTargetCommand(command: MaestroCommand): command is GestureTargetCommand { + return ( + command.kind === 'tapOn' || command.kind === 'doubleTapOn' || command.kind === 'longPressOn' + ); +} + +type SelectorProgressCommand = Extract< + MaestroCommand, + { kind: 'assertVisible' | 'assertNotVisible' | 'extendedWaitUntil' | 'scrollUntilVisible' } +>; + +function isSelectorProgressCommand(command: MaestroCommand): command is SelectorProgressCommand { + return ( + command.kind === 'assertVisible' || + command.kind === 'assertNotVisible' || + command.kind === 'extendedWaitUntil' || + command.kind === 'scrollUntilVisible' + ); +} + +function selectorProgressValue( + command: SelectorProgressCommand, +): Pick { + switch (command.kind) { + case 'assertVisible': + case 'assertNotVisible': + return valueOf(formatSelector(command.target)); + case 'extendedWaitUntil': + return valueOf(formatSelector(command.visible ?? command.notVisible)); + case 'scrollUntilVisible': + return valueOf(formatSelector(command.element)); + } +} + +function commandDetailProgressValue( + command: Exclude, +): Pick { + if (isSimpleDetailCommand(command)) return simpleDetailProgressValue(command); + if (command.kind === 'swipe') return swipeProgressValue(command); + if (command.kind === 'runFlow') { + return valueOf(command.label ?? (command.include.kind === 'file' ? command.include.path : '')); + } + return {}; +} + +type SimpleDetailCommand = Extract< + MaestroCommand, + { kind: 'openLink' | 'takeScreenshot' | 'runScript' | 'pressKey' } +>; + +function isSimpleDetailCommand(command: MaestroCommand): command is SimpleDetailCommand { + return ( + command.kind === 'openLink' || + command.kind === 'takeScreenshot' || + command.kind === 'runScript' || + command.kind === 'pressKey' + ); +} + +function simpleDetailProgressValue( + command: SimpleDetailCommand, +): Pick { + switch (command.kind) { + case 'openLink': + return valueOf(command.link); + case 'takeScreenshot': + return valueOf(command.path); + case 'runScript': + return valueOf(command.file); + case 'pressKey': + return valueOf(command.key); + } +} + +function swipeProgressValue( + command: Extract, +): Pick { + return valueOf( + command.gesture.kind === 'coordinates' + ? `${formatCoordinate(command.gesture.start)} to ${formatCoordinate(command.gesture.end)}` + : command.gesture.direction, + ); +} + +function formatGestureTarget(target: MaestroGestureTarget): string | undefined { + return target.space === 'target' + ? formatSelector(target.selector) + : `${target.x},${target.y}${target.space === 'percent' ? '%' : ''}`; +} + +function formatSelector(selector: MaestroSelector | undefined): string | undefined { + return selector?.id ?? selector?.text ?? selector?.label; +} + +function formatCoordinate(coordinate: { space: 'absolute' | 'percent'; x: number; y: number }) { + return `${coordinate.x},${coordinate.y}${coordinate.space === 'percent' ? '%' : ''}`; +} + +function valueOf(value: string | undefined): Pick { + return value ? { value } : {}; +} diff --git a/src/compat/maestro/replay-flow.ts b/src/compat/maestro/replay-flow.ts deleted file mode 100644 index 929a97e81..000000000 --- a/src/compat/maestro/replay-flow.ts +++ /dev/null @@ -1,332 +0,0 @@ -import fs from 'node:fs'; -import path from 'node:path'; -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { convertMaestroCommandWithLine } from './command-mapper.ts'; -import { parseMaestroYamlDocuments } from './flow-yaml.ts'; -import { MAESTRO_RUNTIME_COMMAND } from './runtime-commands.ts'; -import { isPlainRecord, normalizeCommandList, normalizePlatform, readEnvMap } from './support.ts'; -import type { - MaestroCommand, - MaestroFlowConfig, - MaestroParseContext, - MaestroParseOptions, - MaestroReplayFlow, -} from './types.ts'; - -export function parseMaestroReplayFlow( - script: string, - options: MaestroParseOptions = {}, -): MaestroReplayFlow { - return parseMaestroReplayFlowInternal(script, createParseContext(options)); -} - -export function readMaestroFlowName(script: string): string | undefined { - const values = parseMaestroYamlDocuments(script); - const { config } = splitMaestroDocuments(values); - return config.name; -} - -function parseMaestroReplayFlowInternal( - script: string, - context: MaestroParseContext, -): MaestroReplayFlow { - const values = parseMaestroYamlDocuments(script); - const { config, commands } = splitMaestroDocuments(values); - const nextContext = { - ...context, - env: { ...context.env, ...(config.env ?? {}), ...context.envOverrides }, - }; - const commandLines = findMaestroCommandLines(script); - const { actions, actionLines, actionSourcePaths } = convertRootCommands({ - config, - commands, - commandLines, - context: nextContext, - }); - - return { - actions, - actionLines, - actionSourcePaths, - metadata: { - env: config.env, - }, - }; -} - -type ConvertedFlowActions = { - actions: SessionAction[]; - actionLines: number[]; - actionSourcePaths: (string | undefined)[]; -}; - -function convertRootCommands(params: { - config: MaestroFlowConfig; - commands: MaestroCommand[]; - commandLines: number[]; - context: MaestroParseContext; -}): ConvertedFlowActions { - const { config, commands, commandLines, context } = params; - const allCommands = [ - ...(config.onFlowStart ?? []), - ...commands, - ...(config.onFlowComplete ?? []), - ]; - const allCommandLines = buildRootCommandLines(config, commandLines); - - const actions: SessionAction[] = []; - const actionLines: number[] = []; - const actionSourcePaths: (string | undefined)[] = []; - for (const [index, command] of allCommands.entries()) { - const line = allCommandLines[index] ?? index + 1; - const converted = convertMaestroCommandWithLine(command, config, line, context, { - parseRunFlowFile, - }); - for (const [actionIndex, action] of converted.actions.entries()) { - const source = converted.sources[actionIndex]; - actions.push(action); - actionLines.push(source?.line ?? line); - actionSourcePaths.push(source?.path); - } - } - return optimizeInputTextActions(actions, actionLines, actionSourcePaths); -} - -/** `onFlowStart` commands report line 1; `onFlowComplete` commands report the script's last command line — both hooks live outside the numbered command list. */ -function buildRootCommandLines(config: MaestroFlowConfig, commandLines: number[]): number[] { - return [ - ...Array.from({ length: config.onFlowStart?.length ?? 0 }, () => 1), - ...commandLines, - ...Array.from({ length: config.onFlowComplete?.length ?? 0 }, () => commandLines.at(-1) ?? 1), - ]; -} - -function optimizeInputTextActions( - actions: SessionAction[], - actionLines: number[], - actionSourcePaths: (string | undefined)[], -): ConvertedFlowActions { - const mergedActions: SessionAction[] = []; - const mergedLines: number[] = []; - const mergedSourcePaths: (string | undefined)[] = []; - for (let index = 0; index < actions.length; index += 1) { - const action = actions[index]!; - const optimized = optimizeTypedAfterTap(actions, actionLines, actionSourcePaths, index); - if (optimized) { - mergedActions.push(...optimized.actions); - mergedLines.push(...optimized.actionLines); - mergedSourcePaths.push(...optimized.actionSourcePaths); - index += optimized.consumed - 1; - continue; - } - mergedActions.push(action); - mergedLines.push(actionLines[index] ?? 1); - mergedSourcePaths.push(actionSourcePaths[index]); - } - return { actions: mergedActions, actionLines: mergedLines, actionSourcePaths: mergedSourcePaths }; -} - -function optimizeTypedAfterTap( - actions: SessionAction[], - actionLines: number[], - actionSourcePaths: (string | undefined)[], - index: number, -): (ConvertedFlowActions & { consumed: number }) | null { - const candidate = readTypedAfterTapCandidate(actions, actionLines, index); - if (!candidate) return null; - const { action, nextAction, pressEnterAction, tapSelector, typedAfterTap, line } = candidate; - const sourcePath = actionSourcePaths[index]; - if (!isLikelyTextEntrySelector(tapSelector)) { - return { - actions: [clearMaestroNonHittableTap(action)], - actionLines: [line], - actionSourcePaths: [sourcePath], - consumed: 1, - }; - } - return { - actions: [ - { - ...action, - command: 'wait', - positionals: [tapSelector, '30000'], - }, - { - ...nextAction, - command: 'fill', - positionals: [tapSelector, typedAfterTap], - flags: action.flags, - }, - pressEnterAction, - ], - actionLines: [line, line, actionLines[index + 2] ?? line], - actionSourcePaths: [sourcePath, sourcePath, actionSourcePaths[index + 2]], - consumed: 3, - }; -} - -function readTypedAfterTapCandidate( - actions: SessionAction[], - actionLines: number[], - index: number, -): { - action: SessionAction; - nextAction: SessionAction; - pressEnterAction: SessionAction; - tapSelector: string; - typedAfterTap: string; - line: number; -} | null { - const action = actions[index]!; - const nextAction = actions[index + 1]; - const pressEnterAction = actions[index + 2]; - if (pressEnterAction?.command !== MAESTRO_RUNTIME_COMMAND.pressEnter) return null; - if (action.flags?.maestro?.optional === true) return null; - const typedAfterTap = readPlainTypeText(nextAction); - const tapSelector = readPlainMaestroTapSelector(action); - if (!nextAction || typedAfterTap === null || tapSelector === null) return null; - return { - action, - nextAction, - pressEnterAction, - tapSelector, - typedAfterTap, - line: actionLines[index] ?? 1, - }; -} - -function clearMaestroNonHittableTap(action: SessionAction): SessionAction { - const maestro = { ...(action.flags?.maestro ?? {}) }; - delete maestro.allowNonHittableCoordinateFallback; - return { - ...action, - flags: { - ...(action.flags ?? {}), - maestro: { - ...maestro, - }, - }, - }; -} - -function readPlainMaestroTapSelector(action: SessionAction | undefined): string | null { - if (action?.command !== MAESTRO_RUNTIME_COMMAND.tapOn) return null; - const [selector, ...rest] = action.positionals ?? []; - if (rest.length > 0 || typeof selector !== 'string') return null; - return selector; -} - -function readPlainTypeText(action: SessionAction | undefined): string | null { - if (action?.command !== 'type') return null; - if (action.flags && Object.keys(action.flags).length > 0) return null; - const [text, ...rest] = action.positionals ?? []; - if (rest.length > 0 || typeof text !== 'string') return null; - return text; -} - -function isLikelyTextEntrySelector(selector: string): boolean { - return /\b(input|textfield|textarea|field|email|password|username|search|query)\b/i.test( - selector.replace(/([a-z])([A-Z])/g, '$1 $2'), - ); -} - -function createParseContext(options: MaestroParseOptions): MaestroParseContext { - const visitedPaths = options.visitedPaths ?? new Set(); - if (options.sourcePath) visitedPaths.add(path.resolve(options.sourcePath)); - return { - baseDir: options.sourcePath ? path.dirname(options.sourcePath) : undefined, - platform: normalizePlatform(options.platform), - env: {}, - envOverrides: options.env ?? {}, - visitedPaths, - }; -} - -function findMaestroCommandLines(script: string): number[] { - const lines = script.split(/\r?\n/); - const separatorIndex = lines.findIndex((line) => line.trim() === '---'); - const firstCommandLine = separatorIndex === -1 ? 0 : separatorIndex + 1; - const commandLines: number[] = []; - for (let index = firstCommandLine; index < lines.length; index += 1) { - if (/^-\s+/.test(lines[index] ?? '')) commandLines.push(index + 1); - } - return commandLines; -} - -function splitMaestroDocuments(values: unknown[]): { - config: MaestroFlowConfig; - commands: MaestroCommand[]; -} { - if (values.length === 0) { - throw new AppError('INVALID_ARGS', 'Maestro flow is empty.'); - } - - if (Array.isArray(values[0])) { - return { config: {}, commands: normalizeCommandList(values[0]) }; - } - - const config = normalizeConfig(values[0]); - const commandDocument = values[1]; - if (!Array.isArray(commandDocument)) { - throw new AppError( - 'INVALID_ARGS', - 'Maestro flow must contain a command list after the YAML document separator.', - ); - } - return { config, commands: normalizeCommandList(commandDocument) }; -} - -function normalizeConfig(value: unknown): MaestroFlowConfig { - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'Maestro flow config must be a YAML map.'); - } - return { - ...(typeof value.name === 'string' && value.name.length > 0 ? { name: value.name } : {}), - ...(typeof value.appId === 'string' && value.appId.length > 0 ? { appId: value.appId } : {}), - ...(isPlainRecord(value.env) ? { env: readEnvMap(value.env, 'env') } : {}), - ...(Array.isArray(value.onFlowStart) - ? { onFlowStart: normalizeCommandList(value.onFlowStart) } - : {}), - ...(Array.isArray(value.onFlowComplete) - ? { onFlowComplete: normalizeCommandList(value.onFlowComplete) } - : {}), - }; -} - -/** - * Parses a `runFlow`-included file, resolving `undefined` (this-file) source - * paths to the include's own resolved path; deeper nested includes keep the - * path a recursive call already stamped. - */ -function parseRunFlowFile(filePath: string, context: MaestroParseContext): MaestroReplayFlow { - const resolved = resolveRunFlowPath(filePath, context); - if (context.visitedPaths.has(resolved)) { - throw new AppError('INVALID_ARGS', `Maestro runFlow cycle detected at ${resolved}.`); - } - const script = fs.readFileSync(resolved, 'utf8'); - const visitedPaths = new Set(context.visitedPaths); - visitedPaths.add(resolved); - const flow = parseMaestroReplayFlowInternal(script, { - ...context, - baseDir: path.dirname(resolved), - visitedPaths, - }); - return { - ...flow, - actionSourcePaths: (flow.actionSourcePaths ?? flow.actions.map(() => undefined)).map( - (sourcePath) => sourcePath ?? resolved, - ), - }; -} - -function resolveRunFlowPath(filePath: string, context: MaestroParseContext): string { - if (path.isAbsolute(filePath)) return filePath; - if (!context.baseDir) { - throw new AppError( - 'INVALID_ARGS', - 'runFlow file paths require replay input to have a source path.', - ); - } - return path.resolve(context.baseDir, filePath); -} diff --git a/src/compat/maestro/replay-plan-compilation.ts b/src/compat/maestro/replay-plan-compilation.ts new file mode 100644 index 000000000..619a2a3de --- /dev/null +++ b/src/compat/maestro/replay-plan-compilation.ts @@ -0,0 +1,58 @@ +import { computeMaestroReplayPlanDigest } from './replay-plan-digest.ts'; +import type { MaestroProgram } from './program-ir.ts'; +import { compileMaestroReplayPlanSteps } from './replay-plan-steps.ts'; +import type { MaestroReplayPlan, MaestroReplayPlanOptions } from './replay-plan-types.ts'; +import type { SessionRuntimeHints } from '../../kernel/contracts.ts'; + +export async function compileMaestroReplayPlan( + program: MaestroProgram, + options: MaestroReplayPlanOptions = {}, +): Promise { + const { steps, staticallyExecutedControls, staticallySkippedControls } = + await compileMaestroReplayPlanSteps(program, options); + const runtimeHints = normalizeRuntimeHints(options.runtimeHints); + const planWithoutDigest = { + kind: 'maestroReplayPlan' as const, + ...(options.platform === undefined ? {} : { platform: options.platform }), + ...(options.target === undefined ? {} : { target: options.target }), + ...(runtimeHints === undefined ? {} : { runtimeHints }), + initialStaticEnv: cloneValue({ + ...(options.defaults ?? {}), + ...(options.builtins ?? {}), + ...(program.config.env ?? {}), + ...(options.env ?? {}), + }), + steps: cloneValue(steps), + total: steps.length, + compatibility: { + staticallyExecutedControls, + staticallySkippedControls, + }, + }; + const digest = computeMaestroReplayPlanDigest(planWithoutDigest); + return freezeDeep({ ...planWithoutDigest, digest }); +} + +function normalizeRuntimeHints( + hints: Readonly | undefined, +): Readonly | undefined { + if (!hints) return undefined; + const entries = Object.entries(hints).filter((entry) => entry[1] !== undefined); + return entries.length === 0 ? undefined : Object.fromEntries(entries); +} + +function cloneValue(value: T): T { + if (Array.isArray(value)) return value.map((entry) => cloneValue(entry)) as T; + if (value && typeof value === 'object') { + return Object.fromEntries( + Object.entries(value).map(([key, entry]) => [key, cloneValue(entry)]), + ) as T; + } + return value; +} + +function freezeDeep(value: T): T { + if (!value || typeof value !== 'object' || Object.isFrozen(value)) return value; + for (const child of Object.values(value as Record)) freezeDeep(child); + return Object.freeze(value); +} diff --git a/src/compat/maestro/replay-plan-digest.ts b/src/compat/maestro/replay-plan-digest.ts new file mode 100644 index 000000000..3251da399 --- /dev/null +++ b/src/compat/maestro/replay-plan-digest.ts @@ -0,0 +1,30 @@ +import { createHash } from 'node:crypto'; + +export function computeMaestroReplayPlanDigest(plan: { + readonly platform?: string; + readonly target?: string; + readonly runtimeHints?: Readonly>; + readonly initialStaticEnv: Readonly>; + readonly steps: readonly unknown[]; +}): string { + const canonical = { + version: 2, + platform: plan.platform ?? null, + target: plan.target ?? null, + runtimeHints: plan.runtimeHints ?? null, + initialStaticEnv: plan.initialStaticEnv, + steps: plan.steps, + }; + return createHash('sha256') + .update(JSON.stringify(sortKeysDeep(canonical)), 'utf8') + .digest('hex'); +} + +function sortKeysDeep(value: unknown): unknown { + if (Array.isArray(value)) return value.map(sortKeysDeep); + if (!value || typeof value !== 'object') return value; + const record = value as Record; + const sorted: Record = {}; + for (const key of Object.keys(record).sort()) sorted[key] = sortKeysDeep(record[key]); + return sorted; +} diff --git a/src/compat/maestro/replay-plan-execution.ts b/src/compat/maestro/replay-plan-execution.ts new file mode 100644 index 000000000..5a9b977a8 --- /dev/null +++ b/src/compat/maestro/replay-plan-execution.ts @@ -0,0 +1,96 @@ +import { AppError } from '../../kernel/errors.ts'; +import { createMaestroExecutionContext } from './engine-context.ts'; +import { checkpointMaestroCancellation, resolveMaestroTimingPolicy } from './engine-flow.ts'; +import type { + MaestroEngineOptions, + MaestroEngineResult, + MaestroRuntimePort, +} from './engine-types.ts'; +import type { MaestroReplayPlan } from './replay-plan-types.ts'; +import { + asMaestroReplayPlanStepFailure, + executeMaestroReplayPlanStep, + unwrapMaestroReplayPlanStepFailure, + type MaestroReplayPlanExecutionState, +} from './replay-plan-step-execution.ts'; + +export async function executeMaestroReplayPlan( + plan: MaestroReplayPlan, + port: MaestroRuntimePort, + options: MaestroEngineOptions = {}, +): Promise { + checkpointMaestroCancellation(options.signal); + const startIndex = options.startIndex ?? 0; + if (!Number.isInteger(startIndex) || startIndex < 0 || startIndex > plan.total) { + throw new AppError( + 'INVALID_ARGS', + `Maestro replay startIndex ${startIndex} is out of range for a ${plan.total}-step plan.`, + ); + } + const state: MaestroReplayPlanExecutionState = { + plan, + port, + options, + context: createMaestroExecutionContext( + { ...(options.defaults ?? {}), ...(options.builtins ?? {}) }, + options.env ? { ...options.env } : {}, + ), + timing: resolveMaestroTimingPolicy(options.timing), + artifacts: new Set(), + executed: plan.compatibility.staticallyExecutedControls, + skipped: plan.compatibility.staticallySkippedControls, + }; + for (let index = startIndex; index < plan.steps.length; index += 1) { + checkpointMaestroCancellation(options.signal); + try { + await executeObservedStep(plan.steps[index]!, index, state); + } catch (error) { + throw unwrapMaestroReplayPlanStepFailure(error); + } + } + return { + executed: state.executed, + skipped: state.skipped, + generation: state.context.generation, + artifactPaths: [...state.artifacts], + }; +} + +async function executeObservedStep( + step: MaestroReplayPlanExecutionState['plan']['steps'][number], + index: number, + state: MaestroReplayPlanExecutionState, +): Promise { + checkpointMaestroCancellation(state.options.signal); + const now = state.options.now ?? Date.now; + const startedAt = now(); + const generation = state.context.generation; + const event = { + command: step.command, + source: step.source, + generation, + stepIndex: index + 1, + stepTotal: state.plan.total, + }; + state.options.observer?.commandStarted?.(event); + try { + await executeMaestroReplayPlanStep(step, state); + checkpointMaestroCancellation(state.options.signal); + state.options.observer?.commandCompleted?.({ + ...event, + durationMs: now() - startedAt, + }); + } catch (error) { + const failure = asMaestroReplayPlanStepFailure(error, step); + state.options.observer?.commandFailed?.({ + ...event, + ...(failure.command ? { command: failure.command } : {}), + source: failure.source, + durationMs: now() - startedAt, + error: failure.error, + artifactPaths: [...state.artifacts], + expandedVariables: state.context.expandedVariables, + }); + throw failure; + } +} diff --git a/src/compat/maestro/replay-plan-resume.ts b/src/compat/maestro/replay-plan-resume.ts new file mode 100644 index 000000000..c89aa73f6 --- /dev/null +++ b/src/compat/maestro/replay-plan-resume.ts @@ -0,0 +1,75 @@ +import { AppError } from '../../kernel/errors.ts'; +import type { + MaestroReplayPlan, + MaestroReplayResumePreflight, + MaestroReplayResumeRequest, +} from './replay-plan-types.ts'; + +export function evaluateMaestroReplayResume( + plan: MaestroReplayPlan, + request: MaestroReplayResumeRequest = {}, +): MaestroReplayResumePreflight { + const { from, planDigest } = request; + if (from === undefined && planDigest === undefined) return { allowed: true, startIndex: 0 }; + if (from === undefined || planDigest === undefined) { + return { + allowed: false, + reason: 'replay --from requires --plan-digest (and --plan-digest requires --from).', + }; + } + if (!Number.isInteger(from) || from < 1 || from > plan.total) { + return { + allowed: false, + reason: `replay --from ${from} is out of range for a ${plan.total}-step plan.`, + }; + } + if (planDigest !== plan.digest) { + return { + allowed: false, + reason: 'replay --plan-digest does not match the current plan digest.', + }; + } + if (from === 1) return { allowed: true, startIndex: 0 }; + for (let index = 0; index < from - 1; index += 1) { + const step = plan.steps[index]!; + if (step.kind === 'opaque') { + return { + allowed: false, + reason: `step ${index + 1} is opaque runtime control flow (${step.command.kind}) and cannot be skipped safely.`, + }; + } + if (step.command.kind === 'runScript') { + return { + allowed: false, + reason: `step ${index + 1} (runScript) can produce outputEnv values and cannot be skipped safely.`, + }; + } + } + const target = plan.steps[from - 1]!; + if (target.kind === 'opaque') { + return { + allowed: false, + reason: `step ${from} is opaque runtime control flow (${target.command.kind}) and cannot be resumed into.`, + }; + } + return { allowed: true, startIndex: from - 1 }; +} + +export function resolveMaestroReplayStartIndex( + plan: MaestroReplayPlan, + request: MaestroReplayResumeRequest = {}, +): number { + const result = evaluateMaestroReplayResume(plan, request); + if (!result.allowed) throw new AppError('INVALID_ARGS', result.reason); + return result.startIndex; +} + +export function assertMaestroReplayStartIndex(plan: MaestroReplayPlan, startIndex: number): number { + if (!Number.isInteger(startIndex) || startIndex < 0 || startIndex > plan.total) { + throw new AppError( + 'INVALID_ARGS', + `Maestro replay startIndex ${startIndex} is out of range for a ${plan.total}-step plan.`, + ); + } + return startIndex; +} diff --git a/src/compat/maestro/replay-plan-step-execution.ts b/src/compat/maestro/replay-plan-step-execution.ts new file mode 100644 index 000000000..d0bcc93c3 --- /dev/null +++ b/src/compat/maestro/replay-plan-step-execution.ts @@ -0,0 +1,297 @@ +import { AppError } from '../../kernel/errors.ts'; +import type { MaestroExecutionContext } from './engine-context.ts'; +import { + assertMaestroRepeatExpansionLimit, + checkpointMaestroCancellation, + observationConditions, + readIterationCount, + resolveCommand, + resolveMaestroTimingPolicy, + staticConditionMatches, +} from './engine-flow.ts'; +import type { MaestroCommand, MaestroRunFlowCondition } from './program-ir.ts'; +import type { + MaestroEngineOptions, + MaestroObservation, + MaestroObservationCondition, + MaestroRuntimeCommand, + MaestroRuntimePort, +} from './engine-types.ts'; +import type { + MaestroReplayPlan, + MaestroReplayPlanOpaqueStep, + MaestroReplayPlanStep, +} from './replay-plan-types.ts'; + +export type MaestroReplayPlanExecutionState = { + readonly plan: MaestroReplayPlan; + readonly port: MaestroRuntimePort; + readonly options: MaestroEngineOptions; + readonly context: MaestroExecutionContext; + readonly timing: ReturnType; + readonly artifacts: Set; + executed: number; + skipped: number; +}; + +type PlanStepFailure = { + readonly kind: 'maestroPlanStepFailure'; + readonly error: unknown; + readonly source: MaestroReplayPlanStep['source']; + readonly command?: MaestroCommand; +}; + +export async function executeMaestroReplayPlanStep( + step: MaestroReplayPlanStep, + state: MaestroReplayPlanExecutionState, +): Promise { + checkpointMaestroCancellation(state.options.signal); + try { + await withStepScopes(step, state.context, async () => { + await executeStep(step, state); + }); + } catch (error) { + throw asMaestroReplayPlanStepFailure(error, step); + } +} + +async function executeStep(step: MaestroReplayPlanStep, state: MaestroReplayPlanExecutionState) { + checkpointMaestroCancellation(state.options.signal); + if (step.kind === 'command') { + await executeCommand(step.command, step.appId, state); + return; + } + await executeOpaqueStep(step, state); +} + +async function executeCommand( + rawCommand: MaestroRuntimeCommand, + appId: string | undefined, + state: MaestroReplayPlanExecutionState, +): Promise { + const command = resolveCommand(rawCommand, state.context); + switch (command.kind) { + case 'assertVisible': + await requireObservation( + { kind: 'visible', selector: command.target }, + state.timing.assertVisibleTimeoutMs, + state, + ); + state.executed += 1; + return; + case 'assertNotVisible': + await requireObservation( + { kind: 'notVisible', selector: command.target }, + state.timing.assertNotVisibleTimeoutMs, + state, + ); + state.executed += 1; + return; + case 'extendedWaitUntil': + await requireObservation( + readExtendedWaitCondition(command), + command.timeout ?? state.timing.extendedWaitUntilTimeoutMs, + state, + ); + state.executed += 1; + return; + default: + await executeRuntimeCommand(command, appId, state); + } +} + +async function executeRuntimeCommand( + command: MaestroRuntimeCommand, + appId: string | undefined, + state: MaestroReplayPlanExecutionState, +): Promise { + const request = { + command, + env: state.context.values, + ...(appId === undefined ? {} : { appId: state.context.resolve(appId) }), + generation: state.context.generation, + ...(state.context.observation ? { cachedObservation: state.context.observation } : {}), + ...(state.options.signal ? { signal: state.options.signal } : {}), + }; + const mayMutate = runtimeCommandMayMutate(command); + let result; + try { + result = await state.port.execute(request); + checkpointMaestroCancellation(state.options.signal); + } catch (error) { + if (mayMutate) state.context.recordMutation(); + throw error; + } + if (result.observation) state.context.recordObservation(result.observation); + if (result.outputEnv) state.context.merge(result.outputEnv); + result.artifactPaths?.forEach((entry) => state.artifacts.add(entry)); + if (result.mutated) state.context.recordMutation(); + state.executed += 1; +} + +function runtimeCommandMayMutate(command: MaestroRuntimeCommand): boolean { + return command.kind !== 'takeScreenshot'; +} + +async function executeOpaqueStep( + step: MaestroReplayPlanOpaqueStep, + state: MaestroReplayPlanExecutionState, +): Promise { + const command = resolveCommand(step.command, state.context); + switch (command.kind) { + case 'runFlow': + if (command.when && !(await flowConditionMatches(command.when, state))) { + state.skipped += 1; + return; + } + state.executed += 1; + await executeNestedSteps(step.body, state); + return; + case 'repeat': { + const times = readIterationCount(command.times, 0, state.context, 'repeat.times'); + assertMaestroRepeatExpansionLimit(times); + state.executed += 1; + for (let iteration = 0; iteration < times; iteration += 1) { + checkpointMaestroCancellation(state.options.signal); + await executeNestedSteps(step.body, state); + } + return; + } + case 'retry': { + const retries = readIterationCount(command.maxRetries, 1, state.context, 'retry.maxRetries'); + state.executed += 1; + await executeRetry(step.body, retries, state); + return; + } + } +} + +async function executeNestedSteps( + steps: readonly MaestroReplayPlanStep[], + state: MaestroReplayPlanExecutionState, +): Promise { + for (const step of steps) { + checkpointMaestroCancellation(state.options.signal); + await executeMaestroReplayPlanStep(step, state); + } +} + +async function executeRetry( + steps: readonly MaestroReplayPlanStep[], + maxRetries: number, + state: MaestroReplayPlanExecutionState, +): Promise { + let failure: PlanStepFailure | undefined; + for (let attempt = 0; attempt <= maxRetries; attempt += 1) { + checkpointMaestroCancellation(state.options.signal); + try { + await executeNestedSteps(steps, state); + return; + } catch (error) { + checkpointMaestroCancellation(state.options.signal); + failure = asMaestroReplayPlanStepFailure(error, steps[0]); + } + } + if (failure) throw failure; + throw new AppError('COMMAND_FAILED', 'Maestro retry commands failed.'); +} + +async function flowConditionMatches( + condition: MaestroRunFlowCondition, + state: MaestroReplayPlanExecutionState, +): Promise { + if (!staticConditionMatches(condition, state.context, state.options)) return false; + for (const observation of observationConditions(condition)) { + checkpointMaestroCancellation(state.options.signal); + if (!(await observe(observation, state.timing.runFlowConditionTimeoutMs, state)).matched) { + return false; + } + } + return true; +} + +async function requireObservation( + condition: MaestroObservationCondition, + timeoutMs: number, + state: MaestroReplayPlanExecutionState, +): Promise { + if (!(await observe(condition, timeoutMs, state)).matched) { + throw new AppError('COMMAND_FAILED', `Maestro ${condition.kind} condition did not match.`); + } +} + +async function observe( + condition: MaestroObservationCondition, + timeoutMs: number, + state: MaestroReplayPlanExecutionState, +): Promise { + const request = { + condition, + timeoutMs, + generation: state.context.generation, + env: state.context.values, + ...(state.context.observation ? { cachedObservation: state.context.observation } : {}), + ...(state.options.signal ? { signal: state.options.signal } : {}), + }; + const observation = await state.port.observe(request); + checkpointMaestroCancellation(state.options.signal); + state.context.recordObservation(observation); + return observation; +} + +function readExtendedWaitCondition( + command: Extract, +): MaestroObservationCondition { + if (command.visible) return { kind: 'visible', selector: command.visible }; + if (command.notVisible) return { kind: 'notVisible', selector: command.notVisible }; + throw new AppError('INVALID_ARGS', 'Maestro extendedWaitUntil requires one condition.'); +} + +async function withStepScopes( + step: MaestroReplayPlanStep, + context: MaestroExecutionContext, + callback: () => Promise, +): Promise { + const leaves = step.scopes.map((scope) => context.enter({ ...scope })); + try { + return await callback(); + } finally { + for (let index = leaves.length - 1; index >= 0; index -= 1) leaves[index]!(); + } +} + +export function asMaestroReplayPlanStepFailure( + error: unknown, + step: MaestroReplayPlanStep | undefined, +): PlanStepFailure { + if (isPlanStepFailure(error)) return error; + const source = step?.source ?? { line: 1 }; + return { + kind: 'maestroPlanStepFailure', + error: withSource(error, step?.command), + source, + ...(step ? { command: step.command } : {}), + }; +} + +function isPlanStepFailure(value: unknown): value is PlanStepFailure { + return Boolean( + value && + typeof value === 'object' && + (value as { kind?: unknown }).kind === 'maestroPlanStepFailure', + ); +} + +export function unwrapMaestroReplayPlanStepFailure(error: unknown): unknown { + return isPlanStepFailure(error) ? error.error : error; +} + +function withSource(error: unknown, command: MaestroCommand | undefined): unknown { + if (!(error instanceof AppError) || !command || /\bline \d+\b/.test(error.message)) return error; + const path = command.source.path ? `${command.source.path}:` : ''; + return new AppError( + error.code, + `${error.message} (${path}line ${command.source.line})`, + error.details, + ); +} diff --git a/src/compat/maestro/replay-plan-steps.ts b/src/compat/maestro/replay-plan-steps.ts new file mode 100644 index 000000000..f18e06241 --- /dev/null +++ b/src/compat/maestro/replay-plan-steps.ts @@ -0,0 +1,306 @@ +import { AppError } from '../../kernel/errors.ts'; +import { createMaestroExecutionContext, type MaestroExecutionContext } from './engine-context.ts'; +import { + assertIncludePathAvailable, + assertMaestroRepeatExpansionLimit, + checkpointMaestroCancellation, + readIncludedProgram, + readIterationCount, + registerIncludedProgramPaths, + resolveCommand, + sourcePathKey, +} from './engine-flow.ts'; +import { evaluateMaestroBooleanExpression } from './engine-expression.ts'; +import type { MaestroCommand, MaestroProgram, MaestroRunFlowCommand } from './program-ir.ts'; +import type { + MaestroReplayPlanCommandStep, + MaestroReplayPlanOpaqueStep, + MaestroReplayPlanOptions, + MaestroReplayPlanScope, + MaestroReplayPlanStep, +} from './replay-plan-types.ts'; + +type BuildState = { + readonly options: MaestroReplayPlanOptions; + readonly context: MaestroExecutionContext; + readonly activeIncludePaths: Set; + staticallyExecutedControls: number; + staticallySkippedControls: number; +}; + +type StaticRunFlowDecision = 'omit' | 'flatten' | 'opaque'; + +export type MaestroReplayPlanCompilationResult = { + readonly steps: readonly MaestroReplayPlanStep[]; + readonly staticallyExecutedControls: number; + readonly staticallySkippedControls: number; +}; + +export async function compileMaestroReplayPlanSteps( + program: MaestroProgram, + options: MaestroReplayPlanOptions, +): Promise { + checkpointMaestroCancellation(options.signal); + const rootPath = sourcePathKey(program.source.path); + const state: BuildState = { + options, + context: createMaestroExecutionContext( + { ...(options.defaults ?? {}), ...(options.builtins ?? {}) }, + options.env, + ), + activeIncludePaths: new Set(rootPath === undefined ? [] : [rootPath]), + staticallyExecutedControls: 0, + staticallySkippedControls: 0, + }; + const steps = await compileProgram(program, [], undefined, program.source.path, state); + return { + steps, + staticallyExecutedControls: state.staticallyExecutedControls, + staticallySkippedControls: state.staticallySkippedControls, + }; +} + +async function compileProgram( + program: MaestroProgram, + inheritedScopes: readonly MaestroReplayPlanScope[], + inheritedAppId: string | undefined, + fallbackSourcePath: string | undefined, + state: BuildState, +): Promise { + checkpointMaestroCancellation(state.options.signal); + const scopes = appendScope(inheritedScopes, program.config.env); + const appId = program.config.appId ?? inheritedAppId; + const leave = state.context.enter(program.config.env); + try { + const commands = [ + ...(program.config.onFlowStart ?? []), + ...program.commands, + ...(program.config.onFlowComplete ?? []), + ]; + const steps: MaestroReplayPlanStep[] = []; + for (const command of commands) { + steps.push( + ...(await compileCommand( + command, + scopes, + appId, + program.source.path ?? fallbackSourcePath, + state, + )), + ); + } + return steps; + } finally { + leave(); + } +} + +async function compileCommands( + commands: readonly MaestroCommand[], + scopes: readonly MaestroReplayPlanScope[], + appId: string | undefined, + fallbackSourcePath: string | undefined, + state: BuildState, +): Promise { + const steps: MaestroReplayPlanStep[] = []; + for (const command of commands) { + steps.push(...(await compileCommand(command, scopes, appId, fallbackSourcePath, state))); + } + return steps; +} + +async function compileCommand( + command: MaestroCommand, + scopes: readonly MaestroReplayPlanScope[], + appId: string | undefined, + fallbackSourcePath: string | undefined, + state: BuildState, +): Promise { + checkpointMaestroCancellation(state.options.signal); + const plannedCommand = withFallbackSource(command, fallbackSourcePath); + switch (plannedCommand.kind) { + case 'runFlow': { + const decision = staticRunFlowDecision(plannedCommand, state); + if (decision === 'omit') { + state.staticallySkippedControls += 1; + return []; + } + const body = await compileRunFlowBody( + plannedCommand, + scopes, + appId, + fallbackSourcePath, + state, + ); + if (decision === 'flatten') { + state.staticallyExecutedControls += 1; + return body; + } + return [opaqueStep(plannedCommand, scopes, appId, body)]; + } + case 'repeat': { + const times = staticRepeatCount(plannedCommand.times, state.context); + if (times !== undefined) { + assertMaestroRepeatExpansionLimit(times); + state.staticallyExecutedControls += 1; + const steps: MaestroReplayPlanStep[] = []; + for (let iteration = 0; iteration < times; iteration += 1) { + steps.push( + ...(await compileCommands( + plannedCommand.commands, + scopes, + appId, + fallbackSourcePath, + state, + )), + ); + } + return steps; + } + const body = await compileCommands( + plannedCommand.commands, + scopes, + appId, + fallbackSourcePath, + state, + ); + return [opaqueStep(plannedCommand, scopes, appId, body)]; + } + case 'retry': { + const body = await compileCommands( + plannedCommand.commands, + scopes, + appId, + fallbackSourcePath, + state, + ); + return [opaqueStep(plannedCommand, scopes, appId, body)]; + } + default: + return [commandStep(plannedCommand, scopes, appId)]; + } +} + +async function compileRunFlowBody( + command: MaestroRunFlowCommand, + scopes: readonly MaestroReplayPlanScope[], + appId: string | undefined, + fallbackSourcePath: string | undefined, + state: BuildState, +): Promise { + const resolvedCommand = resolveCommand(command, state.context); + const requestedPath = assertIncludePathAvailable(resolvedCommand, state.activeIncludePaths); + const program = await readIncludedProgram(resolvedCommand, state.options); + checkpointMaestroCancellation(state.options.signal); + const includedPaths = registerIncludedProgramPaths( + resolvedCommand, + program, + requestedPath, + state.activeIncludePaths, + ); + const nextScopes = appendScope(scopes, command.env); + const leave = state.context.enter(command.env); + try { + return await compileProgram( + program, + nextScopes, + appId, + program.source.path ?? command.source.path ?? fallbackSourcePath, + state, + ); + } finally { + leave(); + includedPaths.forEach((value) => state.activeIncludePaths.delete(value)); + } +} + +function staticRunFlowDecision( + command: MaestroRunFlowCommand, + state: BuildState, +): StaticRunFlowDecision { + const condition = command.when; + if (!condition) return 'flatten'; + if (condition.platform !== undefined && condition.platform !== state.options.platform) { + return 'omit'; + } + const booleanDecision = staticBooleanConditionDecision(condition.true, state); + if (booleanDecision) return booleanDecision; + if (condition.visible || condition.notVisible) return 'opaque'; + return 'flatten'; +} + +function staticBooleanConditionDecision( + condition: boolean | string | undefined, + state: BuildState, +): Extract | undefined { + if (condition === false) return 'omit'; + if (typeof condition !== 'string') return undefined; + const resolved = state.context.resolve(condition); + if (hasUnresolvedVariable(resolved)) return 'opaque'; + return evaluateMaestroBooleanExpression(condition, state.context, state.options.platform) + ? undefined + : 'omit'; +} + +function staticRepeatCount( + value: number | string | undefined, + context: MaestroExecutionContext, +): number | undefined { + if (typeof value === 'number') return readIterationCount(value, 0, context, 'repeat.times'); + if (value === undefined) return 0; + const resolved = context.resolve(value); + if (hasUnresolvedVariable(resolved)) return undefined; + return readIterationCount(value, 0, context, 'repeat.times'); +} + +function hasUnresolvedVariable(value: string): boolean { + return /\$\{[A-Za-z_][A-Za-z0-9_.]*\}/.test(value); +} + +function commandStep( + command: Extract, + scopes: readonly MaestroReplayPlanScope[], + appId: string | undefined, +): MaestroReplayPlanCommandStep { + if (command.kind === 'runFlow' || command.kind === 'repeat' || command.kind === 'retry') { + throw new AppError('COMMAND_FAILED', `Unexpected opaque Maestro command ${command.kind}.`); + } + return { + kind: 'command', + command, + source: command.source, + scopes, + ...(appId === undefined ? {} : { appId }), + }; +} + +function opaqueStep( + command: Extract, + scopes: readonly MaestroReplayPlanScope[], + appId: string | undefined, + body: readonly MaestroReplayPlanStep[], +): MaestroReplayPlanOpaqueStep { + return { + kind: 'opaque', + command, + source: command.source, + scopes, + body, + ...(appId === undefined ? {} : { appId }), + }; +} + +function appendScope( + scopes: readonly MaestroReplayPlanScope[], + scope: Record | undefined, +): readonly MaestroReplayPlanScope[] { + return scope === undefined ? scopes : [...scopes, scope]; +} + +function withFallbackSource( + command: T, + fallbackSourcePath: string | undefined, +): T { + if (command.source.path || !fallbackSourcePath) return command; + return { ...command, source: { ...command.source, path: fallbackSourcePath } } as T; +} diff --git a/src/compat/maestro/replay-plan-types.ts b/src/compat/maestro/replay-plan-types.ts new file mode 100644 index 000000000..80fa6dc84 --- /dev/null +++ b/src/compat/maestro/replay-plan-types.ts @@ -0,0 +1,70 @@ +import type { MaestroProgramLoader } from './program-loader.ts'; +import type { MaestroPlatform, MaestroSourceLocation } from './program-ir.ts'; +import type { MaestroControlCommand, MaestroRuntimeCommand } from './engine-types.ts'; +import type { SessionRuntimeHints } from '../../kernel/contracts.ts'; + +export type MaestroReplayPlanScope = Readonly>; + +type MaestroReplayPlanStepBase = { + readonly source: MaestroSourceLocation; + readonly scopes: readonly MaestroReplayPlanScope[]; + readonly appId?: string; +}; + +export type MaestroReplayPlanCommandStep = MaestroReplayPlanStepBase & { + readonly kind: 'command'; + readonly command: MaestroRuntimeCommand; +}; + +export type MaestroReplayPlanOpaqueStep = MaestroReplayPlanStepBase & { + readonly kind: 'opaque'; + readonly command: MaestroControlCommand; + readonly body: readonly MaestroReplayPlanStep[]; +}; + +export type MaestroReplayPlanStep = MaestroReplayPlanCommandStep | MaestroReplayPlanOpaqueStep; + +export type MaestroReplayPlan = { + readonly kind: 'maestroReplayPlan'; + readonly platform?: MaestroPlatform; + readonly target?: string; + readonly runtimeHints?: Readonly; + /** Effective static values used to resolve the plan; runtime output values are excluded. */ + readonly initialStaticEnv: Readonly>; + readonly steps: readonly MaestroReplayPlanStep[]; + readonly total: number; + readonly digest: string; + /** Counts retained for the existing executeMaestroProgram result shape. */ + readonly compatibility: { + readonly staticallyExecutedControls: number; + readonly staticallySkippedControls: number; + }; +}; + +export type MaestroReplayPlanOptions = { + /** Highest-precedence invocation values, normally CLI over shell. */ + readonly env?: Readonly>; + /** Lowest-precedence defaults, normally replay built-ins. */ + readonly defaults?: Readonly>; + /** Alias for defaults for callers that name this layer explicitly. */ + readonly builtins?: Readonly>; + readonly platform?: MaestroPlatform; + readonly target?: string; + readonly runtimeHints?: Readonly; + readonly loadProgram?: MaestroProgramLoader; + readonly signal?: AbortSignal; +}; + +export type MaestroReplayResumeRequest = { + readonly from?: number; + readonly planDigest?: string; +}; + +export type MaestroReplayResumePreflight = + | { readonly allowed: true; readonly startIndex: number } + | { readonly allowed: false; readonly reason: string }; + +export type MaestroReplayStartIndex = { + readonly startIndex: number; + readonly plan: MaestroReplayPlan; +}; diff --git a/src/compat/maestro/replay-plan.ts b/src/compat/maestro/replay-plan.ts new file mode 100644 index 000000000..7de5cd374 --- /dev/null +++ b/src/compat/maestro/replay-plan.ts @@ -0,0 +1,6 @@ +export { compileMaestroReplayPlan } from './replay-plan-compilation.ts'; +export { + assertMaestroReplayStartIndex, + evaluateMaestroReplayResume, + resolveMaestroReplayStartIndex, +} from './replay-plan-resume.ts'; diff --git a/src/compat/maestro/run-script.ts b/src/compat/maestro/run-script-execution.ts similarity index 65% rename from src/compat/maestro/run-script.ts rename to src/compat/maestro/run-script-execution.ts index fac663d35..11df3324e 100644 --- a/src/compat/maestro/run-script.ts +++ b/src/compat/maestro/run-script-execution.ts @@ -1,19 +1,7 @@ import fs from 'node:fs'; -import path from 'node:path'; import vm from 'node:vm'; -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; +import { AppError, normalizeError } from '../../kernel/errors.ts'; import { runCmdSync } from '../../utils/exec.ts'; -import { MAESTRO_RUNTIME_COMMAND } from './runtime-commands.ts'; -import { - action, - assertOnlyKeys, - isPlainRecord, - readEnvMap, - requireStringValue, - resolveMaestroString, -} from './support.ts'; -import type { MaestroParseContext } from './types.ts'; const RUN_SCRIPT_TIMEOUT_MS = 30_000; @@ -46,16 +34,11 @@ fetch(input.url, { }); `; -export function convertRunScript(value: unknown, context: MaestroParseContext): SessionAction { - const scriptConfig = readRunScriptConfig(value, context); - const scriptPath = resolveRunScriptPath(scriptConfig.file, context); - return action(MAESTRO_RUNTIME_COMMAND.runScript, [scriptPath], { - ...(Object.keys(scriptConfig.env).length > 0 - ? { maestro: { runScriptEnv: scriptConfig.env } } - : {}), - }); -} - +/** + * Executes a trusted flow-local script with the compatibility helpers Maestro + * exposes. `node:vm` isolates globals for the run but is not a security + * sandbox; the caller must establish the trust boundary before invoking this. + */ export function executeRunScriptFile(params: { scriptPath: string; env: Record; @@ -65,19 +48,26 @@ export function executeRunScriptFile(params: { const output: Record = Object.create(null) as Record; try { - // Compatibility note: node:vm is not a security sandbox. Maestro runScript - // files are trusted flow-local setup code; the timeout only bounds - // synchronous script execution. Async http.post work is bounded separately - // by the child process timeout in runHttpRequestSync. + // The synchronous script budget is independent from the child-process + // budget used by http.post below. vm.runInNewContext(script, buildScriptGlobals(env, output), { filename: scriptPath, timeout: RUN_SCRIPT_TIMEOUT_MS, }); } catch (error) { + const normalized = normalizeError(error); throw new AppError( - 'COMMAND_FAILED', - `Maestro runScript failed for ${scriptPath}: ${error instanceof Error ? error.message : String(error)}`, - { scriptPath }, + normalized.code, + `Maestro runScript failed: ${normalized.message}`, + { + ...(normalized.details ?? {}), + ...(normalized.hint ? { hint: normalized.hint } : {}), + ...(normalized.diagnosticId ? { diagnosticId: normalized.diagnosticId } : {}), + ...(normalized.logPath ? { logPath: normalized.logPath } : {}), + ...(normalized.retriable === undefined ? {} : { retriable: normalized.retriable }), + ...(normalized.supportedOn ? { supportedOn: normalized.supportedOn } : {}), + scriptPath, + }, error instanceof Error ? error : undefined, ); } @@ -91,36 +81,6 @@ export function executeRunScriptFile(params: { ); } -function readRunScriptConfig( - value: unknown, - context: MaestroParseContext, -): { file: string; env: Record } { - if (typeof value === 'string') { - return { file: resolveMaestroString(value, context), env: {} }; - } - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', 'runScript expects a file path string or map.'); - } - assertOnlyKeys(value, 'runScript', ['file', 'env']); - const file = resolveMaestroString(requireStringValue('runScript.file', value.file), context); - const rawEnv = readEnvMap(value.env, 'runScript.env'); - const env = Object.fromEntries( - Object.entries(rawEnv).map(([key, envValue]) => [key, resolveMaestroString(envValue, context)]), - ); - return { file, env }; -} - -function resolveRunScriptPath(filePath: string, context: MaestroParseContext): string { - if (path.isAbsolute(filePath)) return filePath; - if (!context.baseDir) { - throw new AppError( - 'INVALID_ARGS', - 'runScript file paths require replay input to have a source path.', - ); - } - return path.resolve(context.baseDir, filePath); -} - function buildScriptGlobals( env: Record, output: Record, @@ -144,10 +104,9 @@ function parseRunScriptJson(value: unknown): unknown { ); } if (value.trim().length === 0) { - throw new AppError( - 'COMMAND_FAILED', - 'Maestro runScript json() received an empty body. Check the preceding http response status and setup server output.', - ); + throw new AppError('COMMAND_FAILED', 'Maestro runScript json() received an empty body.', { + hint: 'Check the preceding HTTP response status and setup server output.', + }); } try { return JSON.parse(value, safeRunScriptJsonReviver) as unknown; diff --git a/src/compat/maestro/runtime-assertions.ts b/src/compat/maestro/runtime-assertions.ts deleted file mode 100644 index c6614509f..000000000 --- a/src/compat/maestro/runtime-assertions.ts +++ /dev/null @@ -1,837 +0,0 @@ -import fs from 'node:fs'; -import path from 'node:path'; -import { getSnapshotReferenceFrame } from '../../daemon/touch-reference-frame.ts'; -import { tryParseSelectorChain } from '../../selectors/index.ts'; -import type { DaemonResponse } from '../../daemon/types.ts'; -import type { DaemonFailureResponse } from '../../daemon/handlers/response.ts'; -import type { ReplayVarScope } from '../../replay/vars.ts'; -import { emitDiagnostic } from '../../utils/diagnostics.ts'; -import type { Point, SnapshotState } from '../../kernel/snapshot.ts'; -import { buildSnapshotDisplayLines } from '../../snapshot/snapshot-lines.ts'; -import { sleep } from '../../utils/timeouts.ts'; -import { pointForMaestroTapOnTarget } from './runtime-geometry.ts'; -import { - captureMaestroSnapshot, - consumeMaestroRecoverableInteraction, - errorResponse, - rememberMaestroVisibleContext, - readSnapshotState, - type MaestroRecoverableSwipe, - type MaestroRecoverableTap, - type MaestroRuntimeInvoke, - type ReplayBaseRequest, -} from './runtime-support.ts'; -import { - extractMaestroVisibleTextQuery, - hasMaestroSelectorMatchInSnapshot, - readMaestroSelectorPlatform, - resolveMaestroNodeFromSnapshot, - resolveVisibleMaestroNodeFromSnapshot, -} from './runtime-targets.ts'; - -const MAESTRO_ASSERTION_POLICY = { - animationPollMs: 250, - assertVisibleGraceMs: 1000, - assertVisiblePollMs: 250, - assertVisibleRetryTimeoutMs: 5000, - assertNotVisiblePollMs: 250, - defaultAssertNotVisibleTimeoutMs: 3000, -} as const; - -type MaestroVisibilitySample = - | { visible: true; response: DaemonResponse } - | { - visible: false; - response: DaemonResponse; - infrastructureFailure: boolean; - missKind?: 'missing' | 'notVisible'; - snapshot?: SnapshotState; - }; - -type MaestroVisibilityAssertionArgs = { - selector: string; - timeoutMs: number; -}; - -type MaestroRetryTapTarget = { kind: 'point'; point: Point } | { kind: 'text'; query: string }; - -type MaestroVisibleRecoveryFlag = 'retryTap' | 'retrySwipe'; - -type MaestroAssertionRuntimeParams = { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}; - -type MaestroAssertionRequestParams = MaestroAssertionRuntimeParams & { - positionals: string[]; -}; - -export async function invokeMaestroAssertVisible( - params: MaestroAssertionRequestParams, -): Promise { - const args = readVisibilityAssertionArgs(params.positionals, { - command: 'assertVisible', - defaultTimeoutMs: 17000, - }); - if (!args.ok) return args.response; - - const nativeWaitQuery = readNativeVisibleWaitQuery(params.baseReq, args.selector); - if (nativeWaitQuery) { - return await invokeNativeMaestroVisibleWaitWithSnapshotFallback(params, args, nativeWaitQuery); - } - - return await invokeSnapshotMaestroAssertVisible(params, args); -} - -async function invokeNativeMaestroVisibleWaitWithSnapshotFallback( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - nativeWaitQuery: string, -): Promise { - const nativeStartedAt = Date.now(); - const nativeResponse = await runNativeVisibleWait(params, args, nativeWaitQuery); - if (nativeResponse.ok) { - if (shouldVerifyNativeVisibleWait(params.baseReq)) { - const sample = await readMaestroVisibilitySample(params, args.selector, 'assertVisible'); - if (!sample.visible) { - const failedSample = handleFailedVisibleSample( - params.baseReq, - args, - sample, - nativeStartedAt, - ); - if (failedSample.kind === 'return') return failedSample.response; - return await invokeSnapshotMaestroAssertVisible(params, visibleAssertionRetryArgs(args)); - } - } - rememberMaestroVisibleContext(params.scope, args.selector); - return visibleAssertionResponse( - { - ok: true, - data: { - selector: args.selector, - nativeWait: true, - query: nativeWaitQuery, - response: nativeResponse.data, - }, - }, - args.selector, - nativeStartedAt, - ); - } - - return await invokeSingleSnapshotMaestroAssertVisible( - params, - args, - nativeResponse, - nativeStartedAt, - ); -} - -function shouldVerifyNativeVisibleWait(baseReq: ReplayBaseRequest): boolean { - return baseReq.flags?.platform === 'android'; -} - -async function runNativeVisibleWait( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - nativeWaitQuery: string, -): Promise { - return await params.invoke({ - ...params.baseReq, - command: 'wait', - positionals: [nativeWaitQuery, String(args.timeoutMs)], - }); -} - -async function invokeSnapshotMaestroAssertVisible( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, -): Promise { - // Native wait/is cannot replace this loop: wait only proves existence, while - // is requires unique resolution and does not apply Maestro overlay filtering. - const startedAt = Date.now(); - const deadlineMs = args.timeoutMs + MAESTRO_ASSERTION_POLICY.assertVisibleGraceMs; - let lastResponse: DaemonResponse | undefined; - let lastSnapshot: SnapshotState | undefined; - let capturedAfterDeadline = false; - while (true) { - const captureStartedAt = Date.now(); - const sample = await readMaestroVisibilitySample(params, args.selector, 'assertVisible'); - if (sample.visible) return visibleAssertionResponse(sample.response, args.selector, startedAt); - lastResponse = sample.response; - lastSnapshot = sample.snapshot ?? lastSnapshot; - const failedSample = handleFailedVisibleSample(params.baseReq, args, sample, startedAt); - if (failedSample.kind === 'return') return failedSample.response; - - const deadline = readVisibleAssertionDeadlineAction({ - captureStartedAt, - capturedAfterDeadline, - startedAt, - deadlineMs, - }); - if (deadline === 'capture-again') { - capturedAfterDeadline = true; - continue; - } - if (deadline === 'finish') break; - await sleep(MAESTRO_ASSERTION_POLICY.assertVisiblePollMs); - } - - const response = - lastResponse ?? - errorResponse('COMMAND_FAILED', `Expected visible but did not match: ${args.selector}`, { - selector: args.selector, - timeoutMs: args.timeoutMs, - }); - const recoveryResponse = await recoverFromAndroidVisibleMiss(params, args, lastSnapshot); - if (recoveryResponse) return recoveryResponse; - return withMaestroFailureSnapshotArtifacts(response, lastSnapshot, params.baseReq); -} - -async function invokeSingleSnapshotMaestroAssertVisible( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - fallbackResponse: DaemonFailureResponse, - startedAt: number, -): Promise { - const sample = await readMaestroVisibilitySample(params, args.selector, 'assertVisible'); - if (sample.visible) return visibleAssertionResponse(sample.response, args.selector, startedAt); - const failedSample = handleFailedVisibleSample(params.baseReq, args, sample, startedAt); - if (failedSample.kind === 'return') return failedSample.response; - const recoveryResponse = await recoverFromAndroidVisibleMiss(params, args, sample.snapshot); - if (recoveryResponse) return recoveryResponse; - return withMaestroFailureSnapshotArtifacts(fallbackResponse, sample.snapshot, params.baseReq); -} - -async function recoverFromAndroidVisibleMiss( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - snapshot: SnapshotState | undefined, -): Promise { - if (params.baseReq.flags?.platform !== 'android') return null; - - const recoverableInteraction = consumeMaestroRecoverableInteraction(params.scope); - if (!recoverableInteraction) return null; - if (recoverableInteraction.kind === 'tap') { - return await retryRecentTapAfterVisibleMiss(params, args, snapshot, recoverableInteraction); - } - return await retryRecentSwipeAfterVisibleMiss(params, args, recoverableInteraction); -} - -async function retryRecentTapAfterVisibleMiss( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - snapshot: SnapshotState | undefined, - recentTap: MaestroRecoverableTap, -): Promise { - if (!snapshot) return null; - - const retryTarget = resolveRecentTapTarget(params, snapshot, recentTap); - if (!retryTarget.ok) return null; - - emitDiagnostic({ - level: 'info', - phase: 'maestro_assert_visible_retry_tap', - data: { - selector: args.selector, - tapSelector: recentTap.selector, - originalPoint: recentTap.point, - retryTarget: retryTarget.target, - timeoutMs: MAESTRO_ASSERTION_POLICY.assertVisibleRetryTimeoutMs, - }, - }); - - const clickResponse = await invokeRecentTapRetry(params, retryTarget.target); - if (!clickResponse.ok) return null; - return await confirmVisibleAfterRecovery(params, args, 'retryTap'); -} - -async function retryRecentSwipeAfterVisibleMiss( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - recentSwipe: MaestroRecoverableSwipe, -): Promise { - emitDiagnostic({ - level: 'info', - phase: 'maestro_assert_visible_retry_swipe', - data: { - selector: args.selector, - swipeCommand: recentSwipe.command, - swipeInput: recentSwipe.input, - timeoutMs: MAESTRO_ASSERTION_POLICY.assertVisibleRetryTimeoutMs, - }, - }); - - const swipeResponse = await invokeRecentSwipeRetry(params, recentSwipe); - if (!swipeResponse.ok) return null; - return await confirmVisibleAfterRecovery(params, args, 'retrySwipe'); -} - -async function confirmVisibleAfterRecovery( - params: MaestroAssertionRuntimeParams, - args: MaestroVisibilityAssertionArgs, - recoveryFlag: MaestroVisibleRecoveryFlag, -): Promise { - const retryArgs = { - ...args, - timeoutMs: visibleAssertionRetryTimeoutMs(args.timeoutMs), - }; - const nativeWaitQuery = readNativeVisibleWaitQuery(params.baseReq, retryArgs.selector); - if (!nativeWaitQuery) return await invokeSnapshotMaestroAssertVisible(params, retryArgs); - - const retryStartedAt = Date.now(); - const nativeResponse = await runNativeVisibleWait(params, retryArgs, nativeWaitQuery); - if (nativeResponse.ok) { - rememberMaestroVisibleContext(params.scope, retryArgs.selector); - return visibleAssertionResponse( - { - ok: true, - data: { - selector: retryArgs.selector, - nativeWait: true, - [recoveryFlag]: true, - query: nativeWaitQuery, - response: nativeResponse.data, - }, - }, - retryArgs.selector, - retryStartedAt, - ); - } - - return await invokeSingleSnapshotMaestroAssertVisible( - params, - retryArgs, - nativeResponse, - retryStartedAt, - ); -} - -function resolveRecentTapTarget( - params: MaestroAssertionRuntimeParams, - snapshot: SnapshotState, - tap: MaestroRecoverableTap, -): { ok: true; target: MaestroRetryTapTarget } | { ok: false } { - const platform = readMaestroSelectorPlatform(params.baseReq.flags); - const frame = getSnapshotReferenceFrame(snapshot); - const tapTarget = resolveMaestroNodeFromSnapshot( - snapshot, - tap.selector, - tap.options ?? {}, - platform, - frame, - { promoteTapTarget: true }, - ); - if (tapTarget.ok) { - return { ok: true, target: { kind: 'point', point: pointForMaestroTapOnTarget(tapTarget) } }; - } - - const query = extractMaestroVisibleTextQuery(tap.selector); - if (!query || !snapshotContainsTextQuery(snapshot, query)) return { ok: false }; - return { ok: true, target: { kind: 'text', query } }; -} - -async function invokeRecentTapRetry( - params: MaestroAssertionRuntimeParams, - target: MaestroRetryTapTarget, -): Promise { - if (target.kind === 'text') { - return await params.invoke({ - ...params.baseReq, - command: 'find', - positionals: [target.query, 'click'], - flags: { - ...params.baseReq.flags, - findFirst: true, - postGestureStabilization: true, - }, - }); - } - - return await params.invoke({ - ...params.baseReq, - command: 'click', - positionals: [String(target.point.x), String(target.point.y)], - flags: { - ...params.baseReq.flags, - postGestureStabilization: true, - }, - }); -} - -async function invokeRecentSwipeRetry( - params: MaestroAssertionRuntimeParams, - swipe: MaestroRecoverableSwipe, -): Promise { - return await params.invoke({ - ...params.baseReq, - command: swipe.command, - positionals: [], - input: swipe.input, - }); -} - -function snapshotContainsTextQuery(snapshot: SnapshotState, query: string): boolean { - const needle = query.trim().toLowerCase(); - if (!needle) return false; - return snapshot.nodes.some((node) => - [node.label, node.value, node.identifier].some((value) => - value?.trim().toLowerCase().includes(needle), - ), - ); -} - -function handleFailedVisibleSample( - baseReq: ReplayBaseRequest, - args: MaestroVisibilityAssertionArgs, - sample: Exclude, - startedAt: number, -): { kind: 'continue' } | { kind: 'return'; response: DaemonResponse } { - if (isReactNativeOverlayBlockingAssertion(sample.response)) { - return { kind: 'return', response: sample.response }; - } - if (shouldPassAlreadyPastLoading(baseReq, args.selector, sample.snapshot)) { - return { - kind: 'return', - response: alreadyPastLoadingResponse(args.selector, args.timeoutMs, startedAt), - }; - } - return { kind: 'continue' }; -} - -function shouldPassAlreadyPastLoading( - baseReq: ReplayBaseRequest, - selector: string, - snapshot: SnapshotState | undefined, -): boolean { - return ( - baseReq.flags?.maestro?.allowAlreadyPastLoading === true && - snapshot !== undefined && - isAlreadyPastLoadingState(selector, snapshot) - ); -} - -function readVisibleAssertionDeadlineAction(params: { - captureStartedAt: number; - capturedAfterDeadline: boolean; - startedAt: number; - deadlineMs: number; -}): 'wait' | 'capture-again' | 'finish' { - const elapsedMs = Date.now() - params.startedAt; - if (elapsedMs < params.deadlineMs) return 'wait'; - return shouldCaptureOnceAfterDeadline( - params.capturedAfterDeadline, - params.captureStartedAt, - params.startedAt, - params.deadlineMs, - ) - ? 'capture-again' - : 'finish'; -} - -function visibleAssertionRetryArgs( - args: MaestroVisibilityAssertionArgs, -): MaestroVisibilityAssertionArgs { - return { - ...args, - timeoutMs: visibleAssertionRetryTimeoutMs(args.timeoutMs), - }; -} - -function visibleAssertionRetryTimeoutMs(timeoutMs: number): number { - return Math.min(timeoutMs, MAESTRO_ASSERTION_POLICY.assertVisibleRetryTimeoutMs); -} - -function isReactNativeOverlayBlockingAssertion(response: DaemonResponse): boolean { - return ( - !response.ok && - response.error.code === 'COMMAND_FAILED' && - response.error.message.includes('React Native overlay') - ); -} - -function readNativeVisibleWaitQuery(baseReq: ReplayBaseRequest, selector: string): string | null { - if (baseReq.flags?.platform !== 'ios' && baseReq.flags?.platform !== 'android') return null; - return extractMaestroVisibleTextQuery(selector); -} - -function alreadyPastLoadingResponse( - selector: string, - timeoutMs: number, - startedAt: number, -): DaemonResponse { - return { - ok: true, - data: { - selector, - alreadyPastLoading: true, - waitedMs: Date.now() - startedAt, - timeoutMs, - }, - }; -} - -function readVisibilityAssertionArgs( - positionals: string[], - options: { command: string; defaultTimeoutMs: number }, -): { ok: true; selector: string; timeoutMs: number } | { ok: false; response: DaemonResponse } { - const [selector, timeoutValue = String(options.defaultTimeoutMs)] = positionals; - if (!selector) { - return { - ok: false, - response: errorResponse('INVALID_ARGS', `${options.command} requires a selector.`), - }; - } - const timeoutMs = Number(timeoutValue); - if (!Number.isFinite(timeoutMs) || timeoutMs < 0) { - return { - ok: false, - response: errorResponse( - 'INVALID_ARGS', - `${options.command} timeout must be a non-negative number.`, - ), - }; - } - return { ok: true, selector, timeoutMs }; -} - -async function readMaestroVisibilitySample( - params: MaestroAssertionRuntimeParams, - selector: string, - command: string, -): Promise { - const response = await captureMaestroSnapshot(params); - const sample = readMaestroVisibilitySampleFromResponse(params, selector, command, response); - if (!shouldRetryAndroidRawVisibilitySample(params.baseReq, selector, command, sample)) { - return sample; - } - - const rawResponse = await captureMaestroSnapshot({ ...params, raw: true }); - return readMaestroVisibilitySampleFromResponse(params, selector, command, rawResponse); -} - -function readMaestroVisibilitySampleFromResponse( - params: Pick, - selector: string, - command: string, - response: DaemonResponse, -): MaestroVisibilitySample { - if (!response.ok) return { visible: false, response, infrastructureFailure: true }; - const snapshot = readSnapshotState(response.data); - if (!snapshot) { - return { - visible: false, - response: errorResponse('COMMAND_FAILED', `Unable to read snapshot data for ${command}.`), - infrastructureFailure: true, - }; - } - const platform = readMaestroSelectorPlatform(params.baseReq.flags); - const target = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - selector, - platform, - getSnapshotReferenceFrame(snapshot), - ); - if (!target.ok) { - const missKind = readAndroidMaestroMissKind(snapshot, selector, platform); - return { - visible: false, - response: errorResponse('COMMAND_FAILED', target.message, { selector }), - infrastructureFailure: false, - ...(missKind ? { missKind } : {}), - snapshot, - }; - } - rememberMaestroVisibleContext(params.scope, selector); - return { - visible: true, - response: { - ok: true, - data: { - selector, - matches: target.matches, - nodeIndex: target.node.index, - nodeType: target.node.type, - nodeLabel: target.node.label, - nodeIdentifier: target.node.identifier, - rect: target.rect, - }, - }, - }; -} - -function shouldRetryAndroidRawVisibilitySample( - baseReq: ReplayBaseRequest, - selector: string, - command: string, - sample: MaestroVisibilitySample, -): boolean { - return ( - command === 'assertVisible' && - baseReq.flags?.platform === 'android' && - !sample.visible && - !sample.infrastructureFailure && - sample.missKind === 'missing' && - isIdOnlyMaestroSelector(selector) - ); -} - -function isIdOnlyMaestroSelector(selector: string): boolean { - const chain = tryParseSelectorChain(selector); - if (!chain) return false; - return ( - chain.selectors.length > 0 && - chain.selectors.every( - (entry) => - entry.terms.length > 0 && - entry.terms.every((term) => term.key === 'id' && typeof term.value === 'string'), - ) - ); -} - -function readAndroidMaestroMissKind( - snapshot: SnapshotState, - selector: string, - platform: string, -): 'missing' | 'notVisible' | undefined { - if (platform !== 'android') return undefined; - return hasMaestroSelectorMatchInSnapshot(snapshot, selector, platform) ? 'notVisible' : 'missing'; -} - -function isAlreadyPastLoadingState(selector: string, snapshot: SnapshotState): boolean { - const query = normalizeLoadingText(extractMaestroVisibleTextQuery(selector)); - if (!isLoadingText(query)) return false; - - // Maestro extendedWaitUntil is commonly used as a loading gate. If the exact - // loading label is already gone and the current surface has other content, - // treat the flow as past the transient state instead of timing out on stale text. - const currentTexts = snapshot.nodes - .flatMap((node) => [node.label, node.value, node.identifier]) - .filter((value): value is string => Boolean(value?.trim())) - .map((value) => normalizeLoadingText(value)); - - if (currentTexts.some((text) => text.includes('something went wrong'))) return false; - return currentTexts.some((text) => text !== query && !isLoadingText(text)); -} - -function normalizeLoadingText(value: string | null | undefined): string { - return ( - value - ?.trim() - .toLowerCase() - .replace(/\u2026/g, '...') ?? '' - ); -} - -function isLoadingText(value: string): boolean { - return value === 'loading' || value === 'loading...'; -} - -function visibleAssertionResponse( - response: DaemonResponse, - selector: string, - startedAt: number, -): DaemonResponse { - if (!response.ok) return response; - return { - ok: true, - data: { - selector, - ...response.data, - waitedMs: Date.now() - startedAt, - }, - }; -} - -function withMaestroFailureSnapshotArtifacts( - response: DaemonResponse, - snapshot: SnapshotState | undefined, - baseReq: ReplayBaseRequest, -): DaemonResponse { - if (response.ok || !snapshot) return response; - const artifactsDir = - typeof baseReq.flags?.artifactsDir === 'string' ? baseReq.flags.artifactsDir : undefined; - if (!artifactsDir) return response; - - const artifactPaths = writeMaestroFailureSnapshotArtifacts(snapshot, artifactsDir); - if (artifactPaths.length === 0) return response; - return { - ok: false, - error: { - ...response.error, - details: { - ...(response.error.details ?? {}), - artifactPaths: uniqueStrings([ - ...readExistingArtifactPaths(response.error.details?.artifactPaths), - ...artifactPaths, - ]), - }, - }, - }; -} - -function writeMaestroFailureSnapshotArtifacts( - snapshot: SnapshotState, - artifactsDir: string, -): string[] { - try { - fs.mkdirSync(artifactsDir, { recursive: true }); - const jsonPath = path.join(artifactsDir, 'failure-snapshot.json'); - const textPath = path.join(artifactsDir, 'failure-snapshot.txt'); - fs.writeFileSync(jsonPath, `${JSON.stringify(snapshot, null, 2)}\n`); - const lines = buildSnapshotDisplayLines(snapshot.nodes, { - summarizeTextSurfaces: true, - }).map((line) => line.text); - fs.writeFileSync(textPath, `${lines.join('\n')}\n`); - return [jsonPath, textPath]; - } catch { - return []; - } -} - -function readExistingArtifactPaths(value: unknown): string[] { - return Array.isArray(value) - ? value.filter((entry): entry is string => typeof entry === 'string') - : []; -} - -function uniqueStrings(values: string[]): string[] { - return [...new Set(values)]; -} - -function shouldCaptureOnceAfterDeadline( - capturedAfterDeadline: boolean, - captureStartedAt: number, - startedAt: number, - deadlineMs: number, -): boolean { - return !capturedAfterDeadline && captureStartedAt - startedAt < deadlineMs; -} - -export async function invokeMaestroAssertNotVisible( - params: MaestroAssertionRequestParams, -): Promise { - const args = readVisibilityAssertionArgs(params.positionals, { - command: 'assertNotVisible', - defaultTimeoutMs: MAESTRO_ASSERTION_POLICY.defaultAssertNotVisibleTimeoutMs, - }); - if (!args.ok) return args.response; - - // Native is hidden intentionally fails for absent selectors. Maestro - // assertNotVisible treats absent and overlay-blocked targets as passing, so - // this loop shares the visible resolver instead of delegating to native is. - const startedAt = Date.now(); - let hiddenSamples = 0; - let lastVisibleResponse: DaemonResponse | undefined; - while (Date.now() - startedAt <= args.timeoutMs) { - const sample = await readMaestroVisibilitySample(params, args.selector, 'assertNotVisible'); - if (!sample.visible && sample.infrastructureFailure) return sample.response; - if (sample.visible) { - hiddenSamples = 0; - lastVisibleResponse = sample.response; - } else { - hiddenSamples += 1; - const waitedMs = Date.now() - startedAt; - if (hiddenSamples >= 2 || waitedMs >= args.timeoutMs) { - return { - ok: true, - data: { - pass: true, - selector: args.selector, - stableSamples: hiddenSamples, - waitedMs, - timeoutMs: args.timeoutMs, - }, - }; - } - } - await sleep(MAESTRO_ASSERTION_POLICY.assertNotVisiblePollMs); - } - if (hiddenSamples > 0) { - return { - ok: true, - data: { - pass: true, - selector: args.selector, - stableSamples: hiddenSamples, - waitedMs: Date.now() - startedAt, - timeoutMs: args.timeoutMs, - }, - }; - } - return errorResponse('COMMAND_FAILED', `Expected not visible but matched: ${args.selector}`, { - selector: args.selector, - timeoutMs: args.timeoutMs, - lastResponse: lastVisibleResponse, - }); -} - -export async function invokeMaestroWaitForAnimationToEnd( - params: MaestroAssertionRequestParams, -): Promise { - const timeoutMs = Number(params.positionals[0] ?? 15000); - if (!Number.isFinite(timeoutMs) || timeoutMs < 0) { - return errorResponse('INVALID_ARGS', 'waitForAnimationToEnd timeout must be a number.'); - } - // There is no native wait/is equivalent for "animation has ended"; this is - // snapshot stability polling by design. - const startedAt = Date.now(); - let previousSignature: string | undefined; - let lastResponse: DaemonResponse | undefined; - - while (Date.now() - startedAt < timeoutMs) { - const response = await captureMaestroSnapshot(params); - const poll = readAnimationPollResult(response, previousSignature, timeoutMs); - if (poll.done) return poll.response; - previousSignature = poll.signature ?? previousSignature; - lastResponse = response; - await sleep(MAESTRO_ASSERTION_POLICY.animationPollMs); - } - - return lastResponse?.ok === false - ? lastResponse - : { ok: true, data: { stable: false, timeoutMs } }; -} - -function readAnimationPollResult( - response: DaemonResponse, - previousSignature: string | undefined, - timeoutMs: number, -): { done: true; response: DaemonResponse } | { done: false; signature?: string } { - const signature = readSnapshotStabilitySignature(response); - if (!response.ok) return { done: false }; - if (!signature) return { done: true, response }; - if (previousSignature === signature) { - return { done: true, response: { ok: true, data: { stable: true, timeoutMs } } }; - } - return { done: false, signature }; -} - -function readSnapshotStabilitySignature(response: DaemonResponse): string | null { - if (!response.ok) return null; - const snapshot = readSnapshotState(response.data); - return snapshot ? snapshotStabilitySignature(snapshot) : null; -} - -function snapshotStabilitySignature(snapshot: SnapshotState): string { - return JSON.stringify( - snapshot.nodes.map((node) => ({ - index: node.index, - parentIndex: node.parentIndex, - type: node.type, - identifier: node.identifier, - label: node.label, - value: node.value, - rect: node.rect - ? { - x: Math.round(node.rect.x), - y: Math.round(node.rect.y), - width: Math.round(node.rect.width), - height: Math.round(node.rect.height), - } - : undefined, - })), - ); -} diff --git a/src/compat/maestro/runtime-click.ts b/src/compat/maestro/runtime-click.ts deleted file mode 100644 index fced71eb2..000000000 --- a/src/compat/maestro/runtime-click.ts +++ /dev/null @@ -1,19 +0,0 @@ -import type { DaemonResponse } from '../../daemon/types.ts'; -import type { Point } from '../../kernel/snapshot.ts'; -import type { MaestroRuntimeInvoke, ReplayBaseRequest } from './runtime-support.ts'; - -export async function invokeMaestroClickPoint(params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - point: Point; -}): Promise { - return await params.invoke({ - ...params.baseReq, - command: 'click', - positionals: [String(params.point.x), String(params.point.y)], - flags: { - ...params.baseReq.flags, - postGestureStabilization: true, - }, - }); -} diff --git a/src/compat/maestro/runtime-commands.ts b/src/compat/maestro/runtime-commands.ts deleted file mode 100644 index ad328ee4b..000000000 --- a/src/compat/maestro/runtime-commands.ts +++ /dev/null @@ -1,12 +0,0 @@ -export const MAESTRO_RUNTIME_COMMAND = { - runScript: '__maestroRunScript', - assertVisible: '__maestroAssertVisible', - assertNotVisible: '__maestroAssertNotVisible', - pressEnter: '__maestroPressEnter', - waitForAnimationToEnd: '__maestroWaitForAnimationToEnd', - scrollUntilVisible: '__maestroScrollUntilVisible', - swipeScreen: '__maestroSwipeScreen', - swipeOn: '__maestroSwipeOn', - tapOn: '__maestroTapOn', - tapPointPercent: '__maestroTapPointPercent', -} as const; diff --git a/src/compat/maestro/runtime-flow.ts b/src/compat/maestro/runtime-flow.ts deleted file mode 100644 index 2dbd2916b..000000000 --- a/src/compat/maestro/runtime-flow.ts +++ /dev/null @@ -1,144 +0,0 @@ -import type { DaemonInvokeFn, DaemonResponse, SessionReplayControl } from '../../daemon/types.ts'; -import { getSnapshotReferenceFrame } from '../../daemon/touch-reference-frame.ts'; -import { invokeReplayActionBlock } from '../../replay/control-flow-runtime.ts'; -import { - captureMaestroSnapshot, - errorResponse, - readSnapshotState, - type MaestroReplayInvoker, - type ReplayBaseRequest, -} from './runtime-support.ts'; -import { - readMaestroSelectorPlatform, - resolveVisibleMaestroNodeFromSnapshot, -} from './runtime-targets.ts'; -import { sleep } from '../../utils/timeouts.ts'; - -const MAESTRO_RUN_FLOW_WHEN_POLICY = { - visibleTimeoutMs: 3000, - visiblePollMs: 250, -} as const; - -type MaestroRunFlowWhenControl = Extract; - -export async function invokeMaestroRunFlowWhenControl(params: { - baseReq: ReplayBaseRequest; - control: MaestroRunFlowWhenControl; - line: number; - step: number; - invoke: DaemonInvokeFn; - invokeReplayAction: MaestroReplayInvoker; -}): Promise { - const conditionResult = await evaluateMaestroRunFlowWhenCondition(params, params.control); - if (!conditionResult.ok) return conditionResult.response; - if (!conditionResult.matched) { - return { - ok: true, - data: { skipped: true, condition: params.control.mode, selector: params.control.selector }, - }; - } - return await invokeMaestroRunFlowWhenSteps(params); -} - -async function evaluateMaestroRunFlowWhenCondition( - params: { - baseReq: ReplayBaseRequest; - invoke: DaemonInvokeFn; - }, - condition: MaestroRunFlowWhenControl, -): Promise<{ ok: true; matched: boolean } | { ok: false; response: DaemonResponse }> { - if (condition.mode === 'visible') { - return await waitForMaestroRunFlowVisibleCondition(params, condition); - } - - const result = await readMaestroRunFlowVisibleCondition(params, condition.selector); - if (!result.ok) { - return { - ok: false, - response: result.response, - }; - } - return { ok: true, matched: !result.matched }; -} - -async function waitForMaestroRunFlowVisibleCondition( - params: { - baseReq: ReplayBaseRequest; - invoke: DaemonInvokeFn; - }, - condition: MaestroRunFlowWhenControl, -): Promise<{ ok: true; matched: boolean } | { ok: false; response: DaemonResponse }> { - // Maestro conditionals commonly guard UI that appears immediately after the - // previous command. Keep this bounded and only for visible; notVisible stays - // a point-in-time condition so optional cleanup blocks do not become waits. - const startedAt = Date.now(); - while (true) { - const result = await readMaestroRunFlowVisibleCondition(params, condition.selector); - if (!result.ok) return { ok: false, response: result.response }; - if (result.matched) return { ok: true, matched: true }; - if (Date.now() - startedAt >= MAESTRO_RUN_FLOW_WHEN_POLICY.visibleTimeoutMs) { - return { ok: true, matched: false }; - } - await sleep(MAESTRO_RUN_FLOW_WHEN_POLICY.visiblePollMs); - } -} - -async function readMaestroRunFlowVisibleCondition( - params: { - baseReq: ReplayBaseRequest; - invoke: DaemonInvokeFn; - }, - selector: string, -): Promise<{ ok: true; matched: boolean } | { ok: false; response: DaemonResponse }> { - const response = await captureMaestroSnapshot(params); - if (!response.ok) return { ok: false, response }; - return resolveMaestroRunFlowVisibleCondition(params, selector, response); -} - -function resolveMaestroRunFlowVisibleCondition( - params: { - baseReq: ReplayBaseRequest; - }, - selector: string, - response: Extract, -): { ok: true; matched: boolean } | { ok: false; response: DaemonResponse } { - const snapshot = readSnapshotState(response.data); - if (!snapshot) { - return { - ok: false, - response: errorResponse('COMMAND_FAILED', 'Unable to read snapshot data for runFlow.when.'), - }; - } - const matched = resolveVisibleMaestroNodeFromSnapshot( - snapshot, - selector, - readMaestroSelectorPlatform(params.baseReq.flags), - getSnapshotReferenceFrame(snapshot), - ).ok; - return { ok: true, matched }; -} - -async function invokeMaestroRunFlowWhenSteps(params: { - control: MaestroRunFlowWhenControl; - line: number; - step: number; - invokeReplayAction: MaestroReplayInvoker; -}): Promise { - const response = await invokeReplayActionBlock({ - actions: params.control.actions, - actionSources: params.control.actionSources, - line: params.line, - step: params.step, - invokeReplayAction: params.invokeReplayAction, - }); - if (!response.ok) return response; - - return { - ok: true, - data: { - ran: response.data?.ran, - condition: params.control.mode, - selector: params.control.selector, - }, - }; -} diff --git a/src/compat/maestro/runtime-geometry.ts b/src/compat/maestro/runtime-geometry.ts deleted file mode 100644 index 324672f23..000000000 --- a/src/compat/maestro/runtime-geometry.ts +++ /dev/null @@ -1,87 +0,0 @@ -import { clampToRange } from '../../contracts/scroll-gesture.ts'; -import { pointInsideRect } from '../../utils/rect-center.ts'; -import type { MaestroSnapshotTarget } from './runtime-targets.ts'; - -const MAESTRO_GEOMETRY_POLICY = { - swipe: { - screenRatio: 0.35, - minDistancePx: 120, - maxDistancePx: 360, - marginPx: 8, - }, -} as const; - -export function swipeCoordinatesFromTarget( - target: MaestroSnapshotTarget, - direction: string, -): - | { ok: true; start: { x: number; y: number }; end: { x: number; y: number } } - | { ok: false; message: string } { - const center = pointInsideRect(target.rect); - const frame = target.frame; - const horizontalDistance = swipeDistance(frame?.referenceWidth, target.rect.width); - const verticalDistance = swipeDistance(frame?.referenceHeight, target.rect.height); - const margin = MAESTRO_GEOMETRY_POLICY.swipe.marginPx; - const minX = margin; - const minY = margin; - const maxX = frame ? frame.referenceWidth - margin : center.x + horizontalDistance; - const maxY = frame ? frame.referenceHeight - margin : center.y + verticalDistance; - switch (direction.toLowerCase()) { - case 'up': - return { - ok: true, - start: center, - end: { - x: center.x, - y: clampToRange(center.y - verticalDistance, minY, maxY), - }, - }; - case 'down': - return { - ok: true, - start: center, - end: { - x: center.x, - y: clampToRange(center.y + verticalDistance, minY, maxY), - }, - }; - case 'left': - return { - ok: true, - start: center, - end: { - x: clampToRange(center.x - horizontalDistance, minX, maxX), - y: center.y, - }, - }; - case 'right': - return { - ok: true, - start: center, - end: { - x: clampToRange(center.x + horizontalDistance, minX, maxX), - y: center.y, - }, - }; - default: - return { ok: false, message: 'swipe.label direction must be up, down, left, or right.' }; - } -} - -export function pointForMaestroTapOnTarget(target: MaestroSnapshotTarget): { - x: number; - y: number; -} { - return pointInsideRect(target.rect); -} - -function swipeDistance(frameSize: number | undefined, rectSize: number): number { - const screenRelative = - typeof frameSize === 'number' ? frameSize * MAESTRO_GEOMETRY_POLICY.swipe.screenRatio : 0; - return Math.round( - Math.min( - MAESTRO_GEOMETRY_POLICY.swipe.maxDistancePx, - Math.max(MAESTRO_GEOMETRY_POLICY.swipe.minDistancePx, screenRelative, rectSize * 1.5), - ), - ); -} diff --git a/src/compat/maestro/runtime-interactions.ts b/src/compat/maestro/runtime-interactions.ts deleted file mode 100644 index 8d280e410..000000000 --- a/src/compat/maestro/runtime-interactions.ts +++ /dev/null @@ -1,637 +0,0 @@ -import { getSnapshotReferenceFrame } from '../../daemon/touch-reference-frame.ts'; -import type { DaemonInvokeFn, DaemonResponse } from '../../daemon/types.ts'; -import { - pointFromPercentInFrame, - type GestureReferenceFrame, - type ScrollDirection, -} from '../../contracts/scroll-gesture.ts'; -import type { ReplayVarScope } from '../../replay/vars.ts'; -import type { SnapshotState } from '../../kernel/snapshot.ts'; -import { emitDiagnostic } from '../../utils/diagnostics.ts'; -import { sleep } from '../../utils/timeouts.ts'; -import { invokeMaestroClickPoint } from './runtime-click.ts'; -import { pointForMaestroTapOnTarget, swipeCoordinatesFromTarget } from './runtime-geometry.ts'; -import { - captureMaestroSnapshot, - clearMaestroRecoverableInteraction, - clearMaestroVisibleContext, - errorResponse, - readMaestroVisibleContext, - readSnapshotState, - rememberMaestroRecoverableInteraction, - type FailedDaemonResponse, - type MaestroRuntimeInvoke, - type ReplayBaseRequest, -} from './runtime-support.ts'; -import { - extractMaestroVisibleTextQuery, - readMaestroSelectorPlatform, - resolveMaestroFuzzyTextNodeFromSnapshot, - resolveMaestroNodeFromSnapshot, - resolveVisibleMaestroNodeFromSnapshot, - type MaestroPreferredContext, - type MaestroSnapshotTarget, - type MaestroTapOnOptions, -} from './runtime-targets.ts'; - -const MAESTRO_INTERACTION_POLICY = { - scrollUntilVisibleProbeMs: 500, - tapOnRetryMs: 250, - tapOnTimeoutMs: 30000, - optionalTapOnTimeoutMs: 3000, -} as const; - -type MaestroScrollUntilVisibleParams = { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; -}; - -type MaestroTapOnParams = { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}; - -type MaestroScreenSwipeResolution = - | { - ok: true; - start: { x: number; y: number }; - end: { x: number; y: number }; - durationMs?: string; - } - | { ok: false; response: DaemonResponse }; - -type ResolvedMaestroInteractionTarget = MaestroSnapshotTarget; - -export async function invokeMaestroScrollUntilVisible( - params: MaestroScrollUntilVisibleParams, -): Promise { - const [selector, timeoutValue = '5000', direction = 'down'] = params.positionals; - if (!selector) { - return errorResponse('INVALID_ARGS', 'scrollUntilVisible requires a selector.'); - } - const timeoutMs = Number(timeoutValue); - if (!Number.isFinite(timeoutMs) || timeoutMs <= 0) { - return errorResponse('INVALID_ARGS', 'scrollUntilVisible timeout must be a positive number.'); - } - const fuzzyTextQuery = extractMaestroVisibleTextQuery(selector); - const attempts = Math.max( - 1, - Math.ceil(timeoutMs / MAESTRO_INTERACTION_POLICY.scrollUntilVisibleProbeMs), - ); - let lastWaitResponse: FailedDaemonResponse | null = null; - - for (let index = 0; index < attempts; index += 1) { - const probeResponse = await probeMaestroScrollVisibility( - params, - selector, - fuzzyTextQuery, - scrollProbeMs(timeoutMs, index), - ); - if (probeResponse.ok) return probeResponse; - lastWaitResponse = probeResponse; - - if (index === attempts - 1) break; - - const scrollResponse = await params.invoke({ - ...params.baseReq, - command: 'scroll', - positionals: [direction], - }); - if (!scrollResponse.ok) return scrollResponse; - } - - return withMaestroScrollTimeoutContext(lastWaitResponse, selector, timeoutMs); -} - -export async function invokeMaestroTapPointPercent(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: DaemonInvokeFn; - scope?: ReplayVarScope; -}): Promise { - const [xValue, yValue] = params.positionals; - const xPercent = Number(xValue); - const yPercent = Number(yValue); - if (!Number.isFinite(xPercent) || !Number.isFinite(yPercent)) { - return errorResponse('INVALID_ARGS', 'tapOn percentage point requires numeric x/y values.'); - } - - const snapshotResponse = await captureMaestroSnapshot(params); - if (!snapshotResponse.ok) return snapshotResponse; - - const snapshot = readSnapshotState(snapshotResponse.data); - if (!snapshot) { - return errorResponse( - 'COMMAND_FAILED', - 'Unable to read snapshot data for Maestro percentage point tap.', - ); - } - - const frame = getSnapshotReferenceFrame(snapshot); - if (!frame) { - return errorResponse( - 'COMMAND_FAILED', - 'Unable to resolve screen size for Maestro percentage point tap.', - ); - } - - const point = pointFromPercentInFrame(frame, xPercent, yPercent); - const response = await invokeMaestroClickPoint({ ...params, point }); - if (response.ok) clearMaestroRecoverableInteraction(params.scope); - return response; -} - -export async function invokeMaestroSwipeScreen(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}): Promise { - const presetResponse = await maybeInvokeMaestroDirectionalSwipePreset(params); - if (presetResponse) return presetResponse; - const swipe = await resolveMaestroScreenSwipe(params); - if (!swipe.ok) return swipe.response; - - return await invokeSwipeGesture(params, swipe, swipe.durationMs); -} - -async function maybeInvokeMaestroDirectionalSwipePreset(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}): Promise { - const [mode, direction, durationMs] = params.positionals; - if (mode !== 'direction') return undefined; - if (!isMaestroSwipeDirection(direction)) { - return errorResponse( - 'INVALID_ARGS', - 'Maestro swipe direction must be UP, DOWN, LEFT, or RIGHT.', - ); - } - const input = { - kind: 'swipe', - preset: direction, - ...(durationMs === undefined ? {} : { durationMs: Number(durationMs) }), - }; - const response = await params.invoke({ - ...params.baseReq, - command: 'gesture', - positionals: [], - input, - }); - if (response.ok) { - rememberMaestroRecoverableInteraction(params.scope, { - kind: 'swipe', - command: 'gesture', - input, - }); - } - return response; -} - -export async function invokeMaestroTapOn(params: MaestroTapOnParams): Promise { - const [selector, rawOptions] = params.positionals; - if (!selector) { - return errorResponse('INVALID_ARGS', 'tapOn requires a selector.'); - } - const options = readMaestroTapOnOptions(rawOptions); - if (!options.ok) return options.response; - const startedAt = Date.now(); - const timeoutMs = maestroTapOnTimeoutMs(params); - let lastResponse: DaemonResponse | undefined; - while (Date.now() - startedAt < timeoutMs) { - const attempt = await attemptMaestroTapOn(params, selector, options.value ?? {}); - if (!attempt.retry) return attempt.response; - lastResponse = attempt.response; - await sleep(MAESTRO_INTERACTION_POLICY.tapOnRetryMs); - } - - return maestroTapOnTimeoutResponse(params, selector, lastResponse); -} - -export async function invokeMaestroSwipeOn(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}): Promise { - const [selector, direction = 'up', durationMs] = params.positionals; - if (!selector) return errorResponse('INVALID_ARGS', 'swipe.label requires a label selector.'); - const target = await resolveMaestroInteractionTarget(params, selector, {}, 'swipe.label', { - promoteTapTarget: false, - }); - if (!target.ok) return target.response; - const swipe = swipeCoordinatesFromTarget(target.target, direction); - if (!swipe.ok) return errorResponse('INVALID_ARGS', swipe.message); - return await invokeSwipeGesture(params, swipe, durationMs); -} - -async function invokeSwipeGesture( - params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; - }, - swipe: { - start: { x: number; y: number }; - end: { x: number; y: number }; - }, - durationMs: string | undefined, -): Promise { - const input = { - from: swipe.start, - to: swipe.end, - ...(durationMs === undefined ? {} : { durationMs: Number(durationMs) }), - }; - const response = await params.invoke({ - ...params.baseReq, - command: 'swipe', - positionals: [], - input, - }); - if (response.ok) { - rememberMaestroRecoverableInteraction(params.scope, { - kind: 'swipe', - command: 'swipe', - input, - }); - } - return response; -} - -async function resolveMaestroScreenSwipe(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}): Promise { - const frame = await captureFrameForMaestroScreenSwipe(params); - if (!frame) { - return { - ok: false, - response: errorResponse('COMMAND_FAILED', 'Unable to resolve screen size for Maestro swipe.'), - }; - } - - const [mode, ...args] = params.positionals; - if (mode === 'percent') { - return resolvePercentScreenSwipe(args, frame); - } - return { - ok: false, - response: errorResponse('INVALID_ARGS', 'Maestro screen swipe requires direction or percent.'), - }; -} - -async function captureFrameForMaestroScreenSwipe(params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; -}): Promise { - const snapshotResponse = await captureMaestroSnapshot(params); - if (!snapshotResponse.ok) return undefined; - const snapshot = readSnapshotState(snapshotResponse.data); - return getSnapshotReferenceFrame(snapshot); -} - -function isMaestroSwipeDirection(direction: string | undefined): direction is ScrollDirection { - return ( - direction === 'up' || direction === 'down' || direction === 'left' || direction === 'right' - ); -} - -function resolvePercentScreenSwipe( - args: string[], - frame: GestureReferenceFrame, -): MaestroScreenSwipeResolution { - const [startX, startY, endX, endY, durationMs] = args; - const values = [startX, startY, endX, endY].map(Number); - if (values.some((value) => !Number.isFinite(value))) { - return { - ok: false, - response: errorResponse('INVALID_ARGS', 'Maestro percentage swipe requires numeric points.'), - }; - } - const [x1, y1, x2, y2] = values as [number, number, number, number]; - return { - ok: true, - start: pointFromPercentInFrame(frame, x1, y1), - end: pointFromPercentInFrame(frame, x2, y2), - durationMs, - }; -} - -async function probeMaestroScrollVisibility( - params: MaestroScrollUntilVisibleParams, - selector: string, - fuzzyTextQuery: string | null, - probeMs: number, -): Promise { - const waitResponse = await params.invoke({ - ...params.baseReq, - command: 'wait', - positionals: [selector, String(probeMs)], - }); - if (waitResponse.ok || !fuzzyTextQuery) return waitResponse; - - const fuzzyResponse = await params.invoke({ - ...params.baseReq, - command: 'find', - positionals: [fuzzyTextQuery, 'wait', String(probeMs)], - }); - return fuzzyResponse; -} - -function scrollProbeMs(timeoutMs: number, index: number): number { - return Math.min( - MAESTRO_INTERACTION_POLICY.scrollUntilVisibleProbeMs, - Math.max(1, timeoutMs - index * MAESTRO_INTERACTION_POLICY.scrollUntilVisibleProbeMs), - ); -} - -function maestroTapOnTimeoutMs(params: MaestroTapOnParams): number { - return params.baseReq.flags?.maestro?.optional === true - ? MAESTRO_INTERACTION_POLICY.optionalTapOnTimeoutMs - : MAESTRO_INTERACTION_POLICY.tapOnTimeoutMs; -} - -function maestroTapOnTimeoutResponse( - params: MaestroTapOnParams, - selector: string, - lastResponse: DaemonResponse | undefined, -): DaemonResponse { - if (params.baseReq.flags?.maestro?.optional === true) { - return { ok: true, data: { skipped: true, optional: true, selector } }; - } - return ( - lastResponse ?? errorResponse('COMMAND_FAILED', `tapOn timed out for selector: ${selector}`) - ); -} - -async function attemptMaestroTapOn( - params: MaestroTapOnParams, - selector: string, - options: MaestroTapOnOptions, -): Promise< - { retry: false; response: DaemonResponse } | { retry: true; response: FailedDaemonResponse } -> { - const fuzzyTextQuery = extractMaestroVisibleTextQuery(selector); - const attempt = await invokeMaestroResolvedTapOn(params, selector, options); - if (attempt.response.ok) return { retry: false, response: attempt.response }; - if (attempt.targetResolved && fuzzyTextQuery) { - return await invokeMaestroFuzzyTapOn(params, fuzzyTextQuery); - } - return { retry: true, response: attempt.response }; -} - -async function invokeMaestroResolvedTapOn( - params: MaestroTapOnParams, - selector: string, - options: MaestroTapOnOptions, -): Promise<{ response: DaemonResponse; targetResolved: boolean }> { - const target = await resolveMaestroInteractionTarget(params, selector, options, 'tapOn', { - promoteTapTarget: true, - }); - if (!target.ok) return { response: target.response, targetResolved: false }; - return await clickMaestroResolvedTarget(params, selector, target.target, options); -} - -async function clickMaestroResolvedTarget( - params: MaestroTapOnParams, - selector: string, - target: ResolvedMaestroInteractionTarget, - options: MaestroTapOnOptions, -): Promise<{ response: DaemonResponse; targetResolved: true }> { - const point = pointForMaestroTapOnTarget(target); - emitDiagnostic({ - level: 'debug', - phase: 'maestro_tap_target', - data: { - selector, - node: { - index: target.node.index, - type: target.node.type, - label: target.node.label, - value: target.node.value, - identifier: target.node.identifier, - visibleToUser: target.node.visibleToUser, - }, - rect: target.rect, - point, - }, - }); - const response = await invokeMaestroClickPoint({ ...params, point }); - if (response.ok) { - clearMaestroVisibleContext(params.scope); - rememberMaestroRecoverableInteraction(params.scope, { - kind: 'tap', - selector, - point, - options: { ...options }, - }); - } - return { - response, - targetResolved: true, - }; -} - -async function invokeMaestroFuzzyTapOn( - params: MaestroTapOnParams, - query: string, -): Promise< - { retry: false; response: DaemonResponse } | { retry: true; response: FailedDaemonResponse } -> { - const findResponse = await params.invoke({ - ...params.baseReq, - command: 'find', - positionals: [query, 'click'], - flags: { - ...params.baseReq.flags, - findFirst: true, - postGestureStabilization: true, - }, - }); - emitDiagnostic({ - level: findResponse.ok ? 'info' : 'debug', - phase: 'maestro_fuzzy_tap_fallback', - data: { - query, - ok: findResponse.ok, - }, - }); - if (findResponse.ok) { - clearMaestroRecoverableInteraction(params.scope); - return { retry: false, response: findResponse }; - } - return { retry: true, response: findResponse }; -} - -async function resolveMaestroInteractionTarget( - params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; - }, - selector: string, - options: MaestroTapOnOptions, - commandLabel: string, - resolutionOptions: { promoteTapTarget: boolean }, -): Promise< - | { ok: true; target: ResolvedMaestroInteractionTarget; snapshot: SnapshotState } - | { ok: false; response: DaemonResponse } -> { - const snapshotResponse = await captureMaestroSnapshot({ ...params, raw: true }); - return resolveMaestroInteractionTargetFromResponse( - params, - selector, - options, - commandLabel, - resolutionOptions, - snapshotResponse, - ); -} - -function resolveMaestroInteractionTargetFromResponse( - params: { - baseReq: ReplayBaseRequest; - scope?: ReplayVarScope; - }, - selector: string, - options: MaestroTapOnOptions, - commandLabel: string, - resolutionOptions: { promoteTapTarget: boolean }, - snapshotResponse: DaemonResponse, -): - | { ok: true; target: ResolvedMaestroInteractionTarget; snapshot: SnapshotState } - | { ok: false; response: DaemonResponse } { - if (!snapshotResponse.ok) return { ok: false, response: snapshotResponse }; - const snapshot = readSnapshotState(snapshotResponse.data); - if (!snapshot) { - return { - ok: false, - response: errorResponse( - 'COMMAND_FAILED', - `Unable to read snapshot data for ${commandLabel}.`, - ), - }; - } - - const frame = getSnapshotReferenceFrame(snapshot); - const platform = readMaestroSelectorPlatform(params.baseReq.flags); - const preferredContext = resolvePreferredMaestroContext(params, snapshot, platform, frame); - const resolution = resolveMaestroNodeFromSnapshot(snapshot, selector, options, platform, frame, { - ...resolutionOptions, - preferredContext, - requireOnScreen: true, - }); - if (!resolution.ok) { - const fuzzyTextQuery = extractMaestroVisibleTextQuery(selector); - if (fuzzyTextQuery) { - const fuzzyResolution = resolveMaestroFuzzyTextNodeFromSnapshot( - snapshot, - fuzzyTextQuery, - platform, - frame, - { ...resolutionOptions, preferredContext, requireOnScreen: true }, - ); - if (fuzzyResolution.ok) { - return { - ok: true, - target: { - node: fuzzyResolution.node, - rect: fuzzyResolution.rect, - frame, - }, - snapshot, - }; - } - } - } - if (!resolution.ok) { - return { - ok: false, - response: errorResponse('ELEMENT_NOT_FOUND', resolution.message, { - selector, - options, - command: commandLabel, - }), - }; - } - return { - ok: true, - target: { - node: resolution.node, - rect: resolution.rect, - frame, - }, - snapshot, - }; -} - -function resolvePreferredMaestroContext( - params: { baseReq: ReplayBaseRequest; scope?: ReplayVarScope }, - snapshot: NonNullable>, - platform: ReturnType, - frame: ReturnType, -): MaestroPreferredContext | undefined { - const context = readMaestroVisibleContext(params.scope); - if (!context) return undefined; - const target = resolveVisibleMaestroNodeFromSnapshot(snapshot, context.selector, platform, frame); - if (!target.ok) return undefined; - emitDiagnostic({ - level: 'debug', - phase: 'maestro_preferred_context', - data: { - selector: context.selector, - node: { - index: target.node.index, - type: target.node.type, - label: target.node.label, - value: target.node.value, - identifier: target.node.identifier, - }, - rect: target.rect, - }, - }); - return { node: target.node, rect: target.rect }; -} - -function readMaestroTapOnOptions( - rawOptions: string | undefined, -): { ok: true; value: MaestroTapOnOptions | null } | { ok: false; response: DaemonResponse } { - if (!rawOptions) return { ok: true, value: null }; - try { - const value = JSON.parse(rawOptions) as MaestroTapOnOptions; - return { ok: true, value }; - } catch { - return { - ok: false, - response: errorResponse('INVALID_ARGS', 'tapOn runtime options must be valid JSON.'), - }; - } -} - -function withMaestroScrollTimeoutContext( - response: FailedDaemonResponse | null, - selector: string, - timeoutMs: number, -): DaemonResponse { - if (!response) { - return errorResponse( - 'COMMAND_FAILED', - `scrollUntilVisible timed out after ${timeoutMs}ms for selector: ${selector}`, - ); - } - return { - ok: false, - error: { - ...response.error, - message: `scrollUntilVisible timed out after ${timeoutMs}ms for selector: ${selector}. Last wait: ${response.error.message}`, - }, - }; -} diff --git a/src/compat/maestro/runtime-port-commands.ts b/src/compat/maestro/runtime-port-commands.ts new file mode 100644 index 000000000..dc7201649 --- /dev/null +++ b/src/compat/maestro/runtime-port-commands.ts @@ -0,0 +1,445 @@ +import { AppError } from '../../kernel/errors.ts'; +import { pointInsideRect } from '../../utils/rect-center.ts'; +import type { MaestroGestureTarget } from './program-ir.ts'; +import type { + MaestroObservation, + MaestroRuntimeCommand, + MaestroRuntimeRequest, + MaestroRuntimeResult, +} from './engine-types.ts'; +import { operationContext } from './runtime-port-context.ts'; +import { observationForTarget, resolveMaestroTarget } from './runtime-port-observation.ts'; +import { resolveMaestroCoordinate, resolveMaestroSwipeOperation } from './runtime-port-geometry.ts'; +import type { + MaestroInputTarget, + MaestroRuntimeOperationContext, + MaestroRuntimeOperationResult, + MaestroRuntimeOperations, + MaestroTargetQuery, +} from './runtime-port-types.ts'; + +const DEFAULT_SCROLL_UNTIL_VISIBLE_TIMEOUT_MS = 5_000; +const MAESTRO_INPUT_TARGET_TIMEOUT_MS = 30_000; +const MAESTRO_OPTIONAL_INPUT_TARGET_TIMEOUT_MS = 3_000; + +type MaestroCommandOf = Extract< + MaestroRuntimeCommand, + { kind: K } +>; + +type MaestroLifecycleCommand = MaestroCommandOf<'launchApp' | 'stopApp' | 'openLink'>; +type MaestroTargetCommand = MaestroCommandOf<'tapOn' | 'doubleTapOn' | 'longPressOn'>; +type MaestroTextCommand = MaestroCommandOf<'inputText' | 'eraseText' | 'pasteText'>; +type MaestroNavigationCommand = MaestroCommandOf< + 'scroll' | 'scrollUntilVisible' | 'hideKeyboard' | 'pressKey' | 'back' | 'waitForAnimationToEnd' +>; +type MaestroSupportCommand = MaestroCommandOf<'takeScreenshot' | 'runScript'>; +type MaestroObservationCommand = MaestroCommandOf< + 'assertVisible' | 'assertNotVisible' | 'extendedWaitUntil' +>; +type MaestroCommandKind = MaestroRuntimeCommand['kind']; +type MaestroRuntimeCommandHandler = ( + command: MaestroCommandOf, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +) => Promise; +type MaestroRuntimeCommandHandlers = { + [K in MaestroCommandKind]: MaestroRuntimeCommandHandler; +}; + +const MAESTRO_RUNTIME_COMMAND_HANDLERS = { + launchApp: executeLifecycleCommand, + stopApp: executeLifecycleCommand, + openLink: executeLifecycleCommand, + tapOn: executeTargetCommand, + doubleTapOn: executeTargetCommand, + longPressOn: executeTargetCommand, + swipe: executeSwipeCommand, + inputText: executeTextCommand, + eraseText: executeTextCommand, + pasteText: executeTextCommand, + scroll: executeNavigationCommand, + scrollUntilVisible: executeNavigationCommand, + hideKeyboard: executeNavigationCommand, + pressKey: executeNavigationCommand, + back: executeNavigationCommand, + waitForAnimationToEnd: executeNavigationCommand, + takeScreenshot: executeSupportCommand, + runScript: executeSupportCommand, + assertVisible: executeObservationCommand, + assertNotVisible: executeObservationCommand, + extendedWaitUntil: executeObservationCommand, +} satisfies MaestroRuntimeCommandHandlers; + +export async function executeMaestroRuntimeCommand( + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, +): Promise { + const command = request.command; + const context = operationContext(request, command); + return await dispatchMaestroRuntimeCommand(command, request, operations, context); +} + +function dispatchMaestroRuntimeCommand( + command: MaestroCommandOf, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + const handler = MAESTRO_RUNTIME_COMMAND_HANDLERS[command.kind] as MaestroRuntimeCommandHandler; + return handler(command, request, operations, context); +} + +async function executeLifecycleCommand( + command: MaestroLifecycleCommand, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + switch (command.kind) { + case 'launchApp': + return await invokeMutation( + operations.launchApp, + launchAppInput(command, request), + context, + true, + ); + case 'stopApp': + return await invokeMutation( + operations.stopApp, + { appId: command.appId ?? request.appId }, + context, + true, + ); + case 'openLink': + return await invokeMutation(operations.openLink, { link: command.link }, context, true); + } +} + +function launchAppInput(command: MaestroCommandOf<'launchApp'>, request: MaestroRuntimeRequest) { + return { + appId: command.appId ?? request.appId, + ...(command.stopApp === undefined ? {} : { stopApp: command.stopApp }), + ...(command.clearState === undefined ? {} : { clearState: command.clearState }), + ...(command.arguments === undefined ? {} : { arguments: command.arguments }), + ...(command.launchArguments === undefined ? {} : { launchArguments: command.launchArguments }), + }; +} + +async function executeTargetCommand( + command: MaestroTargetCommand, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + switch (command.kind) { + case 'tapOn': + return await executeTapOnCommand(command, request, operations, context); + case 'doubleTapOn': { + const target = await resolveInputTarget( + command.target, + { purpose: 'doubleTap', timeoutMs: MAESTRO_INPUT_TARGET_TIMEOUT_MS }, + request, + operations, + ); + return await invokeMutation( + operations.doubleTapOn, + { target, ...(command.delay === undefined ? {} : { delay: command.delay }) }, + context, + true, + target.resolution ? observationForTarget(target.resolution) : undefined, + ); + } + case 'longPressOn': { + const target = await resolveInputTarget( + command.target, + { purpose: 'longPress', timeoutMs: MAESTRO_INPUT_TARGET_TIMEOUT_MS }, + request, + operations, + ); + return await invokeMutation( + operations.longPressOn, + { target }, + context, + true, + target.resolution ? observationForTarget(target.resolution) : undefined, + ); + } + } +} + +async function executeTapOnCommand( + command: MaestroCommandOf<'tapOn'>, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + const target = await resolveTapOnTarget(command, request, operations); + if (!target) return { mutated: false }; + return await invokeMutation( + operations.tapOn, + tapOnInput(command, target), + context, + true, + target.resolution ? observationForTarget(target.resolution) : undefined, + ); +} + +async function resolveTapOnTarget( + command: MaestroCommandOf<'tapOn'>, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, +): Promise { + const query = { + purpose: 'tap' as const, + timeoutMs: + command.optional === true + ? MAESTRO_OPTIONAL_INPUT_TARGET_TIMEOUT_MS + : MAESTRO_INPUT_TARGET_TIMEOUT_MS, + index: command.index, + childOf: command.childOf, + allowAtomicSelectorDispatch: command.repeat === undefined && command.delay === undefined, + }; + return command.optional === true + ? await resolveInputTarget(command.target, query, request, operations, true) + : await resolveInputTarget(command.target, query, request, operations); +} + +function tapOnInput(command: MaestroCommandOf<'tapOn'>, target: MaestroInputTarget) { + return { + target, + ...(command.repeat === undefined ? {} : { repeat: command.repeat }), + ...(command.delay === undefined ? {} : { delay: command.delay }), + ...(command.optional === undefined ? {} : { optional: command.optional }), + ...(command.label === undefined ? {} : { label: command.label }), + ...(command.index === undefined ? {} : { index: command.index }), + ...(command.childOf === undefined ? {} : { childOf: command.childOf }), + }; +} + +async function executeSwipeCommand( + command: MaestroCommandOf<'swipe'>, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + const swipe = await resolveMaestroSwipeOperation(command.gesture, request, operations); + return await invokeMutation( + operations.gesture, + swipe.gesture, + { + ...context, + authoredSwipe: swipe.authored, + ...(swipe.target ? { swipeTarget: swipe.target } : {}), + ...(swipe.viewport ? { gestureViewport: swipe.viewport } : {}), + }, + true, + swipe.target ? observationForTarget(swipe.target) : undefined, + ); +} + +async function executeTextCommand( + command: MaestroTextCommand, + _request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + switch (command.kind) { + case 'inputText': + return await invokeMutation( + operations.inputText, + { text: command.text, ...(command.label === undefined ? {} : { label: command.label }) }, + context, + true, + ); + case 'eraseText': + return await invokeMutation( + operations.eraseText, + { + ...(command.charactersToErase === undefined + ? {} + : { charactersToErase: command.charactersToErase }), + }, + context, + true, + ); + case 'pasteText': + return await invokeMutation(operations.pasteText, { text: command.text }, context, true); + } +} + +async function executeNavigationCommand( + command: MaestroNavigationCommand, + _request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + switch (command.kind) { + case 'scroll': + return await invokeMutation(operations.scroll, { direction: 'down' }, context, true); + case 'scrollUntilVisible': + return await invokeMutation( + operations.scrollUntilVisible, + scrollUntilVisibleInput(command), + context, + true, + ); + case 'hideKeyboard': + return await invokeMutation(operations.hideKeyboard, {}, context, true); + case 'pressKey': + return await invokeMutation(operations.pressKey, { key: command.key }, context, true); + case 'back': + return await invokeMutation(operations.back, {}, context, true); + case 'waitForAnimationToEnd': + return await invokeMutation( + operations.waitForAnimationToEnd, + waitForAnimationToEndInput(command), + context, + true, + ); + } +} + +function scrollUntilVisibleInput(command: MaestroCommandOf<'scrollUntilVisible'>) { + return { + selector: command.element, + direction: command.direction ?? 'down', + timeoutMs: command.timeout ?? DEFAULT_SCROLL_UNTIL_VISIBLE_TIMEOUT_MS, + }; +} + +function waitForAnimationToEndInput(command: MaestroCommandOf<'waitForAnimationToEnd'>) { + return { ...(command.timeout === undefined ? {} : { timeoutMs: command.timeout }) }; +} + +async function executeSupportCommand( + command: MaestroSupportCommand, + _request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + context: MaestroRuntimeOperationContext, +): Promise { + switch (command.kind) { + case 'takeScreenshot': { + const result = await operations.takeScreenshot({ path: command.path }, context); + return resultWithArtifacts(false, result, [], undefined, context.generation); + } + case 'runScript': + return await invokeMutation( + operations.runScript, + { + file: command.file, + ...(command.env === undefined ? {} : { env: command.env }), + }, + context, + true, + ); + } +} + +async function executeObservationCommand(command: MaestroObservationCommand): Promise { + throw new AppError( + 'COMMAND_FAILED', + `Maestro ${command.kind} must be executed by the observation engine.`, + ); +} + +async function invokeMutation( + operation: ( + input: TInput, + context: MaestroRuntimeOperationContext, + ) => Promise, + input: TInput, + context: MaestroRuntimeOperationContext, + mutated: boolean, + observation?: MaestroObservation, +): Promise { + const result = await operation(input, context); + return resultWithArtifacts(mutated, result, [], observation, context.generation); +} + +function resultWithArtifacts( + mutated: boolean, + result: MaestroRuntimeOperationResult | void, + defaultArtifacts: readonly string[], + observation?: MaestroObservation, + generation?: number, +): MaestroRuntimeResult { + const operationObservation = result?.observation; + assertObservationGeneration(operationObservation, generation); + return { + mutated, + ...optionalObservation(observation ?? operationObservation), + ...optionalOutputEnv(result?.outputEnv), + ...optionalArtifactPaths(defaultArtifacts, result?.artifactPaths), + }; +} + +function assertObservationGeneration( + observation: MaestroObservation | undefined, + generation: number | undefined, +): void { + if (!observation || generation === undefined || observation.generation === generation) return; + throw new AppError( + 'COMMAND_FAILED', + `Maestro operation evidence generation ${observation.generation} does not match ${generation}.`, + ); +} + +function optionalObservation( + observation: MaestroObservation | undefined, +): Partial> { + return observation ? { observation } : {}; +} + +function optionalOutputEnv( + outputEnv: Record | undefined, +): Partial> { + return outputEnv ? { outputEnv: { ...outputEnv } } : {}; +} + +function optionalArtifactPaths( + defaultArtifacts: readonly string[], + operationArtifacts: readonly string[] | undefined, +): Partial> { + const artifactPaths = [...new Set([...defaultArtifacts, ...(operationArtifacts ?? [])])]; + return artifactPaths.length > 0 ? { artifactPaths } : {}; +} + +function resolveInputTarget( + authored: MaestroGestureTarget, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional: true, +): Promise; +function resolveInputTarget( + authored: MaestroGestureTarget, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional?: false, +): Promise; +async function resolveInputTarget( + authored: MaestroGestureTarget, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional = false, +): Promise { + if (authored.space === 'target') { + const resolution = optional + ? await resolveMaestroTarget(authored.selector, query, request, operations, true) + : await resolveMaestroTarget(authored.selector, query, request, operations); + if (!resolution) return undefined; + return { + authored, + point: pointInsideRect(resolution.rect), + resolution, + }; + } + return { + authored, + point: await resolveMaestroCoordinate(authored, request, operations), + }; +} diff --git a/src/compat/maestro/runtime-port-context.ts b/src/compat/maestro/runtime-port-context.ts new file mode 100644 index 000000000..1751fe884 --- /dev/null +++ b/src/compat/maestro/runtime-port-context.ts @@ -0,0 +1,20 @@ +import type { MaestroRuntimeRequest } from './engine-types.ts'; +import type { MaestroCommand } from './program-ir.ts'; +import type { MaestroRuntimeOperationContext } from './runtime-port-types.ts'; + +export function operationContext( + request: Pick< + MaestroRuntimeRequest, + 'appId' | 'env' | 'generation' | 'cachedObservation' | 'signal' + >, + command?: Pick, +): MaestroRuntimeOperationContext { + return { + ...(request.appId === undefined ? {} : { appId: request.appId }), + env: request.env, + generation: request.generation, + ...(command ? { source: command.source } : {}), + ...(request.cachedObservation ? { cachedObservation: request.cachedObservation } : {}), + ...(request.signal ? { signal: request.signal } : {}), + }; +} diff --git a/src/compat/maestro/runtime-port-geometry-policy.ts b/src/compat/maestro/runtime-port-geometry-policy.ts new file mode 100644 index 000000000..1511f785f --- /dev/null +++ b/src/compat/maestro/runtime-port-geometry-policy.ts @@ -0,0 +1,17 @@ +import type { MaestroDirection } from './program-ir.ts'; + +// Maestro's command model supplies this value even when YAML omits `duration`. +export const MAESTRO_DEFAULT_SWIPE_DURATION_MS = 400; + +/** + * Target-relative swipes need enough travel to move a small target, but must + * remain bounded and inside the viewport to avoid system-edge gesture zones. + */ +export const MAESTRO_TARGET_SWIPE_POLICY = { + defaultDirection: 'up' as MaestroDirection, + viewportMarginPx: 8, + screenTravelFraction: 0.35, + targetTravelMultiplier: 1.5, + minTravelPx: 120, + maxTravelPx: 360, +} as const; diff --git a/src/compat/maestro/runtime-port-geometry.ts b/src/compat/maestro/runtime-port-geometry.ts new file mode 100644 index 000000000..b32a9d430 --- /dev/null +++ b/src/compat/maestro/runtime-port-geometry.ts @@ -0,0 +1,222 @@ +import { AppError } from '../../kernel/errors.ts'; +import { normalizePublicSwipeMotion } from '../../contracts/gesture-normalization.ts'; +import { buildInPageSwipeGesturePlan, clampToRange } from '../../contracts/scroll-gesture.ts'; +import { pointInsideRect } from '../../utils/rect-center.ts'; +import type { MaestroRuntimeRequest } from './engine-types.ts'; +import type { MaestroCoordinate, MaestroDirection, MaestroSwipeGesture } from './program-ir.ts'; +import { operationContext } from './runtime-port-context.ts'; +import { resolveMaestroTarget } from './runtime-port-observation.ts'; +import { + MAESTRO_DEFAULT_SWIPE_DURATION_MS, + MAESTRO_TARGET_SWIPE_POLICY, +} from './runtime-port-geometry-policy.ts'; +import type { + MaestroRuntimeOperations, + MaestroSinglePointerGestureInput, + MaestroSwipeOperation, + MaestroTargetResolution, +} from './runtime-port-types.ts'; + +const MAESTRO_SWIPE_TARGET_TIMEOUT_MS = 0; + +export async function resolveMaestroSwipeOperation( + authored: MaestroSwipeGesture, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, +): Promise { + if (authored.kind === 'coordinates') { + if (authored.start.space !== authored.end.space) { + throw new AppError( + 'INVALID_ARGS', + 'Maestro swipe endpoints must use the same coordinate space.', + ); + } + const viewport = + authored.start.space === 'percent' + ? await operations.resolveGestureViewport(operationContext(request, request.command)) + : undefined; + const start = await resolveMaestroCoordinate(authored.start, request, operations, viewport); + const end = await resolveMaestroCoordinate(authored.end, request, operations, viewport); + return { + authored, + gesture: normalizedSwipeFromEndpoints(start, end, authored.duration), + ...(viewport ? { viewport } : {}), + }; + } + + if (authored.kind === 'screen') { + if (authored.direction === 'left' || authored.direction === 'right') { + return { + authored, + gesture: normalizedScreenHorizontalSwipe(authored.direction, authored.duration), + }; + } + const viewport = await operations.resolveGestureViewport( + operationContext(request, request.command), + ); + const { start, end } = screenSwipeEndpoints(viewport, authored.direction); + return { + authored, + gesture: normalizedScreenVerticalSwipe(start, end, authored.duration), + viewport, + }; + } + + const target = await resolveMaestroTarget( + authored.from, + { purpose: 'swipe', timeoutMs: MAESTRO_SWIPE_TARGET_TIMEOUT_MS }, + request, + operations, + ); + const viewport = + target.viewport ?? + (await operations.resolveGestureViewport(operationContext(request, request.command))); + const { start, end } = targetSwipeEndpoints( + target, + authored.direction ?? MAESTRO_TARGET_SWIPE_POLICY.defaultDirection, + viewport, + ); + return { + authored, + gesture: normalizedSwipeFromEndpoints(start, end, authored.duration), + target, + viewport, + }; +} + +export async function resolveMaestroCoordinate( + coordinate: MaestroCoordinate, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + knownViewport?: { + readonly x: number; + readonly y: number; + readonly width: number; + readonly height: number; + }, +): Promise<{ x: number; y: number }> { + if (coordinate.space === 'absolute') return { x: coordinate.x, y: coordinate.y }; + const viewport = + knownViewport ?? + (await operations.resolveGestureViewport(operationContext(request, request.command))); + return { + x: Math.round(viewport.x + (viewport.width * coordinate.x) / 100), + y: Math.round(viewport.y + (viewport.height * coordinate.y) / 100), + }; +} + +function normalizedSwipeFromEndpoints( + start: { x: number; y: number }, + end: { x: number; y: number }, + durationMs: number | undefined, +): MaestroSinglePointerGestureInput { + const gesture = normalizePublicSwipeMotion({ + from: start, + to: end, + durationMs: durationMs ?? MAESTRO_DEFAULT_SWIPE_DURATION_MS, + }).gesture; + if (gesture.intent === 'pan' && 'origin' in gesture) { + return { + intent: 'pan', + origin: gesture.origin, + delta: gesture.delta, + durationMs: gesture.durationMs, + executionProfile: 'endpoint-hold', + }; + } + throw new AppError('COMMAND_FAILED', 'Swipe normalization produced a non-swipe gesture.'); +} + +function normalizedScreenHorizontalSwipe( + direction: Extract, + durationMs: number | undefined, +): MaestroSinglePointerGestureInput { + return { + intent: 'pan', + preset: direction, + durationMs: durationMs ?? MAESTRO_DEFAULT_SWIPE_DURATION_MS, + executionProfile: 'endpoint-hold', + }; +} + +function normalizedScreenVerticalSwipe( + start: { x: number; y: number }, + end: { x: number; y: number }, + durationMs: number | undefined, +): MaestroSinglePointerGestureInput { + return normalizedSwipeFromEndpoints(start, end, durationMs); +} + +function screenSwipeEndpoints( + viewport: { + readonly x: number; + readonly y: number; + readonly width: number; + readonly height: number; + }, + direction: MaestroDirection, +): { start: { x: number; y: number }; end: { x: number; y: number } } { + const plan = buildInPageSwipeGesturePlan(direction, { + referenceWidth: viewport.width, + referenceHeight: viewport.height, + }); + return { + start: { x: viewport.x + plan.x1, y: viewport.y + plan.y1 }, + end: { x: viewport.x + plan.x2, y: viewport.y + plan.y2 }, + }; +} + +function targetSwipeEndpoints( + target: MaestroTargetResolution, + direction: MaestroDirection, + viewport: { + readonly x: number; + readonly y: number; + readonly width: number; + readonly height: number; + }, +): { start: { x: number; y: number }; end: { x: number; y: number } } { + const start = pointInsideRect(target.rect); + const horizontalDistance = swipeDistance(viewport.width, target.rect.width); + const verticalDistance = swipeDistance(viewport.height, target.rect.height); + const minX = viewport.x + MAESTRO_TARGET_SWIPE_POLICY.viewportMarginPx; + const minY = viewport.y + MAESTRO_TARGET_SWIPE_POLICY.viewportMarginPx; + const maxX = viewport.x + viewport.width - MAESTRO_TARGET_SWIPE_POLICY.viewportMarginPx; + const maxY = viewport.y + viewport.height - MAESTRO_TARGET_SWIPE_POLICY.viewportMarginPx; + switch (direction) { + case 'up': + return { + start, + end: { x: start.x, y: clampToRange(start.y - verticalDistance, minY, maxY) }, + }; + case 'down': + return { + start, + end: { x: start.x, y: clampToRange(start.y + verticalDistance, minY, maxY) }, + }; + case 'left': + return { + start, + end: { x: clampToRange(start.x - horizontalDistance, minX, maxX), y: start.y }, + }; + case 'right': + return { + start, + end: { x: clampToRange(start.x + horizontalDistance, minX, maxX), y: start.y }, + }; + } +} + +function swipeDistance(viewportSize: number, targetSize: number): number { + const screenRelative = viewportSize * MAESTRO_TARGET_SWIPE_POLICY.screenTravelFraction; + return Math.round( + Math.min( + MAESTRO_TARGET_SWIPE_POLICY.maxTravelPx, + Math.max( + MAESTRO_TARGET_SWIPE_POLICY.minTravelPx, + screenRelative, + targetSize * MAESTRO_TARGET_SWIPE_POLICY.targetTravelMultiplier, + ), + ), + ); +} diff --git a/src/compat/maestro/runtime-port-observation.ts b/src/compat/maestro/runtime-port-observation.ts new file mode 100644 index 000000000..942566d5c --- /dev/null +++ b/src/compat/maestro/runtime-port-observation.ts @@ -0,0 +1,138 @@ +import { AppError } from '../../kernel/errors.ts'; +import type { + MaestroObservation, + MaestroObservationCondition, + MaestroRuntimeRequest, +} from './engine-types.ts'; +import type { MaestroSelector } from './program-ir.ts'; +import { operationContext } from './runtime-port-context.ts'; +import type { + MaestroRuntimeOperations, + MaestroSelectorEvidence, + MaestroTargetMatch, + MaestroTargetQuery, + MaestroTargetResolution, +} from './runtime-port-types.ts'; + +export async function observeMaestroCondition( + request: { + condition: MaestroObservationCondition; + timeoutMs: number; + generation: number; + env: Readonly>; + cachedObservation?: MaestroObservation; + signal?: AbortSignal; + }, + operations: MaestroRuntimeOperations, +): Promise { + const match = validateTargetMatch( + await operations.observe( + { condition: request.condition, timeoutMs: request.timeoutMs }, + operationContext(request), + ), + request.generation, + ); + const evidence: MaestroSelectorEvidence = { + kind: 'selector', + selector: request.condition.selector, + visible: match.visible, + ...(match.rect ? { frame: match.rect } : {}), + candidateCount: match.candidateCount, + ...(match.ref ? { ref: match.ref } : {}), + }; + return { + generation: request.generation, + matched: + request.condition.kind === 'visible' + ? match.matched && match.visible + : !match.matched || !match.visible, + candidateCount: match.candidateCount, + evidence, + }; +} + +export function resolveMaestroTarget( + selector: MaestroSelector, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional: true, +): Promise; +export function resolveMaestroTarget( + selector: MaestroSelector, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional?: false, +): Promise; +export async function resolveMaestroTarget( + selector: MaestroSelector, + query: Pick, + request: MaestroRuntimeRequest, + operations: MaestroRuntimeOperations, + optional = false, +): Promise { + const match = await operations.resolveTarget( + { selector, ...query }, + operationContext(request, request.command), + ); + const validated = validateTargetMatch(match, request.generation); + if (!validated.matched || !validated.visible || !validated.rect) { + if (optional) return undefined; + throw new AppError('COMMAND_FAILED', 'Maestro target did not resolve to a visible element.', { + selector, + candidateCount: validated.candidateCount, + }); + } + return { + kind: 'selector', + selector, + query: { selector, ...query }, + ...validated, + rect: validated.rect, + }; +} + +export function observationForTarget(target: MaestroTargetResolution): MaestroObservation { + const evidence: MaestroSelectorEvidence = { + kind: 'selector', + selector: target.selector, + visible: target.visible, + frame: target.rect, + candidateCount: target.candidateCount, + ...(target.ref ? { ref: target.ref } : {}), + }; + return { + generation: target.generation, + matched: target.matched && target.visible, + candidateCount: target.candidateCount, + evidence, + }; +} + +function isRect(value: unknown): value is { x: number; y: number; width: number; height: number } { + if (!value || typeof value !== 'object') return false; + const rect = value as Record; + return ['x', 'y', 'width', 'height'].every( + (key) => typeof rect[key] === 'number' && Number.isFinite(rect[key]), + ); +} + +function validateTargetMatch(match: MaestroTargetMatch, generation: number): MaestroTargetMatch { + if (match.generation !== generation) { + throw new AppError( + 'COMMAND_FAILED', + `Maestro target evidence generation ${match.generation} does not match ${generation}.`, + ); + } + if (!Number.isInteger(match.candidateCount) || match.candidateCount < 0) { + throw new AppError('COMMAND_FAILED', 'Maestro target evidence has an invalid candidate count.'); + } + if ( + (match.rect !== undefined && !isRect(match.rect)) || + (match.viewport !== undefined && !isRect(match.viewport)) + ) { + throw new AppError('COMMAND_FAILED', 'Maestro target evidence has invalid geometry.'); + } + return match; +} diff --git a/src/compat/maestro/runtime-port-types.ts b/src/compat/maestro/runtime-port-types.ts new file mode 100644 index 000000000..bbf9a0efd --- /dev/null +++ b/src/compat/maestro/runtime-port-types.ts @@ -0,0 +1,160 @@ +import type { GestureSemanticInput } from '../../contracts/gesture-plan-types.ts'; +import type { SwipePreset } from '../../contracts/scroll-gesture.ts'; +import type { Point, Rect } from '../../kernel/snapshot.ts'; +import type { + MaestroDirection, + MaestroGestureTarget, + MaestroLaunchArguments, + MaestroSelector, + MaestroSourceLocation, + MaestroSwipeGesture, +} from './program-ir.ts'; +import type { + MaestroObservation, + MaestroObservationCondition, + MaestroObservationEvidence, +} from './engine-types.ts'; + +export type MaestroRuntimeOperationContext = { + readonly appId?: string; + readonly env: Readonly>; + readonly generation: number; + readonly source?: MaestroSourceLocation; + readonly cachedObservation?: MaestroObservation; + readonly signal?: AbortSignal; + readonly authoredSwipe?: MaestroSwipeGesture; + readonly swipeTarget?: MaestroTargetResolution; + readonly gestureViewport?: Rect; +}; + +/** Evidence returned by the shared selector runtime for one observation generation. */ +export type MaestroTargetMatch = { + readonly generation: number; + readonly matched: boolean; + readonly visible: boolean; + readonly candidateCount: number; + readonly rect?: Rect; + readonly viewport?: Rect; + readonly ref?: string; + readonly dispatchSelector?: MaestroDispatchSelector; +}; + +export type MaestroDispatchSelector = { + readonly key: 'id' | 'label' | 'text'; + readonly value: string; +}; + +export type MaestroSelectorEvidence = MaestroObservationEvidence; + +export type MaestroTargetResolution = MaestroTargetMatch & { + readonly kind: 'selector'; + readonly selector: MaestroSelector; + readonly query: MaestroTargetQuery; + readonly rect: Rect; +}; + +export type MaestroTargetQuery = { + readonly selector: MaestroSelector; + readonly purpose: 'tap' | 'doubleTap' | 'longPress' | 'swipe'; + readonly timeoutMs: number; + readonly index?: number; + readonly childOf?: MaestroSelector; + readonly allowAtomicSelectorDispatch?: boolean; +}; + +export type MaestroInputTarget = { + readonly authored: MaestroGestureTarget; + readonly point?: Point; + readonly resolution?: MaestroTargetResolution; +}; + +export type MaestroSwipeOperation = { + /** The authored Maestro coordinate space and target mode, preserved for policy and diagnostics. */ + readonly authored: MaestroSwipeGesture; + /** The normalized contract consumed by the shared input runtime. */ + readonly gesture: MaestroSinglePointerGestureInput; + readonly target?: MaestroTargetResolution; + readonly viewport?: Rect; +}; + +type GestureFlingInput = Extract; +type GesturePresetPanInput = Extract; +type GesturePointPanInput = Extract; + +export type MaestroSinglePointerGestureInput = + | GestureFlingInput + | GesturePresetPanInput + | (Omit & { readonly pointerCount?: never }); + +export type MaestroRuntimeOperationResult = { + readonly observation?: MaestroObservation; + readonly outputEnv?: Record; + readonly artifactPaths?: readonly string[]; +}; + +export type MaestroRuntimeOperation = ( + input: TInput, + context: MaestroRuntimeOperationContext, +) => Promise; + +export type MaestroRuntimeOperations = { + readonly resolveTarget: ( + input: MaestroTargetQuery, + context: MaestroRuntimeOperationContext, + ) => Promise; + readonly observe: ( + input: { + readonly condition: MaestroObservationCondition; + readonly timeoutMs: number; + }, + context: MaestroRuntimeOperationContext, + ) => Promise; + readonly resolveGestureViewport: (context: MaestroRuntimeOperationContext) => Promise; + + readonly launchApp: MaestroRuntimeOperation<{ + readonly appId?: string; + readonly stopApp?: boolean; + readonly clearState?: boolean; + readonly arguments?: MaestroLaunchArguments; + readonly launchArguments?: MaestroLaunchArguments; + }>; + readonly stopApp: MaestroRuntimeOperation<{ readonly appId?: string }>; + readonly openLink: MaestroRuntimeOperation<{ readonly link: string }>; + + readonly tapOn: MaestroRuntimeOperation<{ + readonly target: MaestroInputTarget; + readonly repeat?: number; + readonly delay?: number; + readonly optional?: boolean; + readonly label?: string; + readonly index?: number; + readonly childOf?: MaestroSelector; + }>; + readonly doubleTapOn: MaestroRuntimeOperation<{ + readonly target: MaestroInputTarget; + readonly delay?: number; + }>; + readonly longPressOn: MaestroRuntimeOperation<{ readonly target: MaestroInputTarget }>; + readonly gesture: MaestroRuntimeOperation; + readonly inputText: MaestroRuntimeOperation<{ readonly text: string; readonly label?: string }>; + readonly eraseText: MaestroRuntimeOperation<{ readonly charactersToErase?: number }>; + readonly pasteText: MaestroRuntimeOperation<{ readonly text: string }>; + readonly scroll: MaestroRuntimeOperation<{ readonly direction: MaestroDirection }>; + readonly scrollUntilVisible: MaestroRuntimeOperation<{ + readonly selector: MaestroSelector; + readonly direction: MaestroDirection; + readonly timeoutMs: number; + }>; + readonly pressKey: MaestroRuntimeOperation<{ + readonly key: 'back' | 'enter' | 'return' | 'home'; + }>; + readonly back: MaestroRuntimeOperation>; + readonly hideKeyboard: MaestroRuntimeOperation>; + readonly waitForAnimationToEnd: MaestroRuntimeOperation<{ readonly timeoutMs?: number }>; + + readonly takeScreenshot: MaestroRuntimeOperation<{ readonly path: string }>; + readonly runScript: MaestroRuntimeOperation<{ + readonly file: string; + readonly env?: Record; + }>; +}; diff --git a/src/compat/maestro/runtime-port.ts b/src/compat/maestro/runtime-port.ts new file mode 100644 index 000000000..3bd4ceaf3 --- /dev/null +++ b/src/compat/maestro/runtime-port.ts @@ -0,0 +1,29 @@ +import type { + MaestroRuntimePort, + MaestroRuntimeRequest, + MaestroRuntimeResult, +} from './engine-types.ts'; +import { executeMaestroRuntimeCommand } from './runtime-port-commands.ts'; +import { observeMaestroCondition } from './runtime-port-observation.ts'; +import type { MaestroRuntimeOperations } from './runtime-port-types.ts'; + +export type { MaestroRuntimePort } from './engine-types.ts'; +export type { + MaestroInputTarget, + MaestroRuntimeOperationContext, + MaestroRuntimeOperationResult, + MaestroRuntimeOperations, + MaestroSelectorEvidence, + MaestroSwipeOperation, + MaestroTargetMatch, + MaestroTargetQuery, + MaestroTargetResolution, +} from './runtime-port-types.ts'; + +export function createMaestroRuntimePort(operations: MaestroRuntimeOperations): MaestroRuntimePort { + return { + execute: async (request: MaestroRuntimeRequest): Promise => + await executeMaestroRuntimeCommand(request, operations), + observe: async (request) => await observeMaestroCondition(request, operations), + }; +} diff --git a/src/compat/maestro/runtime-support.ts b/src/compat/maestro/runtime-support.ts deleted file mode 100644 index 0a13b99ba..000000000 --- a/src/compat/maestro/runtime-support.ts +++ /dev/null @@ -1,119 +0,0 @@ -import type { - DaemonInvokeFn, - DaemonRequest, - DaemonResponse, - DaemonResponseData, -} from '../../daemon/types.ts'; -import type { DaemonFailureResponse } from '../../daemon/handlers/response.ts'; -import type { ReplayActionBlockInvoker } from '../../replay/control-flow-runtime.ts'; -import type { ReplayVarScope } from '../../replay/vars.ts'; -import type { Point, SnapshotState } from '../../kernel/snapshot.ts'; - -export type ReplayBaseRequest = Omit; - -export type MaestroReplayInvoker = ReplayActionBlockInvoker; - -export type MaestroRuntimeInvoke = DaemonInvokeFn; - -export type FailedDaemonResponse = DaemonFailureResponse; - -const maestroVisibleContextCache = new WeakMap(); -const maestroRecoverableInteractionCache = new WeakMap< - ReplayVarScope, - MaestroRecoverableInteraction ->(); - -export type MaestroRecoverableInteraction = - | ({ - kind: 'tap'; - } & MaestroRecoverableTap) - | ({ - kind: 'swipe'; - } & MaestroRecoverableSwipe); - -export type MaestroRecoverableTap = { - selector: string; - point: Point; - options?: { - childOf?: string; - index?: number; - }; -}; - -export type MaestroRecoverableSwipe = { - command: 'gesture' | 'swipe'; - input: Record; -}; - -export function errorResponse( - code: string, - message: string, - details?: Record, -): FailedDaemonResponse { - return { - ok: false, - error: { code, message, ...(details ? { details } : {}) }, - }; -} - -export async function captureMaestroSnapshot(params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; - scope?: ReplayVarScope; - raw?: boolean; -}): Promise { - const useRawSnapshot = - params.raw === true || process.env.AGENT_DEVICE_MAESTRO_RAW_SNAPSHOTS === '1'; - const response = await params.invoke({ - ...params.baseReq, - command: 'snapshot', - positionals: [], - flags: { - ...params.baseReq.flags, - noRecord: true, - ...(useRawSnapshot ? { snapshotRaw: true } : {}), - }, - }); - return response; -} - -export function readSnapshotState(data: DaemonResponseData | undefined): SnapshotState | undefined { - return Array.isArray(data?.nodes) ? (data as SnapshotState) : undefined; -} - -export function rememberMaestroVisibleContext( - scope: ReplayVarScope | undefined, - selector: string, -): void { - if (scope) maestroVisibleContextCache.set(scope, { selector }); -} - -export function readMaestroVisibleContext( - scope: ReplayVarScope | undefined, -): { selector: string } | undefined { - return scope ? maestroVisibleContextCache.get(scope) : undefined; -} - -export function clearMaestroVisibleContext(scope: ReplayVarScope | undefined): void { - if (scope) maestroVisibleContextCache.delete(scope); -} - -export function rememberMaestroRecoverableInteraction( - scope: ReplayVarScope | undefined, - interaction: MaestroRecoverableInteraction, -): void { - if (scope) maestroRecoverableInteractionCache.set(scope, interaction); -} - -export function consumeMaestroRecoverableInteraction( - scope: ReplayVarScope | undefined, -): MaestroRecoverableInteraction | undefined { - if (!scope) return undefined; - const interaction = maestroRecoverableInteractionCache.get(scope); - maestroRecoverableInteractionCache.delete(scope); - return interaction; -} - -export function clearMaestroRecoverableInteraction(scope: ReplayVarScope | undefined): void { - if (scope) maestroRecoverableInteractionCache.delete(scope); -} diff --git a/src/compat/maestro/runtime-target-matching.ts b/src/compat/maestro/runtime-target-matching.ts new file mode 100644 index 000000000..4087ad8ce --- /dev/null +++ b/src/compat/maestro/runtime-target-matching.ts @@ -0,0 +1,15 @@ +import type { SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import type { MaestroSelector } from './program-ir.ts'; +import { + matchesMaestroTypedSelector, + type MaestroSelectorMatchOptions, +} from './runtime-target-policy.ts'; + +/** Resolve a source-preserving Maestro selector without stringifying it. */ +export function findMaestroTypedSelectorMatches( + snapshot: SnapshotState, + selector: MaestroSelector, + options: MaestroSelectorMatchOptions = {}, +): SnapshotNode[] { + return snapshot.nodes.filter((node) => matchesMaestroTypedSelector(node, selector, options)); +} diff --git a/src/compat/maestro/runtime-target-policy.ts b/src/compat/maestro/runtime-target-policy.ts new file mode 100644 index 000000000..84e163b26 --- /dev/null +++ b/src/compat/maestro/runtime-target-policy.ts @@ -0,0 +1,206 @@ +import type { MaestroSelector } from './program-ir.ts'; +import type { SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import { evaluateIsPredicate } from '../../selectors/predicates.ts'; +import { normalizeText } from '../../selectors/find.ts'; +import { extractNodeText } from '../../snapshot/snapshot-processing.ts'; +import { + detectReactNativeOverlay, + readReactNativeOverlayActionNodes, +} from '../../core/react-native-overlay.ts'; + +export type MaestroPlatform = 'ios' | 'android'; + +export type MaestroSelectorMatchOptions = { + allowLeadingCompositeLabelMatch?: boolean; +}; + +export type MaestroResolvedSnapshotMatch = { + node: SnapshotNode; + rect: NonNullable; + inheritedRect: boolean; +}; + +export type MaestroPreferredContext = { + node: SnapshotNode; + rect: NonNullable; +}; + +export type ReactNativeOverlayFilterResult = { + matches: SnapshotNode[]; + blockedByReactNativeOverlay: boolean; +}; + +/** + * Match the source-preserving selector IR directly. In particular, this does + * not lower the selector to the legacy `key=value` expression grammar. + * + * Maestro's compatibility parser treats id as the strongest primary field, + * then text, then label. Text is the visible-text form used by scalar + * selectors, so it checks label, readable node text, and identifier values. + * Enabled and selected are independent state constraints. + */ +export function matchesMaestroTypedSelector( + node: SnapshotNode, + selector: MaestroSelector, + options: MaestroSelectorMatchOptions = {}, +): boolean { + const primary = readTypedPrimarySelector(selector); + if (primary) { + const matched = + primary.key === 'text' + ? matchesMaestroVisibleText(node, primary.value, options) + : textEqualsOrRegex(readMaestroTextTermValue(node, primary.key), primary.value, options); + if (!matched) return false; + } else if (selector.enabled === undefined && selector.selected === undefined) { + return false; + } + + if (selector.enabled !== undefined && Boolean(node.enabled !== false) !== selector.enabled) { + return false; + } + if (selector.selected !== undefined && Boolean(node.selected === true) !== selector.selected) { + return false; + } + return true; +} + +/** Read the visible-text fallback query represented by the typed selector. */ +export function extractMaestroVisibleTextQueryFromSelector( + selector: MaestroSelector, +): string | null { + if (selector.id !== undefined) return null; + const query = selector.text ?? selector.label; + return query ? query : null; +} + +export function filterVisibleMaestroMatches(params: { + nodes: SnapshotState['nodes']; + matches: SnapshotNode[]; + platform: MaestroPlatform; +}): ReactNativeOverlayFilterResult { + const visibleMatches = params.matches.filter( + (node) => + evaluateIsPredicate({ + predicate: 'visible', + node, + nodes: params.nodes, + platform: params.platform, + }).pass, + ); + const overlayFilter = filterReactNativeOverlayBlockedMatches( + params.nodes, + visibleMatches, + params.platform, + ); + return { + matches: overlayFilter.matches, + blockedByReactNativeOverlay: overlayFilter.blockedByReactNativeOverlay, + }; +} + +function filterReactNativeOverlayBlockedMatches( + nodes: SnapshotState['nodes'], + matches: SnapshotNode[], + platform: MaestroPlatform, +): ReactNativeOverlayFilterResult { + const overlay = detectReactNativeOverlay(nodes); + if (!overlay.detected || !overlay.redBox) { + return { matches, blockedByReactNativeOverlay: false }; + } + + const overlayControls = readReactNativeOverlayActionNodes(overlay); + const visibleOverlayControls = overlayControls.filter( + (node) => + evaluateIsPredicate({ + predicate: 'visible', + node, + nodes, + platform, + }).pass, + ); + if (visibleOverlayControls.length === 0) { + if (overlayControls.length === 0) { + return { matches: [], blockedByReactNativeOverlay: true }; + } + return { matches, blockedByReactNativeOverlay: false }; + } + + const overlayNodeIndexes = new Set(visibleOverlayControls.map((node) => node.index)); + const overlayMatches = matches.filter((node) => overlayNodeIndexes.has(node.index)); + return { + matches: overlayMatches, + blockedByReactNativeOverlay: matches.length > 0 && overlayMatches.length === 0, + }; +} + +export function maestroVisibleTextMatchRank(node: SnapshotNode, query: string): number { + const values = [node.label, extractNodeText(node), node.identifier, node.value].filter( + (value): value is string => Boolean(value), + ); + if (values.some((value) => value === query)) return 0; + if (values.some((value) => normalizeText(value) === normalizeText(query))) return 1; + if (values.some((value) => textEqualsOrRegex(value, query))) return 2; + return 3; +} + +function textEqualsOrRegex( + value: string | undefined, + query: string, + options: MaestroSelectorMatchOptions = {}, +): boolean { + const text = value ?? ''; + const normalizedText = normalizeText(text); + const normalizedQuery = normalizeText(query); + if (normalizedText === normalizedQuery) return true; + if ( + options.allowLeadingCompositeLabelMatch !== false && + isLeadingCompositeLabelMatch(normalizedText, normalizedQuery) + ) { + return true; + } + if (!looksLikeMaestroRegex(query)) return false; + try { + return new RegExp(query).test(text); + } catch { + return false; + } +} + +function readTypedPrimarySelector( + selector: MaestroSelector, +): { key: 'id' | 'text' | 'label'; value: string } | null { + if (selector.id !== undefined) return { key: 'id', value: selector.id }; + if (selector.text !== undefined) return { key: 'text', value: selector.text }; + if (selector.label !== undefined) return { key: 'label', value: selector.label }; + return null; +} + +function matchesMaestroVisibleText( + node: SnapshotNode, + query: string, + options: MaestroSelectorMatchOptions, +): boolean { + return [node.label, extractNodeText(node), node.identifier] + .filter((value): value is string => Boolean(value)) + .some((value) => textEqualsOrRegex(value, query, options)); +} + +function readMaestroTextTermValue( + node: SnapshotNode, + key: 'id' | 'label' | 'text', +): string | undefined { + if (key === 'id') return node.identifier; + if (key === 'label') return node.label; + return extractNodeText(node); +} + +function isLeadingCompositeLabelMatch(normalizedText: string, normalizedQuery: string): boolean { + if (!normalizedText || !normalizedQuery) return false; + if (!normalizedText.startsWith(normalizedQuery)) return false; + const next = normalizedText.at(normalizedQuery.length); + return next === ',' || next === ':' || next === ';'; +} + +function looksLikeMaestroRegex(query: string): boolean { + return /(?:\.\*|\.\+|\[[^\]]+\]|\([^)]*\)|\||\^|\$|\\[dDsSwWbB])/.test(query); +} diff --git a/src/compat/maestro/runtime-target-ranking-duplicates.ts b/src/compat/maestro/runtime-target-ranking-duplicates.ts new file mode 100644 index 000000000..971acfd47 --- /dev/null +++ b/src/compat/maestro/runtime-target-ranking-duplicates.ts @@ -0,0 +1,331 @@ +import type { Rect, SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import { + buildSnapshotNodeByIndex, + findSnapshotAncestor, + isDescendantOfSnapshotNode, + normalizeType, +} from '../../snapshot/snapshot-processing.ts'; +import { + maestroVisibleTextMatchRank, + type MaestroPreferredContext, + type MaestroResolvedSnapshotMatch, +} from './runtime-target-policy.ts'; +import { + rectArea, + rectContains, + rectOverlapRatio, + type SnapshotNodeByIndex, +} from './runtime-target-ranking-geometry.ts'; + +type MaestroMatchWithScreenContainer = { + candidate: MaestroResolvedSnapshotMatch; + container: SnapshotNode & { rect: Rect }; +}; + +export function selectPreferredMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + visibleTextQuery: string | null, + promoteTapTarget: boolean, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch | null { + if (!promoteTapTarget || !visibleTextQuery) { + return selectBestMaestroSnapshotMatch(nodes, candidates, visibleTextQuery, preferredContext); + } + return ( + selectLocalizedMaestroVisibleTextMatch(nodes, candidates, visibleTextQuery, preferredContext) ?? + selectBestMaestroSnapshotMatch(nodes, candidates, visibleTextQuery, preferredContext) + ); +} + +function selectBestMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + visibleTextQuery: string | null, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch | null { + const foregroundCandidates = preferForegroundContainerDuplicateMatches( + nodes, + candidates, + visibleTextQuery, + preferredContext, + ); + return ( + foregroundCandidates.sort((left, right) => + compareMaestroSnapshotMatches(left, right, visibleTextQuery), + )[0] ?? null + ); +} + +function selectLocalizedMaestroVisibleTextMatch( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + query: string, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch | null { + const exactMatches = candidates.filter( + (candidate) => maestroVisibleTextMatchRank(candidate.node, query) === 0, + ); + if (exactMatches.length >= 2) { + const localizedExact = selectLocalizedMaestroVisibleTextMatchFromCandidates( + nodes, + exactMatches, + query, + preferredContext, + ); + if (localizedExact) return localizedExact; + } + + const normalizedMatches = candidates.filter( + (candidate) => maestroVisibleTextMatchRank(candidate.node, query) === 1, + ); + if (exactMatches.length > 0 || normalizedMatches.length < 2) return null; + + return selectLocalizedMaestroVisibleTextMatchFromCandidates( + nodes, + normalizedMatches, + query, + preferredContext, + ); +} + +function selectLocalizedMaestroVisibleTextMatchFromCandidates( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + query: string, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch | null { + const nodeByIndex = buildSnapshotNodeByIndex(nodes); + const localized = candidates.filter( + (candidate) => + isLocalizedMaestroVisibleTextCandidate(candidate) && + candidates.some((container) => + isMaestroVisibleTextContainerForCandidate(nodes, container, candidate, nodeByIndex), + ), + ); + + return selectBestMaestroSnapshotMatch(nodes, localized, query, preferredContext); +} + +// fallow-ignore-next-line complexity +function preferForegroundContainerDuplicateMatches( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + visibleTextQuery: string | null, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch[] { + if (!visibleTextQuery || candidates.length < 2) return candidates; + const exact = candidates.filter( + (candidate) => maestroVisibleTextMatchRank(candidate.node, visibleTextQuery) === 0, + ); + if (exact.length < 2) return candidates; + + const nodeByIndex = buildSnapshotNodeByIndex(nodes); + const withContainers = exact + .map((candidate) => ({ + candidate, + container: findMaestroScreenContainer(nodes, candidate.node, nodeByIndex), + })) + .filter((entry): entry is MaestroMatchWithScreenContainer => Boolean(entry.container)); + if (withContainers.length < 2 || withContainers.length !== exact.length) return candidates; + + const overlapping = withContainers.filter((entry) => + hasOverlappingScreenContainer(entry, withContainers), + ); + if (overlapping.length < 2) return candidates; + + const foregroundByArea = selectLargestOverlappingScreenContainerMatches(overlapping); + if (foregroundByArea.length > 0) return foregroundByArea; + + const foregroundByContext = selectContextualOverlappingScreenContainerMatches( + nodes, + overlapping, + preferredContext, + nodeByIndex, + ); + if (foregroundByContext.length > 0) return foregroundByContext; + + // UIAutomator reports foreground transparent-stack screens later in the + // hierarchy while preserving both screens. Prefer the later overlapping + // screen only for exact duplicate text, so ordinary duplicate rows keep + // Maestro's read-order behavior. + const foregroundContainerIndex = Math.max(...overlapping.map((entry) => entry.container.index)); + const foreground = overlapping + .filter((entry) => entry.container.index === foregroundContainerIndex) + .map((entry) => entry.candidate); + return foreground.length > 0 ? foreground : candidates; +} + +function selectContextualOverlappingScreenContainerMatches( + nodes: SnapshotState['nodes'], + entries: MaestroMatchWithScreenContainer[], + context: MaestroPreferredContext | undefined, + nodeByIndex: SnapshotNodeByIndex, +): MaestroResolvedSnapshotMatch[] { + if (!context) return []; + const rawContextContainer = findMaestroScreenContainer(nodes, context.node, nodeByIndex); + const contextContainer = + rawContextContainer && rectContains(rawContextContainer.rect, context.rect) + ? rawContextContainer + : null; + const scored = entries.map((entry) => ({ + entry, + score: scoreScreenContainerAgainstContext(entry.container, context, contextContainer), + })); + const bestScore = Math.min(...scored.map((entry) => entry.score)); + if (!Number.isFinite(bestScore)) return []; + return scored.filter((entry) => entry.score === bestScore).map((entry) => entry.entry.candidate); +} + +function scoreScreenContainerAgainstContext( + container: SnapshotNode & { rect: Rect }, + context: MaestroPreferredContext, + contextContainer: (SnapshotNode & { rect: Rect }) | null, +): number { + if (contextContainer) { + if (container.index === contextContainer.index) return 0; + if (rectOverlapRatio(container.rect, contextContainer.rect) < 0.6) + return Number.POSITIVE_INFINITY; + return Math.abs(container.index - contextContainer.index); + } + + if (rectOverlapRatio(container.rect, context.rect) >= 0.6) return 0; + const orderDistance = container.index - context.node.index; + return orderDistance >= 0 ? orderDistance : 100_000 + Math.abs(orderDistance); +} + +function selectLargestOverlappingScreenContainerMatches( + entries: MaestroMatchWithScreenContainer[], +): MaestroResolvedSnapshotMatch[] { + const areas = entries.map((entry) => rectArea(entry.container.rect)); + const largestArea = Math.max(...areas); + const smallestArea = Math.min(...areas); + if (smallestArea <= 0 || largestArea < smallestArea * 1.2) return []; + return entries + .filter((entry) => rectArea(entry.container.rect) === largestArea) + .map((entry) => entry.candidate); +} + +function hasOverlappingScreenContainer( + entry: MaestroMatchWithScreenContainer, + candidates: MaestroMatchWithScreenContainer[], +): boolean { + return candidates.some( + (other) => + other !== entry && + entry.container.index !== other.container.index && + rectOverlapRatio(entry.container.rect, other.container.rect) >= 0.6, + ); +} + +function findMaestroScreenContainer( + nodes: SnapshotState['nodes'], + node: SnapshotNode, + nodeByIndex: SnapshotNodeByIndex, +): (SnapshotNode & { rect: Rect }) | null { + return findSnapshotAncestor(nodes, node, nodeByIndex, (ancestor) => { + if (!ancestor.rect) return null; + if (!isMaestroScreenContainerType(ancestor)) return null; + if (ancestor.rect.width < 240 || ancestor.rect.height < 320) return null; + return ancestor as SnapshotNode & { rect: Rect }; + }); +} + +function isMaestroScreenContainerType(node: SnapshotNode): boolean { + const type = normalizeType(node.type ?? ''); + return type === 'scrollview' || type === 'scroll-area' || type === 'list'; +} + +function isLocalizedMaestroVisibleTextCandidate(match: MaestroResolvedSnapshotMatch): boolean { + return ( + match.rect.width >= 16 && + match.rect.width <= 260 && + match.rect.height >= 24 && + match.rect.height <= 80 + ); +} + +function isMaestroVisibleTextContainerForCandidate( + nodes: SnapshotState['nodes'], + container: MaestroResolvedSnapshotMatch, + candidate: MaestroResolvedSnapshotMatch, + nodeByIndex: SnapshotNodeByIndex, +): boolean { + if (container.node.index === candidate.node.index) return false; + if (!rectContains(container.rect, candidate.rect)) return false; + if (rectArea(container.rect) < rectArea(candidate.rect) * 2) return false; + return isDescendantOfSnapshotNode(nodes, candidate.node, container.node, nodeByIndex); +} + +function compareMaestroSnapshotMatches( + left: MaestroResolvedSnapshotMatch, + right: MaestroResolvedSnapshotMatch, + visibleTextQuery: string | null, +): number { + const priorityRank = compareMaestroSnapshotMatchPriority(left, right, visibleTextQuery); + if (priorityRank !== 0) return priorityRank; + + if (!sameRoundedRect(left.rect, right.rect)) { + return left.node.index - right.node.index; + } + + const depthRank = (right.node.depth ?? 0) - (left.node.depth ?? 0); + if (depthRank !== 0) return depthRank; + + // Android transparent stacks can expose both the background screen and the + // foreground screen at the same coordinates. UIAutomator reports the + // foreground duplicate later in the snapshot, which matches Maestro's + // practical tap target for overlapping duplicates. + return right.node.index - left.node.index; +} + +function compareMaestroSnapshotMatchPriority( + left: MaestroResolvedSnapshotMatch, + right: MaestroResolvedSnapshotMatch, + visibleTextQuery: string | null, +): number { + if (visibleTextQuery) { + const textRank = + maestroVisibleTextMatchRank(left.node, visibleTextQuery) - + maestroVisibleTextMatchRank(right.node, visibleTextQuery); + if (textRank !== 0) return textRank; + } + + const typeRank = maestroTapTargetTypeRank(left.node) - maestroTapTargetTypeRank(right.node); + if (typeRank !== 0) return typeRank; + + const rectSourceRank = Number(left.inheritedRect) - Number(right.inheritedRect); + if (rectSourceRank !== 0) return rectSourceRank; + + const areaRank = + visibleTextQuery && maestroTapTargetTypeRank(left.node) === maestroTapTargetTypeRank(right.node) + ? rectArea(right.rect) - rectArea(left.rect) + : rectArea(left.rect) - rectArea(right.rect); + if (areaRank !== 0) return areaRank; + return 0; +} + +function sameRoundedRect(left: Rect, right: Rect): boolean { + return ( + Math.round(left.x) === Math.round(right.x) && + Math.round(left.y) === Math.round(right.y) && + Math.round(left.width) === Math.round(right.width) && + Math.round(left.height) === Math.round(right.height) + ); +} + +function maestroTapTargetTypeRank(node: SnapshotNode): number { + return MAESTRO_TAP_TARGET_TYPE_RANK.get(normalizeType(node.type ?? '')) ?? 3; +} + +const MAESTRO_TAP_TARGET_TYPE_RANK = new Map([ + ['button', 0], + ['link', 0], + ['textfield', 0], + ['textview', 0], + ['searchfield', 0], + ['switch', 0], + ['slider', 0], + ['cell', 1], + ['statictext', 2], +]); diff --git a/src/compat/maestro/runtime-target-ranking-geometry.ts b/src/compat/maestro/runtime-target-ranking-geometry.ts new file mode 100644 index 000000000..22f2919bc --- /dev/null +++ b/src/compat/maestro/runtime-target-ranking-geometry.ts @@ -0,0 +1,158 @@ +import type { TouchReferenceFrame } from '../../daemon/touch-reference-frame.ts'; +import type { Rect, SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import { + buildSnapshotNodeByIndex, + findSnapshotAncestor, + normalizeType, +} from '../../snapshot/snapshot-processing.ts'; +import type { MaestroResolvedSnapshotMatch } from './runtime-target-policy.ts'; + +const RECT_CONTAINS_EPSILON = 1; + +export type SnapshotNodeByIndex = ReturnType; + +export function resolveMaestroNodeRect( + nodes: SnapshotState['nodes'], + node: SnapshotNode, + nodeByIndex: SnapshotNodeByIndex, +): { rect: Rect; inherited: boolean } | null { + if (node.rect && node.rect.width > 0 && node.rect.height > 0) { + return { rect: node.rect, inherited: false }; + } + if (node.rect) return null; + const rect = resolveRectlessNodeAncestorRect(nodes, node, nodeByIndex); + return rect ? { rect, inherited: true } : null; +} + +export function preferOnScreenMatches( + matches: MaestroResolvedSnapshotMatch[], + frame: TouchReferenceFrame | undefined, + requireOnScreen: boolean, +): MaestroResolvedSnapshotMatch[] { + const onScreen = matches.filter((match) => isRectOnScreen(match.rect, frame)); + if (requireOnScreen) return onScreen; + return onScreen.length > 0 ? onScreen : matches; +} + +export function promoteMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + match: MaestroResolvedSnapshotMatch | null, + nodeByIndex: SnapshotNodeByIndex, + promoteTapTarget: boolean, + frame: TouchReferenceFrame | undefined, +): { node: SnapshotNode; rect: Rect } | null { + if (!match) return null; + if (!promoteTapTarget) { + return { node: match.node, rect: match.rect }; + } + const ancestor = findMaestroTapAncestor(nodes, match, nodeByIndex, frame); + return ancestor ?? { node: match.node, rect: match.rect }; +} + +export function rectArea(rect: Rect): number { + return rect.width * rect.height; +} + +export function rectContains(container: Rect, child: Rect): boolean { + return ( + child.x >= container.x - RECT_CONTAINS_EPSILON && + child.y >= container.y - RECT_CONTAINS_EPSILON && + child.x + child.width <= container.x + container.width + RECT_CONTAINS_EPSILON && + child.y + child.height <= container.y + container.height + RECT_CONTAINS_EPSILON + ); +} + +export function rectOverlapRatio(a: Rect, b: Rect): number { + const left = Math.max(a.x, b.x); + const right = Math.min(a.x + a.width, b.x + b.width); + const top = Math.max(a.y, b.y); + const bottom = Math.min(a.y + a.height, b.y + b.height); + const overlapArea = Math.max(0, right - left) * Math.max(0, bottom - top); + return overlapArea / Math.max(1, Math.min(rectArea(a), rectArea(b))); +} + +export function verticalOverlapRatio(a: Rect, b: Rect): number { + const top = Math.max(a.y, b.y); + const bottom = Math.min(a.y + a.height, b.y + b.height); + const overlap = Math.max(0, bottom - top); + return overlap / Math.max(1, Math.min(a.height, b.height)); +} + +function resolveRectlessNodeAncestorRect( + nodes: SnapshotState['nodes'], + node: SnapshotNode, + nodeByIndex: SnapshotNodeByIndex, +): Rect | null { + let current: SnapshotNode | undefined = node; + while (typeof current.parentIndex === 'number') { + current = nodeByIndex.get(current.parentIndex) ?? nodes[current.parentIndex]; + if (!current) return null; + if (!current.rect) continue; + return current.rect.width > 0 && current.rect.height > 0 ? current.rect : null; + } + return null; +} + +function isRectOnScreen(rect: Rect, frame: TouchReferenceFrame | undefined): boolean { + const maxX = frame?.referenceWidth ?? Number.POSITIVE_INFINITY; + const maxY = frame?.referenceHeight ?? Number.POSITIVE_INFINITY; + return rect.x < maxX && rect.y < maxY && rect.x + rect.width > 0 && rect.y + rect.height > 0; +} + +function findMaestroTapAncestor( + nodes: SnapshotState['nodes'], + match: MaestroResolvedSnapshotMatch, + nodeByIndex: SnapshotNodeByIndex, + frame: TouchReferenceFrame | undefined, +): { node: SnapshotNode; rect: Rect } | null { + if (isActionableMaestroTapTarget(match.node)) return null; + return findSnapshotAncestor(nodes, match.node, nodeByIndex, (ancestor) => { + if (!isActionableMaestroTapTarget(ancestor)) return null; + const ancestorRect = resolveMaestroNodeRect(nodes, ancestor, nodeByIndex); + if (!ancestorRect || !isUsefulMaestroTapAncestorRect(match.rect, ancestorRect.rect, frame)) { + return null; + } + return { node: ancestor, rect: ancestorRect.rect }; + }); +} + +function isActionableMaestroTapTarget(node: SnapshotNode): boolean { + const type = normalizeType(node.type ?? ''); + return ( + node.hittable === true || + type === 'button' || + type === 'link' || + type === 'cell' || + type === 'textfield' || + type === 'searchfield' || + type === 'switch' || + type === 'slider' + ); +} + +function isUsefulMaestroTapAncestorRect( + matchRect: Rect, + ancestorRect: Rect, + frame: TouchReferenceFrame | undefined, +): boolean { + if (!rectContains(ancestorRect, matchRect)) return false; + if (wouldPromoteTabSlotToWholeStrip(matchRect, ancestorRect)) return false; + const ancestorArea = rectArea(ancestorRect); + const matchArea = rectArea(matchRect); + // Keep promotion close to the matched label/id instead of jumping to a broad container. + if (matchArea > 0 && ancestorArea > matchArea * 30) return false; + if (frame) { + const frameArea = frame.referenceWidth * frame.referenceHeight; + // Full-screen ancestors are usually layout containers, not meaningful tap targets. + if (frameArea > 0 && ancestorArea > frameArea * 0.5) return false; + } + return true; +} + +function wouldPromoteTabSlotToWholeStrip(matchRect: Rect, ancestorRect: Rect): boolean { + if (ancestorRect.height < 32 || ancestorRect.height > 80) return false; + if (matchRect.height < ancestorRect.height * 0.75) return false; + if (verticalOverlapRatio(matchRect, ancestorRect) < 0.75) return false; + if (ancestorRect.width < 240) return false; + return ancestorRect.width >= matchRect.width * 3; +} diff --git a/src/compat/maestro/runtime-target-ranking-tabs.ts b/src/compat/maestro/runtime-target-ranking-tabs.ts new file mode 100644 index 000000000..4565f11ea --- /dev/null +++ b/src/compat/maestro/runtime-target-ranking-tabs.ts @@ -0,0 +1,176 @@ +import type { Rect, SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import { + buildSnapshotNodeByIndex, + isDescendantOfSnapshotNode, + normalizeType, +} from '../../snapshot/snapshot-processing.ts'; +import { + maestroVisibleTextMatchRank, + type MaestroResolvedSnapshotMatch, +} from './runtime-target-policy.ts'; +import { + rectContains, + verticalOverlapRatio, + type SnapshotNodeByIndex, +} from './runtime-target-ranking-geometry.ts'; + +export function inferMaestroMissingTabSlotMatch( + nodes: SnapshotState['nodes'], + match: MaestroResolvedSnapshotMatch, + query: string, +): MaestroResolvedSnapshotMatch | null { + if (!isMaestroTabStripContainerMatch(match, query)) return null; + const children = collectMaestroTabStripChildCandidates( + nodes, + match, + query, + buildSnapshotNodeByIndex(nodes), + ); + if (children.length === 0) return null; + const medianChildWidth = median(children.map((child) => child.rect.width)); + const allGaps = resolveHorizontalGaps( + match.rect, + children.map((child) => child.rect), + ); + const gap = selectMaestroMissingSlotGap(match, query, allGaps, medianChildWidth); + if (!gap) return null; + return matchWithRect(match, gap); +} + +function collectMaestroTabStripChildCandidates( + nodes: SnapshotState['nodes'], + match: MaestroResolvedSnapshotMatch, + query: string, + nodeByIndex: SnapshotNodeByIndex, +): Array { + return nodes + .filter((node): node is SnapshotNode & { rect: Rect } => { + return ( + node.index !== match.node.index && + Boolean(node.rect) && + isDescendantOfSnapshotNode(nodes, node, match.node, nodeByIndex) && + isMaestroTabStripChildCandidate(node as SnapshotNode & { rect: Rect }, match.rect, query) + ); + }) + .sort((left, right) => left.rect.x - right.rect.x); +} + +function selectMaestroMissingSlotGap( + match: MaestroResolvedSnapshotMatch, + query: string, + gaps: Array<{ x: number; width: number }>, + medianChildWidth: number, +): { x: number; width: number } | null { + const plausibleGaps = gaps.filter((gap) => + isPlausibleMissingTabSlot(gap.width, medianChildWidth), + ); + const leadingTextSlot = inferMaestroLeadingTextSlotGap(match, query, gaps); + const hasPlausibleLeadingGap = plausibleGaps.some((gap) => isLeadingGap(match.rect, gap)); + if (leadingTextSlot && !hasPlausibleLeadingGap) return leadingTextSlot; + if (plausibleGaps.length === 1) return plausibleGaps[0] ?? null; + return leadingTextSlot; +} + +function inferMaestroLeadingTextSlotGap( + match: MaestroResolvedSnapshotMatch, + query: string, + gaps: Array<{ x: number; width: number }>, +): { x: number; width: number } | null { + const leadingGap = gaps.find((gap) => Math.abs(gap.x - match.rect.x) < 1); + const estimatedLabelWidth = Math.max(48, Math.min(220, query.length * 8 + 24)); + if (!isLeadingTextSlotCandidate(match, query, leadingGap, estimatedLabelWidth)) return null; + return { + x: match.rect.x, + width: Math.min(estimatedLabelWidth, leadingGap.width), + }; +} + +function isLeadingTextSlotCandidate( + match: MaestroResolvedSnapshotMatch, + query: string, + gap: { x: number; width: number } | undefined, + estimatedLabelWidth: number, +): gap is { x: number; width: number } { + if (!gap) return false; + return ( + normalizeType(match.node.type ?? '') === 'scrollview' && + maestroVisibleTextMatchRank(match.node, query) <= 1 && + match.rect.width >= 240 && + match.rect.height >= 32 && + match.rect.height <= 80 && + gap.width <= match.rect.width * 0.55 && + gap.width >= estimatedLabelWidth * 0.6 + ); +} + +function isLeadingGap(rect: Rect, gap: { x: number; width: number }): boolean { + return Math.abs(gap.x - rect.x) < 1; +} + +function matchWithRect( + match: MaestroResolvedSnapshotMatch, + gap: { x: number; width: number }, +): MaestroResolvedSnapshotMatch { + return { + ...match, + rect: { + x: gap.x, + y: match.rect.y, + width: gap.width, + height: match.rect.height, + }, + }; +} + +function isMaestroTabStripContainerMatch( + match: MaestroResolvedSnapshotMatch, + query: string, +): boolean { + const type = normalizeType(match.node.type ?? ''); + if (type !== 'cell' && type !== 'other' && type !== 'scrollview' && type !== 'scroll-area') { + return false; + } + if (match.rect.width < 120 || match.rect.height < 32 || match.rect.height > 80) return false; + return maestroVisibleTextMatchRank(match.node, query) <= 1; +} + +function isMaestroTabStripChildCandidate( + node: SnapshotNode & { rect: Rect }, + container: Rect, + query: string, +): boolean { + const type = normalizeType(node.type ?? ''); + if (type !== 'button' && type !== 'cell' && type !== 'other') return false; + if (maestroVisibleTextMatchRank(node, query) <= 1) return false; + if (node.rect.width < 16 || node.rect.height < 16) return false; + if (!rectContains(container, node.rect)) return false; + return verticalOverlapRatio(container, node.rect) >= 0.5; +} + +function resolveHorizontalGaps( + container: Rect, + occupied: Rect[], +): Array<{ x: number; width: number }> { + const gaps: Array<{ x: number; width: number }> = []; + let cursor = container.x; + const containerRight = container.x + container.width; + for (const rect of occupied) { + const start = Math.max(container.x, rect.x); + const end = Math.min(containerRight, rect.x + rect.width); + if (start > cursor) gaps.push({ x: cursor, width: start - cursor }); + cursor = Math.max(cursor, end); + } + if (containerRight > cursor) gaps.push({ x: cursor, width: containerRight - cursor }); + return gaps; +} + +function isPlausibleMissingTabSlot(gapWidth: number, medianChildWidth: number): boolean { + if (gapWidth < 24 || medianChildWidth < 24) return false; + return gapWidth >= medianChildWidth * 0.4 && gapWidth <= medianChildWidth * 1.6; +} + +function median(values: number[]): number { + const sorted = [...values].sort((left, right) => left - right); + const middle = Math.floor(sorted.length / 2); + return sorted[middle] ?? 0; +} diff --git a/src/compat/maestro/runtime-target-ranking.ts b/src/compat/maestro/runtime-target-ranking.ts new file mode 100644 index 000000000..ac45a248a --- /dev/null +++ b/src/compat/maestro/runtime-target-ranking.ts @@ -0,0 +1,101 @@ +import type { TouchReferenceFrame } from '../../daemon/touch-reference-frame.ts'; +import type { Rect, SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; +import { buildSnapshotNodeByIndex } from '../../snapshot/snapshot-processing.ts'; +import { + promoteMaestroSnapshotMatch, + preferOnScreenMatches, + resolveMaestroNodeRect, + type SnapshotNodeByIndex, +} from './runtime-target-ranking-geometry.ts'; +import { selectPreferredMaestroSnapshotMatch } from './runtime-target-ranking-duplicates.ts'; +import { inferMaestroMissingTabSlotMatch } from './runtime-target-ranking-tabs.ts'; +import type { + MaestroPreferredContext, + MaestroResolvedSnapshotMatch, +} from './runtime-target-policy.ts'; + +export function selectMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + matches: SnapshotNode[], + index: number | undefined, + visibleTextQuery: string | null, + frame: TouchReferenceFrame | undefined, + requireOnScreen = false, + promoteTapTarget = false, + preferredContext?: MaestroPreferredContext, +): { node: SnapshotNode; rect: Rect } | null { + const nodeByIndex = buildSnapshotNodeByIndex(nodes); + const candidates = resolveMaestroSnapshotMatchCandidates( + nodes, + matches, + nodeByIndex, + visibleTextQuery, + index, + frame, + requireOnScreen, + ); + const target = chooseMaestroSnapshotMatch( + nodes, + candidates, + index, + visibleTextQuery, + promoteTapTarget, + preferredContext, + ); + return promoteMaestroSnapshotMatch(nodes, target, nodeByIndex, promoteTapTarget, frame); +} + +function resolveMaestroSnapshotMatchCandidates( + nodes: SnapshotState['nodes'], + matches: SnapshotNode[], + nodeByIndex: SnapshotNodeByIndex, + visibleTextQuery: string | null, + index: number | undefined, + frame: TouchReferenceFrame | undefined, + requireOnScreen: boolean, +): MaestroResolvedSnapshotMatch[] { + const resolved = matches + .map((node) => resolveMaestroSnapshotMatch(nodes, node, nodeByIndex)) + .filter((candidate): candidate is MaestroResolvedSnapshotMatch => Boolean(candidate)); + const concrete = resolved.filter((candidate) => !candidate.inheritedRect); + const candidates = concrete.length > 0 ? concrete : resolved; + if (!visibleTextQuery || index !== undefined) return resolved; + return preferOnScreenMatches(candidates, frame, requireOnScreen); +} + +function resolveMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + node: SnapshotNode, + nodeByIndex: SnapshotNodeByIndex, +): MaestroResolvedSnapshotMatch | null { + const match = resolveMaestroNodeRect(nodes, node, nodeByIndex); + return match ? { node, rect: match.rect, inheritedRect: match.inherited } : null; +} + +function chooseMaestroSnapshotMatch( + nodes: SnapshotState['nodes'], + candidates: MaestroResolvedSnapshotMatch[], + index: number | undefined, + visibleTextQuery: string | null, + promoteTapTarget: boolean, + preferredContext?: MaestroPreferredContext, +): MaestroResolvedSnapshotMatch | null { + if (index !== undefined) return candidates[index] ?? null; + const best = selectPreferredMaestroSnapshotMatch( + nodes, + candidates, + visibleTextQuery, + promoteTapTarget, + preferredContext, + ); + if (!shouldInferMaestroTabSlot(best, visibleTextQuery, promoteTapTarget)) return best; + return inferMaestroMissingTabSlotMatch(nodes, best, visibleTextQuery!) ?? best; +} + +function shouldInferMaestroTabSlot( + match: MaestroResolvedSnapshotMatch | null, + visibleTextQuery: string | null, + promoteTapTarget: boolean, +): match is MaestroResolvedSnapshotMatch { + return Boolean(promoteTapTarget && visibleTextQuery && match); +} diff --git a/src/compat/maestro/runtime-targets.ts b/src/compat/maestro/runtime-targets.ts index d8d470f1c..569c80c17 100644 --- a/src/compat/maestro/runtime-targets.ts +++ b/src/compat/maestro/runtime-targets.ts @@ -1,95 +1,72 @@ -import type { ElementSelectorKey } from '../../core/interactor-types.ts'; +import type { TouchReferenceFrame } from '../../daemon/touch-reference-frame.ts'; import type { Rect, SnapshotNode, SnapshotState } from '../../kernel/snapshot.ts'; -import { parseSelectorChain } from '../../selectors/index.ts'; -import { matchesSelector } from '../../selectors/match.ts'; -import { evaluateIsPredicate } from '../../selectors/predicates.ts'; -import { normalizeText } from '../../selectors/find.ts'; import { buildSnapshotNodeByIndex, - extractNodeText, - findSnapshotAncestor, isDescendantOfSnapshotNode, - normalizeType, } from '../../snapshot/snapshot-processing.ts'; -import type { TouchReferenceFrame } from '../../daemon/touch-reference-frame.ts'; -import type { DaemonRequest } from '../../daemon/types.ts'; -import type { Selector, SelectorTerm } from '../../selectors/arguments.ts'; +import type { MaestroSelector } from './program-ir.ts'; +import { findMaestroTypedSelectorMatches } from './runtime-target-matching.ts'; import { - detectReactNativeOverlay, - readReactNativeOverlayActionNodes, -} from '../../core/react-native-overlay.ts'; - -const MAESTRO_TAP_TARGET_TYPE_RANK = new Map([ - ['button', 0], - ['link', 0], - ['textfield', 0], - ['textview', 0], - ['searchfield', 0], - ['switch', 0], - ['slider', 0], - ['cell', 1], - ['statictext', 2], -]); - -const RECT_CONTAINS_EPSILON = 1; - -export type MaestroTapOnOptions = { - childOf?: string; - index?: number; -}; - -export type MaestroSnapshotTarget = { - node: SnapshotNode; - rect: Rect; - frame?: TouchReferenceFrame; -}; - -type MaestroResolvedSnapshotMatch = { - node: SnapshotNode; - rect: Rect; - inheritedRect: boolean; -}; - -export type MaestroPreferredContext = { - node: SnapshotNode; - rect: Rect; -}; - -type MaestroMatchResolutionOptions = { + extractMaestroVisibleTextQueryFromSelector, + filterVisibleMaestroMatches, + type MaestroPlatform, + type MaestroPreferredContext, +} from './runtime-target-policy.ts'; +import { selectMaestroSnapshotMatch } from './runtime-target-ranking.ts'; + +export type MaestroMatchResolutionOptions = { promoteTapTarget?: boolean; preferredContext?: MaestroPreferredContext; requireOnScreen?: boolean; + allowLeadingCompositeLabelMatch?: boolean; }; -type MaestroSelectorMatchOptions = { - allowLeadingCompositeLabelMatch?: boolean; +export type MaestroTargetQuery = { + selector: MaestroSelector; + index?: number; + childOf?: MaestroSelector; }; -type ReactNativeOverlayFilterResult = { - matches: SnapshotNode[]; - blockedByReactNativeOverlay: boolean; +export type MaestroTargetEvidence = { + selector: MaestroSelector; + childOf?: MaestroSelector; + matched: boolean; + visible: boolean; + candidateCount: number; + ref?: string; }; -type SnapshotNodeByIndex = ReturnType; +export type MaestroTargetResolution = + | { + ok: true; + node: SnapshotNode; + rect: Rect; + matches: number; + evidence: MaestroTargetEvidence; + } + | { ok: false; message: string; evidence: MaestroTargetEvidence }; -type MaestroMatchWithScreenContainer = { - candidate: MaestroResolvedSnapshotMatch; - container: SnapshotNode & { rect: Rect }; -}; +export type { MaestroPreferredContext } from './runtime-target-policy.ts'; -export function resolveMaestroNodeFromSnapshot( +export function resolveMaestroTargetFromSnapshot( snapshot: SnapshotState, - selector: string, - options: MaestroTapOnOptions, - platform: 'ios' | 'android', + query: MaestroTargetQuery, + platform: MaestroPlatform, frame: TouchReferenceFrame | undefined, - resolutionOptions: MaestroMatchResolutionOptions = {}, -): { ok: true; node: SnapshotNode; rect: Rect } | { ok: false; message: string } { - let matches = findMaestroSelectorMatches(snapshot, selector, platform); - if (options.childOf) { - const parents = findMaestroSelectorMatches(snapshot, options.childOf, platform); + options: MaestroMatchResolutionOptions = {}, +): MaestroTargetResolution { + const matchOptions = { + allowLeadingCompositeLabelMatch: options.allowLeadingCompositeLabelMatch, + }; + let matches = findMaestroTypedSelectorMatches(snapshot, query.selector, matchOptions); + if (query.childOf) { + const parents = findMaestroTypedSelectorMatches(snapshot, query.childOf, matchOptions); if (parents.length === 0) { - return { ok: false, message: `Maestro childOf parent did not match: ${options.childOf}` }; + return { + ok: false, + message: 'Maestro childOf parent did not match.', + evidence: buildMaestroTargetEvidence(query, matches, [], undefined), + }; } const nodeByIndex = buildSnapshotNodeByIndex(snapshot.nodes); matches = matches.filter((node) => @@ -98,994 +75,52 @@ export function resolveMaestroNodeFromSnapshot( ), ); } - const visibleMatchesResult = filterVisibleMaestroMatches({ - nodes: snapshot.nodes, - matches, - platform, - }); + const visible = filterVisibleMaestroMatches({ nodes: snapshot.nodes, matches, platform }); const target = selectMaestroSnapshotMatch( snapshot.nodes, - visibleMatchesResult.matches, - options.index, - extractMaestroVisibleTextQuery(selector), + visible.matches, + query.index, + extractMaestroVisibleTextQueryFromSelector(query.selector), frame, - resolutionOptions.requireOnScreen === true, - resolutionOptions.promoteTapTarget, - resolutionOptions.preferredContext, + options.requireOnScreen === true, + options.promoteTapTarget, + options.preferredContext, ); + const evidence = buildMaestroTargetEvidence(query, matches, visible.matches, target?.node); if (!target) { - const index = options.index ?? 0; + const index = query.index === undefined ? '' : ` index ${query.index}`; return { ok: false, - message: visibleMatchesResult.blockedByReactNativeOverlay - ? `React Native overlay is covering app content: ${selector}` - : matches.length > 0 && visibleMatchesResult.matches.length === 0 - ? `Maestro selector matched ${matches.length} element(s), but none were visible: ${selector}` - : `Maestro selector did not match index ${index}: ${selector}`, - }; - } - return { ok: true, node: target.node, rect: target.rect }; -} - -export function resolveMaestroFuzzyTextNodeFromSnapshot( - snapshot: SnapshotState, - query: string, - platform: 'ios' | 'android', - frame: TouchReferenceFrame | undefined, - resolutionOptions: MaestroMatchResolutionOptions = {}, -): { ok: true; node: SnapshotNode; rect: Rect } | { ok: false; message: string } { - const matches = findMaestroFuzzyTextMatches(snapshot, query); - const visibleMatchesResult = filterVisibleMaestroMatches({ - nodes: snapshot.nodes, - matches, - platform, - }); - const target = selectMaestroSnapshotMatch( - snapshot.nodes, - visibleMatchesResult.matches, - undefined, - query, - frame, - resolutionOptions.requireOnScreen === true, - resolutionOptions.promoteTapTarget, - resolutionOptions.preferredContext, - ); - if (!target) { - return { ok: false, message: `Maestro fuzzy text did not match: ${query}` }; - } - return { ok: true, node: target.node, rect: target.rect }; -} - -export function resolveVisibleMaestroNodeFromSnapshot( - snapshot: SnapshotState, - selector: string, - platform: 'ios' | 'android', - frame: TouchReferenceFrame | undefined, -): { ok: true; node: SnapshotNode; rect: Rect; matches: number } | { ok: false; message: string } { - const matches = findMaestroSelectorMatches(snapshot, selector, platform, { - allowLeadingCompositeLabelMatch: false, - }); - const visibleMatchesResult = filterVisibleMaestroMatches({ - nodes: snapshot.nodes, - matches, - platform, - }); - const target = selectMaestroSnapshotMatch( - snapshot.nodes, - visibleMatchesResult.matches, - undefined, - extractMaestroVisibleTextQuery(selector), - frame, - true, - ); - if (!target) { - return { - ok: false, - message: visibleMatchesResult.blockedByReactNativeOverlay - ? `React Native overlay is covering app content: ${selector}` - : matches.length > 0 - ? `Maestro selector matched ${matches.length} element(s), but none were visible: ${selector}` - : `Maestro selector did not match: ${selector}`, + message: visible.blockedByReactNativeOverlay + ? 'React Native overlay is covering app content.' + : matches.length > 0 && visible.matches.length === 0 + ? `Maestro selector matched ${matches.length} element(s), but none were visible.` + : `Maestro selector did not match${index}.`, + evidence, }; } return { ok: true, node: target.node, rect: target.rect, - matches: visibleMatchesResult.matches.length, + matches: visible.matches.length, + evidence, }; } -export function hasMaestroSelectorMatchInSnapshot( - snapshot: SnapshotState, - selector: string, - platform: 'ios' | 'android', -): boolean { - return ( - findMaestroSelectorMatches(snapshot, selector, platform, { - allowLeadingCompositeLabelMatch: false, - }).length > 0 - ); -} - -function filterVisibleMaestroMatches(params: { - nodes: SnapshotState['nodes']; - matches: SnapshotNode[]; - platform: 'ios' | 'android'; -}): { matches: SnapshotNode[]; blockedByReactNativeOverlay: boolean } { - const visibleMatches = params.matches.filter( - (node) => - evaluateIsPredicate({ - predicate: 'visible', - node, - nodes: params.nodes, - platform: params.platform, - }).pass, - ); - const overlayFilter = filterReactNativeOverlayBlockedMatches( - params.nodes, - visibleMatches, - params.platform, - ); - return { - matches: overlayFilter.matches, - blockedByReactNativeOverlay: overlayFilter.blockedByReactNativeOverlay, - }; -} - -function filterReactNativeOverlayBlockedMatches( - nodes: SnapshotState['nodes'], +function buildMaestroTargetEvidence( + query: MaestroTargetQuery, matches: SnapshotNode[], - platform: 'ios' | 'android', -): ReactNativeOverlayFilterResult { - const overlay = detectReactNativeOverlay(nodes); - if (!overlay.detected) { - return { matches, blockedByReactNativeOverlay: false }; - } - if (!overlay.redBox) { - return { matches, blockedByReactNativeOverlay: false }; - } - const overlayControls = readReactNativeOverlayActionNodes(overlay); - const visibleOverlayControls = overlayControls.filter( - (node) => - evaluateIsPredicate({ - predicate: 'visible', - node, - nodes, - platform, - }).pass, - ); - if (visibleOverlayControls.length === 0) { - if (overlayControls.length === 0) { - return { matches: [], blockedByReactNativeOverlay: true }; - } - return { matches, blockedByReactNativeOverlay: false }; - } - const overlayNodeIndexes = new Set(visibleOverlayControls.map((node) => node.index)); - const overlayMatches = matches.filter((node) => overlayNodeIndexes.has(node.index)); - return { - matches: overlayMatches, - blockedByReactNativeOverlay: matches.length > 0 && overlayMatches.length === 0, - }; -} - -export function readMaestroSelectorPlatform(flags: DaemonRequest['flags']): 'ios' | 'android' { - return flags?.platform === 'android' ? 'android' : 'ios'; -} - -export function extractMaestroVisibleTextQuery(selectorExpression: string): string | null { - const chain = parseSelectorChain(selectorExpression); - const terms = chain.selectors.flatMap((selector) => selector.terms); - if (terms.length === 0) return null; - // Mixed selectors may encode more than a visible-text lookup, so they keep - // the exact selector path instead of fuzzy text fallback. - if (!terms.some((term) => term.key === 'label' || term.key === 'text')) return null; - if (!terms.every((term) => ['label', 'text', 'id'].includes(term.key))) return null; - const values = terms.map((term) => (typeof term.value === 'string' ? term.value : '')); - const first = values[0]; - if (!first || !values.every((value) => value === first)) return null; - return first; -} - -function findMaestroSelectorMatches( - snapshot: SnapshotState, - selectorExpression: string, - platform: 'ios' | 'android', - options: MaestroSelectorMatchOptions = {}, -): SnapshotNode[] { - const chain = parseSelectorChain(selectorExpression); - for (const selector of chain.selectors) { - const matches = snapshot.nodes.filter((node) => - matchesMaestroSelector(node, selector, platform, options), - ); - if (matches.length > 0) return matches; - } - return []; -} - -function findMaestroFuzzyTextMatches(snapshot: SnapshotState, query: string): SnapshotNode[] { - const normalizedQuery = normalizeText(query); - if (!normalizedQuery) return []; - const exact: SnapshotNode[] = []; - const partial: SnapshotNode[] = []; - for (const node of snapshot.nodes) { - const values = [node.label, extractNodeText(node), node.identifier, node.value].filter( - (value): value is string => Boolean(value), - ); - const normalizedValues = values.map((value) => normalizeText(value)); - if (normalizedValues.some((value) => value === normalizedQuery)) { - exact.push(node); - } else if (normalizedValues.some((value) => value.includes(normalizedQuery))) { - partial.push(node); - } - } - return exact.length > 0 ? exact : partial; -} - -function matchesMaestroSelector( - node: SnapshotNode, - selector: Selector, - platform: 'ios' | 'android', - options: MaestroSelectorMatchOptions, -): boolean { - if (matchesSelector(node, selector, platform)) return true; - return selector.terms.every((term) => matchesMaestroTerm(node, term, platform, options)); -} - -function matchesMaestroTerm( - node: SnapshotNode, - term: SelectorTerm, - platform: 'ios' | 'android', - options: MaestroSelectorMatchOptions, -): boolean { - if (typeof term.value !== 'string' || !isMaestroRegexTextKey(term.key)) { - return matchesSelector(node, { raw: term.key, terms: [term] }, platform); - } - const value = readMaestroTextTermValue(node, term.key); - return textEqualsOrRegex(value, term.value, options); -} - -function isMaestroRegexTextKey(key: SelectorTerm['key']): key is ElementSelectorKey { - return key === 'id' || key === 'label' || key === 'text' || key === 'value'; -} - -function readMaestroTextTermValue(node: SnapshotNode, key: ElementSelectorKey): string | undefined { - if (key === 'id') return node.identifier; - if (key === 'label') return node.label; - if (key === 'value') return node.value; - return extractNodeText(node); -} - -function textEqualsOrRegex( - value: string | undefined, - query: string, - options: MaestroSelectorMatchOptions = {}, -): boolean { - const text = value ?? ''; - const normalizedText = normalizeText(text); - const normalizedQuery = normalizeText(query); - if (normalizedText === normalizedQuery) return true; - if ( - options.allowLeadingCompositeLabelMatch !== false && - isLeadingCompositeLabelMatch(normalizedText, normalizedQuery) - ) { - return true; - } - if (!looksLikeMaestroRegex(query)) return false; - try { - return new RegExp(query).test(text); - } catch { - return false; - } -} - -function isLeadingCompositeLabelMatch(normalizedText: string, normalizedQuery: string): boolean { - if (!normalizedText || !normalizedQuery) return false; - if (!normalizedText.startsWith(normalizedQuery)) return false; - const next = normalizedText.at(normalizedQuery.length); - return next === ',' || next === ':' || next === ';'; -} - -function looksLikeMaestroRegex(query: string): boolean { - return /(?:\.\*|\.\+|\[[^\]]+\]|\([^)]*\)|\||\^|\$|\\[dDsSwWbB])/.test(query); -} - -function resolveNodeRect( - nodes: SnapshotState['nodes'], - node: SnapshotNode, - nodeByIndex: SnapshotNodeByIndex, -): { rect: Rect; inherited: boolean } | null { - if (node.rect && node.rect.width > 0 && node.rect.height > 0) { - return { rect: node.rect, inherited: false }; - } - if (node.rect) return null; - const rect = resolveRectlessNodeAncestorRect(nodes, node, nodeByIndex); - return rect ? { rect, inherited: true } : null; -} - -function resolveRectlessNodeAncestorRect( - nodes: SnapshotState['nodes'], - node: SnapshotNode, - nodeByIndex: SnapshotNodeByIndex, -): Rect | null { - let current: SnapshotNode | undefined = node; - while (typeof current.parentIndex === 'number') { - current = nodeByIndex.get(current.parentIndex) ?? nodes[current.parentIndex]; - if (!current) return null; - if (!current.rect) continue; - return current.rect.width > 0 && current.rect.height > 0 ? current.rect : null; - } - return null; -} - -function selectMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - matches: SnapshotNode[], - index: number | undefined, - visibleTextQuery: string | null, - frame: TouchReferenceFrame | undefined, - requireOnScreen = false, - promoteTapTarget = false, - preferredContext?: MaestroPreferredContext, -): { node: SnapshotNode; rect: Rect } | null { - const nodeByIndex = buildSnapshotNodeByIndex(nodes); - const candidates = resolveMaestroSnapshotMatchCandidates( - nodes, - matches, - nodeByIndex, - visibleTextQuery, - index, - frame, - requireOnScreen, - ); - const target = chooseMaestroSnapshotMatch( - nodes, - candidates, - index, - visibleTextQuery, - promoteTapTarget, - preferredContext, - ); - return promoteMaestroSnapshotMatch(nodes, target, nodeByIndex, promoteTapTarget, frame); -} - -function resolveMaestroSnapshotMatchCandidates( - nodes: SnapshotState['nodes'], - matches: SnapshotNode[], - nodeByIndex: SnapshotNodeByIndex, - visibleTextQuery: string | null, - index: number | undefined, - frame: TouchReferenceFrame | undefined, - requireOnScreen: boolean, -): MaestroResolvedSnapshotMatch[] { - const resolved = matches - .map((node) => resolveMaestroSnapshotMatch(nodes, node, nodeByIndex)) - .filter((candidate): candidate is MaestroResolvedSnapshotMatch => Boolean(candidate)); - const concrete = resolved.filter((candidate) => !candidate.inheritedRect); - const candidates = concrete.length > 0 ? concrete : resolved; - if (!visibleTextQuery || index !== undefined) return resolved; - return preferOnScreenMatches(candidates, frame, requireOnScreen); -} - -function resolveMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - node: SnapshotNode, - nodeByIndex: SnapshotNodeByIndex, -): MaestroResolvedSnapshotMatch | null { - const match = resolveNodeRect(nodes, node, nodeByIndex); - return match ? { node, rect: match.rect, inheritedRect: match.inherited } : null; -} - -function chooseMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - index: number | undefined, - visibleTextQuery: string | null, - promoteTapTarget: boolean, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch | null { - if (index !== undefined) return candidates[index] ?? null; - const best = selectPreferredMaestroSnapshotMatch( - nodes, - candidates, - visibleTextQuery, - promoteTapTarget, - preferredContext, - ); - if (!shouldInferMaestroTabSlot(best, visibleTextQuery, promoteTapTarget)) return best; - return inferMaestroMissingTabSlotMatch(nodes, best, visibleTextQuery!) ?? best; -} - -function selectPreferredMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - visibleTextQuery: string | null, - promoteTapTarget: boolean, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch | null { - if (!promoteTapTarget || !visibleTextQuery) { - return selectBestMaestroSnapshotMatch(nodes, candidates, visibleTextQuery, preferredContext); - } - return ( - selectLocalizedMaestroVisibleTextMatch(nodes, candidates, visibleTextQuery, preferredContext) ?? - selectBestMaestroSnapshotMatch(nodes, candidates, visibleTextQuery, preferredContext) - ); -} - -function shouldInferMaestroTabSlot( - match: MaestroResolvedSnapshotMatch | null, - visibleTextQuery: string | null, - promoteTapTarget: boolean, -): match is MaestroResolvedSnapshotMatch { - return Boolean(promoteTapTarget && visibleTextQuery && match); -} - -function selectBestMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - visibleTextQuery: string | null, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch | null { - const foregroundCandidates = preferForegroundContainerDuplicateMatches( - nodes, - candidates, - visibleTextQuery, - preferredContext, - ); - return ( - foregroundCandidates.sort((left, right) => - compareMaestroSnapshotMatches(left, right, visibleTextQuery), - )[0] ?? null - ); -} - -function selectLocalizedMaestroVisibleTextMatch( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - query: string, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch | null { - const exactMatches = candidates.filter( - (candidate) => maestroVisibleTextMatchRank(candidate.node, query) === 0, - ); - if (exactMatches.length >= 2) { - const localizedExact = selectLocalizedMaestroVisibleTextMatchFromCandidates( - nodes, - exactMatches, - query, - preferredContext, - ); - if (localizedExact) return localizedExact; - } - - const normalizedMatches = candidates.filter( - (candidate) => maestroVisibleTextMatchRank(candidate.node, query) === 1, - ); - if (exactMatches.length > 0 || normalizedMatches.length < 2) return null; - - return selectLocalizedMaestroVisibleTextMatchFromCandidates( - nodes, - normalizedMatches, - query, - preferredContext, - ); -} - -function selectLocalizedMaestroVisibleTextMatchFromCandidates( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - query: string, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch | null { - const nodeByIndex = buildSnapshotNodeByIndex(nodes); - const localized = candidates.filter( - (candidate) => - isLocalizedMaestroVisibleTextCandidate(candidate) && - candidates.some((container) => - isMaestroVisibleTextContainerForCandidate(nodes, container, candidate, nodeByIndex), - ), - ); - - return selectBestMaestroSnapshotMatch(nodes, localized, query, preferredContext); -} - -// fallow-ignore-next-line complexity -function preferForegroundContainerDuplicateMatches( - nodes: SnapshotState['nodes'], - candidates: MaestroResolvedSnapshotMatch[], - visibleTextQuery: string | null, - preferredContext?: MaestroPreferredContext, -): MaestroResolvedSnapshotMatch[] { - if (!visibleTextQuery || candidates.length < 2) return candidates; - const exact = candidates.filter( - (candidate) => maestroVisibleTextMatchRank(candidate.node, visibleTextQuery) === 0, - ); - if (exact.length < 2) return candidates; - - const nodeByIndex = buildSnapshotNodeByIndex(nodes); - const withContainers = exact - .map((candidate) => ({ - candidate, - container: findMaestroScreenContainer(nodes, candidate.node, nodeByIndex), - })) - .filter((entry): entry is MaestroMatchWithScreenContainer => Boolean(entry.container)); - if (withContainers.length < 2 || withContainers.length !== exact.length) return candidates; - - const overlapping = withContainers.filter((entry) => - hasOverlappingScreenContainer(entry, withContainers), - ); - if (overlapping.length < 2) return candidates; - - const foregroundByArea = selectLargestOverlappingScreenContainerMatches(overlapping); - if (foregroundByArea.length > 0) return foregroundByArea; - - const foregroundByContext = selectContextualOverlappingScreenContainerMatches( - nodes, - overlapping, - preferredContext, - nodeByIndex, - ); - if (foregroundByContext.length > 0) return foregroundByContext; - - // UIAutomator reports foreground transparent-stack screens later in the - // hierarchy while preserving both screens. Prefer the later overlapping - // screen only for exact duplicate text, so ordinary duplicate rows keep - // Maestro's read-order behavior. - const foregroundContainerIndex = Math.max(...overlapping.map((entry) => entry.container.index)); - const foreground = overlapping - .filter((entry) => entry.container.index === foregroundContainerIndex) - .map((entry) => entry.candidate); - return foreground.length > 0 ? foreground : candidates; -} - -function selectContextualOverlappingScreenContainerMatches( - nodes: SnapshotState['nodes'], - entries: MaestroMatchWithScreenContainer[], - context: MaestroPreferredContext | undefined, - nodeByIndex: SnapshotNodeByIndex, -): MaestroResolvedSnapshotMatch[] { - if (!context) return []; - const rawContextContainer = findMaestroScreenContainer(nodes, context.node, nodeByIndex); - const contextContainer = - rawContextContainer && rectContains(rawContextContainer.rect, context.rect) - ? rawContextContainer - : null; - const scored = entries.map((entry) => ({ - entry, - score: scoreScreenContainerAgainstContext(entry.container, context, contextContainer), - })); - const bestScore = Math.min(...scored.map((entry) => entry.score)); - if (!Number.isFinite(bestScore)) return []; - return scored.filter((entry) => entry.score === bestScore).map((entry) => entry.entry.candidate); -} - -function scoreScreenContainerAgainstContext( - container: SnapshotNode & { rect: Rect }, - context: MaestroPreferredContext, - contextContainer: (SnapshotNode & { rect: Rect }) | null, -): number { - if (contextContainer) { - if (container.index === contextContainer.index) return 0; - if (rectOverlapRatio(container.rect, contextContainer.rect) < 0.6) - return Number.POSITIVE_INFINITY; - return Math.abs(container.index - contextContainer.index); - } - - if (rectOverlapRatio(container.rect, context.rect) >= 0.6) return 0; - const orderDistance = container.index - context.node.index; - return orderDistance >= 0 ? orderDistance : 100_000 + Math.abs(orderDistance); -} - -function selectLargestOverlappingScreenContainerMatches( - entries: MaestroMatchWithScreenContainer[], -): MaestroResolvedSnapshotMatch[] { - const areas = entries.map((entry) => rectArea(entry.container.rect)); - const largestArea = Math.max(...areas); - const smallestArea = Math.min(...areas); - if (smallestArea <= 0 || largestArea < smallestArea * 1.2) return []; - return entries - .filter((entry) => rectArea(entry.container.rect) === largestArea) - .map((entry) => entry.candidate); -} - -function hasOverlappingScreenContainer( - entry: MaestroMatchWithScreenContainer, - candidates: MaestroMatchWithScreenContainer[], -): boolean { - return candidates.some( - (other) => - other !== entry && - entry.container.index !== other.container.index && - rectOverlapRatio(entry.container.rect, other.container.rect) >= 0.6, - ); -} - -function findMaestroScreenContainer( - nodes: SnapshotState['nodes'], - node: SnapshotNode, - nodeByIndex: SnapshotNodeByIndex, -): (SnapshotNode & { rect: Rect }) | null { - return findSnapshotAncestor(nodes, node, nodeByIndex, (ancestor) => { - if (!ancestor.rect) return null; - if (!isMaestroScreenContainerType(ancestor)) return null; - if (ancestor.rect.width < 240 || ancestor.rect.height < 320) return null; - return ancestor as SnapshotNode & { rect: Rect }; - }); -} - -function isMaestroScreenContainerType(node: SnapshotNode): boolean { - const type = normalizeType(node.type ?? ''); - return type === 'scrollview' || type === 'scroll-area' || type === 'list'; -} - -function isLocalizedMaestroVisibleTextCandidate(match: MaestroResolvedSnapshotMatch): boolean { - return ( - match.rect.width >= 16 && - match.rect.width <= 260 && - match.rect.height >= 24 && - match.rect.height <= 80 - ); -} - -function isMaestroVisibleTextContainerForCandidate( - nodes: SnapshotState['nodes'], - container: MaestroResolvedSnapshotMatch, - candidate: MaestroResolvedSnapshotMatch, - nodeByIndex: SnapshotNodeByIndex, -): boolean { - if (container.node.index === candidate.node.index) return false; - if (!rectContains(container.rect, candidate.rect)) return false; - if (rectArea(container.rect) < rectArea(candidate.rect) * 2) return false; - return isDescendantOfSnapshotNode(nodes, candidate.node, container.node, nodeByIndex); -} - -function preferOnScreenMatches( - matches: MaestroResolvedSnapshotMatch[], - frame: TouchReferenceFrame | undefined, - requireOnScreen: boolean, -): MaestroResolvedSnapshotMatch[] { - const onScreen = matches.filter((match) => isRectOnScreen(match.rect, frame)); - if (requireOnScreen) return onScreen; - return onScreen.length > 0 ? onScreen : matches; -} - -function isRectOnScreen(rect: Rect, frame: TouchReferenceFrame | undefined): boolean { - const maxX = frame?.referenceWidth ?? Number.POSITIVE_INFINITY; - const maxY = frame?.referenceHeight ?? Number.POSITIVE_INFINITY; - return rect.x < maxX && rect.y < maxY && rect.x + rect.width > 0 && rect.y + rect.height > 0; -} - -function compareMaestroSnapshotMatches( - left: MaestroResolvedSnapshotMatch, - right: MaestroResolvedSnapshotMatch, - visibleTextQuery: string | null, -): number { - const priorityRank = compareMaestroSnapshotMatchPriority(left, right, visibleTextQuery); - if (priorityRank !== 0) return priorityRank; - - if (!sameRoundedRect(left.rect, right.rect)) { - return left.node.index - right.node.index; - } - - const depthRank = (right.node.depth ?? 0) - (left.node.depth ?? 0); - if (depthRank !== 0) return depthRank; - - // Android transparent stacks can expose both the background screen and the - // foreground screen at the same coordinates. UIAutomator reports the - // foreground duplicate later in the snapshot, which matches Maestro's - // practical tap target for overlapping duplicates. - return right.node.index - left.node.index; -} - -function sameRoundedRect(left: Rect, right: Rect): boolean { - return ( - Math.round(left.x) === Math.round(right.x) && - Math.round(left.y) === Math.round(right.y) && - Math.round(left.width) === Math.round(right.width) && - Math.round(left.height) === Math.round(right.height) - ); -} - -function compareMaestroSnapshotMatchPriority( - left: MaestroResolvedSnapshotMatch, - right: MaestroResolvedSnapshotMatch, - visibleTextQuery: string | null, -): number { - if (visibleTextQuery) { - const textRank = - maestroVisibleTextMatchRank(left.node, visibleTextQuery) - - maestroVisibleTextMatchRank(right.node, visibleTextQuery); - if (textRank !== 0) return textRank; - } - - const typeRank = maestroTapTargetTypeRank(left.node) - maestroTapTargetTypeRank(right.node); - if (typeRank !== 0) return typeRank; - - const rectSourceRank = Number(left.inheritedRect) - Number(right.inheritedRect); - if (rectSourceRank !== 0) return rectSourceRank; - - const areaRank = - visibleTextQuery && maestroTapTargetTypeRank(left.node) === maestroTapTargetTypeRank(right.node) - ? rectArea(right.rect) - rectArea(left.rect) - : rectArea(left.rect) - rectArea(right.rect); - if (areaRank !== 0) return areaRank; - return 0; -} - -function rectArea(rect: Rect): number { - return rect.width * rect.height; -} - -function maestroTapTargetTypeRank(node: SnapshotNode): number { - return MAESTRO_TAP_TARGET_TYPE_RANK.get(normalizeType(node.type ?? '')) ?? 3; -} - -function inferMaestroMissingTabSlotMatch( - nodes: SnapshotState['nodes'], - match: MaestroResolvedSnapshotMatch, - query: string, -): MaestroResolvedSnapshotMatch | null { - if (!isMaestroTabStripContainerMatch(match, query)) return null; - const children = collectMaestroTabStripChildCandidates( - nodes, - match, - query, - buildSnapshotNodeByIndex(nodes), - ); - if (children.length === 0) return null; - const medianChildWidth = median(children.map((child) => child.rect.width)); - const allGaps = resolveHorizontalGaps( - match.rect, - children.map((child) => child.rect), - ); - const gap = selectMaestroMissingSlotGap(match, query, allGaps, medianChildWidth); - if (!gap) return null; - return matchWithRect(match, gap); -} - -function collectMaestroTabStripChildCandidates( - nodes: SnapshotState['nodes'], - match: MaestroResolvedSnapshotMatch, - query: string, - nodeByIndex: SnapshotNodeByIndex, -): Array { - return nodes - .filter((node): node is SnapshotNode & { rect: Rect } => { - return ( - node.index !== match.node.index && - Boolean(node.rect) && - isDescendantOfSnapshotNode(nodes, node, match.node, nodeByIndex) && - isMaestroTabStripChildCandidate(node as SnapshotNode & { rect: Rect }, match.rect, query) - ); - }) - .sort((left, right) => left.rect.x - right.rect.x); -} - -function selectMaestroMissingSlotGap( - match: MaestroResolvedSnapshotMatch, - query: string, - gaps: Array<{ x: number; width: number }>, - medianChildWidth: number, -): { x: number; width: number } | null { - const plausibleGaps = gaps.filter((gap) => - isPlausibleMissingTabSlot(gap.width, medianChildWidth), - ); - const leadingTextSlot = inferMaestroLeadingTextSlotGap(match, query, gaps); - const hasPlausibleLeadingGap = plausibleGaps.some((gap) => isLeadingGap(match.rect, gap)); - if (leadingTextSlot && !hasPlausibleLeadingGap) return leadingTextSlot; - if (plausibleGaps.length === 1) return plausibleGaps[0] ?? null; - return leadingTextSlot; -} - -function inferMaestroLeadingTextSlotGap( - match: MaestroResolvedSnapshotMatch, - query: string, - gaps: Array<{ x: number; width: number }>, -): { x: number; width: number } | null { - const leadingGap = gaps.find((gap) => Math.abs(gap.x - match.rect.x) < 1); - const estimatedLabelWidth = Math.max(48, Math.min(220, query.length * 8 + 24)); - if (!isLeadingTextSlotCandidate(match, query, leadingGap, estimatedLabelWidth)) return null; + visibleMatches: SnapshotNode[], + target: SnapshotNode | undefined, +): MaestroTargetEvidence { return { - x: match.rect.x, - width: Math.min(estimatedLabelWidth, leadingGap.width), + selector: query.selector, + ...(query.childOf === undefined ? {} : { childOf: query.childOf }), + matched: matches.length > 0, + visible: visibleMatches.length > 0, + candidateCount: matches.length, + ...(target?.ref === undefined ? {} : { ref: target.ref }), }; } - -function isLeadingTextSlotCandidate( - match: MaestroResolvedSnapshotMatch, - query: string, - gap: { x: number; width: number } | undefined, - estimatedLabelWidth: number, -): gap is { x: number; width: number } { - if (!gap) return false; - return ( - normalizeType(match.node.type ?? '') === 'scrollview' && - maestroVisibleTextMatchRank(match.node, query) <= 1 && - match.rect.width >= 240 && - match.rect.height >= 32 && - match.rect.height <= 80 && - gap.width <= match.rect.width * 0.55 && - gap.width >= estimatedLabelWidth * 0.6 - ); -} - -function isLeadingGap(rect: Rect, gap: { x: number; width: number }): boolean { - return Math.abs(gap.x - rect.x) < 1; -} - -function matchWithRect( - match: MaestroResolvedSnapshotMatch, - gap: { x: number; width: number }, -): MaestroResolvedSnapshotMatch { - return { - ...match, - rect: { - x: gap.x, - y: match.rect.y, - width: gap.width, - height: match.rect.height, - }, - }; -} - -function isMaestroTabStripContainerMatch( - match: MaestroResolvedSnapshotMatch, - query: string, -): boolean { - const type = normalizeType(match.node.type ?? ''); - if (type !== 'cell' && type !== 'other' && type !== 'scrollview' && type !== 'scroll-area') { - return false; - } - if (match.rect.width < 120 || match.rect.height < 32 || match.rect.height > 80) return false; - return maestroVisibleTextMatchRank(match.node, query) <= 1; -} - -function isMaestroTabStripChildCandidate( - node: SnapshotNode & { rect: Rect }, - container: Rect, - query: string, -): boolean { - const type = normalizeType(node.type ?? ''); - if (type !== 'button' && type !== 'cell' && type !== 'other') return false; - if (maestroVisibleTextMatchRank(node, query) <= 1) return false; - if (node.rect.width < 16 || node.rect.height < 16) return false; - if (!rectContains(container, node.rect)) return false; - return verticalOverlapRatio(container, node.rect) >= 0.5; -} - -function resolveHorizontalGaps( - container: Rect, - occupied: Rect[], -): Array<{ x: number; width: number }> { - const gaps: Array<{ x: number; width: number }> = []; - let cursor = container.x; - const containerRight = container.x + container.width; - for (const rect of occupied) { - const start = Math.max(container.x, rect.x); - const end = Math.min(containerRight, rect.x + rect.width); - if (start > cursor) gaps.push({ x: cursor, width: start - cursor }); - cursor = Math.max(cursor, end); - } - if (containerRight > cursor) gaps.push({ x: cursor, width: containerRight - cursor }); - return gaps; -} - -function isPlausibleMissingTabSlot(gapWidth: number, medianChildWidth: number): boolean { - if (gapWidth < 24 || medianChildWidth < 24) return false; - return gapWidth >= medianChildWidth * 0.4 && gapWidth <= medianChildWidth * 1.6; -} - -function median(values: number[]): number { - const sorted = [...values].sort((left, right) => left - right); - const middle = Math.floor(sorted.length / 2); - return sorted[middle] ?? 0; -} - -function verticalOverlapRatio(a: Rect, b: Rect): number { - const top = Math.max(a.y, b.y); - const bottom = Math.min(a.y + a.height, b.y + b.height); - const overlap = Math.max(0, bottom - top); - return overlap / Math.max(1, Math.min(a.height, b.height)); -} - -function promoteMaestroSnapshotMatch( - nodes: SnapshotState['nodes'], - match: MaestroResolvedSnapshotMatch | null, - nodeByIndex: SnapshotNodeByIndex, - promoteTapTarget: boolean, - frame: TouchReferenceFrame | undefined, -): { node: SnapshotNode; rect: Rect } | null { - if (!match) return null; - if (!promoteTapTarget) { - return { node: match.node, rect: match.rect }; - } - const ancestor = findMaestroTapAncestor(nodes, match, nodeByIndex, frame); - return ancestor ?? { node: match.node, rect: match.rect }; -} - -function findMaestroTapAncestor( - nodes: SnapshotState['nodes'], - match: MaestroResolvedSnapshotMatch, - nodeByIndex: SnapshotNodeByIndex, - frame: TouchReferenceFrame | undefined, -): { node: SnapshotNode; rect: Rect } | null { - if (isActionableMaestroTapTarget(match.node)) return null; - return findSnapshotAncestor(nodes, match.node, nodeByIndex, (ancestor) => { - if (!isActionableMaestroTapTarget(ancestor)) return null; - const ancestorRect = resolveNodeRect(nodes, ancestor, nodeByIndex); - if (!ancestorRect || !isUsefulMaestroTapAncestorRect(match.rect, ancestorRect.rect, frame)) { - return null; - } - return { node: ancestor, rect: ancestorRect.rect }; - }); -} - -function isActionableMaestroTapTarget(node: SnapshotNode): boolean { - const type = normalizeType(node.type ?? ''); - return ( - node.hittable === true || - type === 'button' || - type === 'link' || - type === 'cell' || - type === 'textfield' || - type === 'searchfield' || - type === 'switch' || - type === 'slider' - ); -} - -function isUsefulMaestroTapAncestorRect( - matchRect: Rect, - ancestorRect: Rect, - frame: TouchReferenceFrame | undefined, -): boolean { - if (!rectContains(ancestorRect, matchRect)) return false; - if (wouldPromoteTabSlotToWholeStrip(matchRect, ancestorRect)) return false; - const ancestorArea = rectArea(ancestorRect); - const matchArea = rectArea(matchRect); - // Keep promotion close to the matched label/id instead of jumping to a broad container. - if (matchArea > 0 && ancestorArea > matchArea * 30) return false; - if (frame) { - const frameArea = frame.referenceWidth * frame.referenceHeight; - // Full-screen ancestors are usually layout containers, not meaningful tap targets. - if (frameArea > 0 && ancestorArea > frameArea * 0.5) return false; - } - return true; -} - -function wouldPromoteTabSlotToWholeStrip(matchRect: Rect, ancestorRect: Rect): boolean { - if (ancestorRect.height < 32 || ancestorRect.height > 80) return false; - if (matchRect.height < ancestorRect.height * 0.75) return false; - if (verticalOverlapRatio(matchRect, ancestorRect) < 0.75) return false; - if (ancestorRect.width < 240) return false; - return ancestorRect.width >= matchRect.width * 3; -} - -function rectContains(container: Rect, child: Rect): boolean { - return ( - child.x >= container.x - RECT_CONTAINS_EPSILON && - child.y >= container.y - RECT_CONTAINS_EPSILON && - child.x + child.width <= container.x + container.width + RECT_CONTAINS_EPSILON && - child.y + child.height <= container.y + container.height + RECT_CONTAINS_EPSILON - ); -} - -function rectOverlapRatio(a: Rect, b: Rect): number { - const left = Math.max(a.x, b.x); - const right = Math.min(a.x + a.width, b.x + b.width); - const top = Math.max(a.y, b.y); - const bottom = Math.min(a.y + a.height, b.y + b.height); - const overlapArea = Math.max(0, right - left) * Math.max(0, bottom - top); - return overlapArea / Math.max(1, Math.min(rectArea(a), rectArea(b))); -} - -function maestroVisibleTextMatchRank(node: SnapshotNode, query: string): number { - const values = [node.label, extractNodeText(node), node.identifier, node.value].filter( - (value): value is string => Boolean(value), - ); - if (values.some((value) => value === query)) return 0; - if (values.some((value) => normalizeText(value) === normalizeText(query))) return 1; - if (values.some((value) => textEqualsOrRegex(value, query))) return 2; - return 3; -} diff --git a/src/compat/maestro/runtime.ts b/src/compat/maestro/runtime.ts deleted file mode 100644 index e1b03e4b5..000000000 --- a/src/compat/maestro/runtime.ts +++ /dev/null @@ -1,101 +0,0 @@ -import { asAppError } from '../../kernel/errors.ts'; -import type { ReplayVarScope } from '../../replay/vars.ts'; -import type { DaemonInvokeFn, DaemonResponse } from '../../daemon/types.ts'; -import { executeRunScriptFile } from './run-script.ts'; -import { MAESTRO_RUNTIME_COMMAND } from './runtime-commands.ts'; -import { - invokeMaestroAssertNotVisible, - invokeMaestroAssertVisible, - invokeMaestroWaitForAnimationToEnd, -} from './runtime-assertions.ts'; -import { - errorResponse, - type MaestroReplayInvoker, - type MaestroRuntimeInvoke, - type ReplayBaseRequest, -} from './runtime-support.ts'; -import { - invokeMaestroScrollUntilVisible, - invokeMaestroSwipeScreen, - invokeMaestroSwipeOn, - invokeMaestroTapOn, - invokeMaestroTapPointPercent, -} from './runtime-interactions.ts'; - -export async function invokeMaestroRuntimeCommand(params: { - command: string; - baseReq: ReplayBaseRequest; - positionals: string[]; - scope: ReplayVarScope; - line: number; - step: number; - invoke: DaemonInvokeFn; - invokeReplayAction: MaestroReplayInvoker; -}): Promise { - switch (params.command) { - case MAESTRO_RUNTIME_COMMAND.assertVisible: - return await invokeMaestroAssertVisible(params); - case MAESTRO_RUNTIME_COMMAND.assertNotVisible: - return await invokeMaestroAssertNotVisible(params); - case MAESTRO_RUNTIME_COMMAND.pressEnter: - return await invokeMaestroPressEnter(params); - case MAESTRO_RUNTIME_COMMAND.waitForAnimationToEnd: - return await invokeMaestroWaitForAnimationToEnd(params); - case MAESTRO_RUNTIME_COMMAND.scrollUntilVisible: - return await invokeMaestroScrollUntilVisible(params); - case MAESTRO_RUNTIME_COMMAND.swipeScreen: - return await invokeMaestroSwipeScreen(params); - case MAESTRO_RUNTIME_COMMAND.swipeOn: - return await invokeMaestroSwipeOn(params); - case MAESTRO_RUNTIME_COMMAND.tapOn: - return await invokeMaestroTapOn(params); - case MAESTRO_RUNTIME_COMMAND.tapPointPercent: - return await invokeMaestroTapPointPercent(params); - case MAESTRO_RUNTIME_COMMAND.runScript: - return invokeMaestroRunScript(params); - default: - return undefined; - } -} - -async function invokeMaestroPressEnter(params: { - baseReq: ReplayBaseRequest; - invoke: MaestroRuntimeInvoke; -}): Promise { - const keyboardResponse = await params.invoke({ - ...params.baseReq, - command: 'keyboard', - positionals: ['enter'], - }); - if (keyboardResponse.ok) return keyboardResponse; - - return await params.invoke({ - ...params.baseReq, - command: 'type', - positionals: ['\n'], - }); -} - -function invokeMaestroRunScript(params: { - baseReq: ReplayBaseRequest; - positionals: string[]; - scope: ReplayVarScope; -}): DaemonResponse { - const [scriptPath] = params.positionals; - if (!scriptPath) { - return errorResponse('INVALID_ARGS', 'runScript requires a file path.'); - } - try { - const outputEnv = executeRunScriptFile({ - scriptPath, - env: { - ...params.scope.values, - ...(params.baseReq.flags?.maestro?.runScriptEnv ?? {}), - }, - }); - return { ok: true, data: { outputEnv } }; - } catch (error) { - const appError = asAppError(error); - return errorResponse(appError.code, appError.message, appError.details); - } -} diff --git a/src/compat/maestro/support-matrix.ts b/src/compat/maestro/support-matrix.ts index d8d008d16..3e8d5e8e0 100644 --- a/src/compat/maestro/support-matrix.ts +++ b/src/compat/maestro/support-matrix.ts @@ -18,8 +18,6 @@ export const MAESTRO_COMPAT_SUPPORTED_CAPABILITIES = [ export const MAESTRO_COMPAT_TRACKER_URL = 'https://github.com/callstack/agent-device/issues/558'; -export const MAESTRO_NEW_ISSUE_URL = 'https://github.com/callstack/agent-device/issues/new'; - export function formatMaestroSupportedSubsetForCli(): string { return `Supported subset: ${formatMaestroCapabilityList(MAESTRO_COMPAT_SUPPORTED_CAPABILITIES)}.`; } diff --git a/src/compat/maestro/support.ts b/src/compat/maestro/support.ts deleted file mode 100644 index c2b3cc58a..000000000 --- a/src/compat/maestro/support.ts +++ /dev/null @@ -1,107 +0,0 @@ -import type { SessionAction } from '../../daemon/types.ts'; -import { AppError } from '../../kernel/errors.ts'; -import { MAESTRO_COMPAT_TRACKER_URL, MAESTRO_NEW_ISSUE_URL } from './support-matrix.ts'; -import type { MaestroCommand, MaestroFlowConfig, MaestroParseContext } from './types.ts'; - -export function action( - command: string, - positionals: string[] = [], - flags?: SessionAction['flags'], -): SessionAction { - return { - ts: Date.now(), - command, - positionals, - flags: flags ?? {}, - }; -} - -export function assertOnlyKeys( - value: Record, - command: string, - supportedKeys: readonly string[], -): void { - const supported = new Set(supportedKeys); - const unsupported = Object.keys(value).filter((key) => !supported.has(key)); - if (unsupported.length > 0) { - throw unsupportedMaestroSyntax( - `Maestro ${command} field "${unsupported[0]}" is not supported yet.`, - ); - } -} - -export function isPlainRecord(value: unknown): value is Record { - return typeof value === 'object' && value !== null && !Array.isArray(value); -} - -export function normalizeCommandList(value: unknown[]): MaestroCommand[] { - return value.map((entry, index) => { - if (typeof entry === 'string') return entry; - if (isPlainRecord(entry)) return entry; - throw new AppError( - 'INVALID_ARGS', - `Unsupported Maestro command at index ${index + 1}: expected a scalar or one-key map.`, - ); - }); -} - -export function normalizePlatform(value: string | undefined): 'android' | 'ios' | undefined { - if (!value) return undefined; - return normalizePlatformName(value); -} - -export function readEnvMap(value: unknown, name: string): Record { - if (value === undefined || value === null) return {}; - if (!isPlainRecord(value)) { - throw new AppError('INVALID_ARGS', `${name} expects a map.`); - } - const env: Record = {}; - for (const [key, raw] of Object.entries(value)) { - if (typeof raw === 'string' || typeof raw === 'number' || typeof raw === 'boolean') { - env[key] = String(raw); - } - } - return env; -} - -export function readTimeoutMs(value: unknown, fallback: number): number { - if (isPlainRecord(value) && typeof value.timeout === 'number' && Number.isFinite(value.timeout)) { - return Math.max(0, Math.floor(value.timeout)); - } - return fallback; -} - -export function requireAppId(config: MaestroFlowConfig, command: string): string { - if (config.appId) return config.appId; - throw new AppError('INVALID_ARGS', `${command} requires appId in the Maestro flow config.`); -} - -export function requireStringValue(command: string, value: unknown): string { - if (typeof value === 'string') return value; - throw new AppError('INVALID_ARGS', `${command} expects a string value.`); -} - -export function resolveMaestroString(value: string, context: MaestroParseContext): string { - return value.replace(/\$\{([A-Za-z_][A-Za-z0-9_.]*)\}/g, (match, key: string) => { - if (!Object.prototype.hasOwnProperty.call(context.env, key)) return match; - return String(context.env[key]); - }); -} - -export function unsupportedCommand(command: string): never { - throw unsupportedMaestroSyntax(`Maestro command "${command}" is not supported yet.`); -} - -export function unsupportedMaestroSyntax(message: string): never { - throw new AppError( - 'INVALID_ARGS', - `${message} See supported/unsupported Maestro compatibility at ${MAESTRO_COMPAT_TRACKER_URL}. If this syntax matters for your flows, comment there or open a focused issue at ${MAESTRO_NEW_ISSUE_URL}.`, - ); -} - -function normalizePlatformName(value: string): 'android' | 'ios' | undefined { - const normalized = value.trim().toLowerCase(); - if (normalized === 'android') return 'android'; - if (normalized === 'ios') return 'ios'; - return undefined; -} diff --git a/src/compat/maestro/types.ts b/src/compat/maestro/types.ts deleted file mode 100644 index 9a921746e..000000000 --- a/src/compat/maestro/types.ts +++ /dev/null @@ -1,45 +0,0 @@ -import type { ReplayControlActionSource, SessionAction } from '../../daemon/types.ts'; -import type { ParsedReplayScript, ReplayScriptMetadata } from '../../replay/script.ts'; - -export type MaestroFlowConfig = { - name?: string; - appId?: string; - env?: Record; - onFlowStart?: MaestroCommand[]; - onFlowComplete?: MaestroCommand[]; -}; - -export type MaestroReplayFlow = ParsedReplayScript & { - metadata: ReplayScriptMetadata; -}; - -/** - * The Maestro conversion pipeline's uniform result: actions plus a parallel - * per-action source array (`undefined` = the file currently being parsed; - * concrete `{path, line}` = a `runFlow` include's own file). - */ -export type MaestroConvertedActions = { - actions: SessionAction[]; - sources: (ReplayControlActionSource | undefined)[]; -}; - -export type MaestroCommand = string | Record; - -export type MaestroParseOptions = { - sourcePath?: string; - platform?: string; - env?: Record; - visitedPaths?: Set; -}; - -export type MaestroParseContext = { - baseDir?: string; - platform?: 'android' | 'ios'; - env: Record; - envOverrides: Record; - visitedPaths: Set; -}; - -export type MaestroCommandMapperDeps = { - parseRunFlowFile(filePath: string, context: MaestroParseContext): MaestroReplayFlow; -}; diff --git a/src/compat/replay-input.ts b/src/compat/replay-input.ts index e86f5d636..de7497651 100644 --- a/src/compat/replay-input.ts +++ b/src/compat/replay-input.ts @@ -1,12 +1,5 @@ import type { CommandFlags } from '../core/dispatch.ts'; import { AppError } from '../kernel/errors.ts'; -import { - collectReplayShellEnv, - parseReplayCliEnvEntries, - readReplayCliEnvEntries, - readReplayShellEnvSource, -} from '../replay/vars.ts'; -import { parseMaestroReplayFlow } from './maestro/replay-flow.ts'; import { parseReplayScriptDetailed, readReplayScriptMetadata, @@ -14,44 +7,16 @@ import { type ReplayScriptMetadata, } from '../replay/script.ts'; -type ReplayCompatParser = { - parse: ( - script: string, - options: ReplayCompatParseOptions, - ) => ParsedReplayScript & { metadata: ReplayScriptMetadata }; -}; - export type ParsedReplayInput = ParsedReplayScript & { metadata: ReplayScriptMetadata; }; -type ReplayInputParseOptions = { - sourcePath?: string; -}; - -type ReplayCompatParseOptions = ReplayInputParseOptions & { - platform?: string; - env?: Record; -}; - -const REPLAY_COMPAT_PARSERS: Record = { - maestro: { - parse: parseMaestroReplayFlow, - }, -}; - export function parseReplayInput( script: string, flags: CommandFlags | undefined, - options: ReplayInputParseOptions = {}, ): ParsedReplayInput { - const compatParser = readReplayCompatParser(flags); - if (compatParser) { - return compatParser.parse(script, { - ...options, - platform: flags?.platform, - env: readReplayCompatEnv(flags), - }); + if (flags?.replayBackend && flags.replayBackend !== 'maestro') { + throw new AppError('INVALID_ARGS', `Unsupported replay backend "${flags.replayBackend}".`); } return { @@ -59,20 +24,3 @@ export function parseReplayInput( metadata: readReplayScriptMetadata(script), }; } - -function readReplayCompatEnv(flags: CommandFlags | undefined): Record { - return { - ...collectReplayShellEnv(readReplayShellEnvSource(flags?.replayShellEnv)), - ...parseReplayCliEnvEntries(readReplayCliEnvEntries(flags?.replayEnv)), - }; -} - -function readReplayCompatParser(flags: CommandFlags | undefined): ReplayCompatParser | undefined { - const backend = flags?.replayBackend; - if (typeof backend !== 'string') return undefined; - const parser = REPLAY_COMPAT_PARSERS[backend]; - if (!parser) { - throw new AppError('INVALID_ARGS', `Unsupported replay backend "${backend}".`); - } - return parser; -} diff --git a/src/contracts/gesture-normalization.test.ts b/src/contracts/gesture-normalization.test.ts index 1704d7079..2973df3b9 100644 --- a/src/contracts/gesture-normalization.test.ts +++ b/src/contracts/gesture-normalization.test.ts @@ -4,6 +4,7 @@ import { gesturePayloadFromPositionals, gesturePayloadToPositionals, normalizePublicGesture, + normalizePublicSwipeMotion, swipePayloadFromPositionals, } from './gesture-normalization.ts'; @@ -62,6 +63,7 @@ test('gesture recording codec preserves timed fling duration when distance is om origin: { x: 10, y: 20 }, delta: { x: -180, y: 0 }, durationMs: 600, + executionProfile: 'endpoint-hold', }, deprecations: [{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' }], }); @@ -83,17 +85,44 @@ test('rotate serialization omits behaviorless velocity when no origin can delimi ]); }); -test('swipe is fling sugar unless legacy duration requests a pan', () => { +test('swipe is fling sugar unless legacy duration preserves swipe timing', () => { assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left' }), { gesture: { intent: 'fling', preset: 'left' }, deprecations: [], }); assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left', durationMs: 400 }), { - gesture: { intent: 'pan', preset: 'left', durationMs: 400 }, + gesture: { + intent: 'pan', + preset: 'left', + durationMs: 400, + executionProfile: 'endpoint-hold', + }, deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }], }); }); +test('coordinate swipe duration preserves swipe timing and endpoints', () => { + assert.deepEqual( + normalizePublicSwipeMotion({ + from: { x: 360, y: 400 }, + to: { x: 40, y: 400 }, + durationMs: 500, + }), + { + gesture: { + intent: 'pan', + origin: { x: 360, y: 400 }, + delta: { x: -320, y: 0 }, + durationMs: 500, + executionProfile: 'endpoint-hold', + }, + deprecations: [ + { rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }, + ], + }, + ); +}); + test('duration-bearing fling is an explicit compatibility alias for pan', () => { assert.deepEqual( normalizePublicGesture({ @@ -109,6 +138,7 @@ test('duration-bearing fling is an explicit compatibility alias for pan', () => origin: { x: 200, y: 300 }, delta: { x: -80, y: 0 }, durationMs: 500, + executionProfile: 'endpoint-hold', }, deprecations: [ { rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' }, diff --git a/src/contracts/gesture-normalization.ts b/src/contracts/gesture-normalization.ts index 5adcdaf06..de80f4ada 100644 --- a/src/contracts/gesture-normalization.ts +++ b/src/contracts/gesture-normalization.ts @@ -171,6 +171,7 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG input.distance ?? GESTURE_FLING_DEFAULT_DISTANCE, ), durationMs: input.durationMs, + executionProfile: 'endpoint-hold', }, deprecations: [ { rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' }, @@ -191,7 +192,12 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG return input.durationMs === undefined ? { gesture: { intent: 'fling', preset: input.preset }, deprecations: [] } : { - gesture: { intent: 'pan', preset: input.preset, durationMs: input.durationMs }, + gesture: { + intent: 'pan', + preset: input.preset, + durationMs: input.durationMs, + executionProfile: 'endpoint-hold', + }, deprecations: [ { rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }, ], @@ -246,6 +252,7 @@ export function normalizePublicSwipeMotion(input: { origin: input.from, delta: { x: input.to.x - input.from.x, y: input.to.y - input.from.y }, durationMs: input.durationMs, + executionProfile: 'endpoint-hold', }, deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }], }; diff --git a/src/contracts/gesture-plan-types.ts b/src/contracts/gesture-plan-types.ts index 797628b5c..a9657e8ec 100644 --- a/src/contracts/gesture-plan-types.ts +++ b/src/contracts/gesture-plan-types.ts @@ -8,6 +8,9 @@ export const GESTURE_DURATION_MAX_MS = 10_000; export type GestureIntent = 'fling' | 'pan' | 'pinch' | 'rotate' | 'transform'; +/** Selects one-pointer release timing without changing semantic gesture intent. */ +export type GestureExecutionProfile = 'endpoint-hold' | 'timed-pan'; + export type GestureSemanticInput = | { intent: 'fling'; from: Point; to: Point } | { intent: 'fling'; preset: SwipePreset } @@ -23,8 +26,14 @@ export type GestureSemanticInput = delta: Point; pointerCount?: GesturePointerCount; durationMs?: number; + executionProfile?: GestureExecutionProfile; + } + | { + intent: 'pan'; + preset: SwipePreset; + durationMs: number; + executionProfile?: GestureExecutionProfile; } - | { intent: 'pan'; preset: SwipePreset; durationMs: number } | { intent: 'pinch'; origin?: Point; scale: number } | { intent: 'rotate'; origin?: Point; degrees: number } | { @@ -46,6 +55,7 @@ export type PointerTrajectory = { export type SinglePointerGesturePlan = { topology: 'single'; intent: 'fling' | 'pan'; + executionProfile: GestureExecutionProfile; durationMs: number; viewport: Rect; pointers: readonly [PointerTrajectory]; diff --git a/src/contracts/gesture-plan.ts b/src/contracts/gesture-plan.ts index ac05b9c45..32840780d 100644 --- a/src/contracts/gesture-plan.ts +++ b/src/contracts/gesture-plan.ts @@ -9,6 +9,7 @@ import { import { GESTURE_DURATION_MAX_MS, GESTURE_DURATION_MIN_MS } from './gesture-plan-types.ts'; import type { GesturePlan, + GestureExecutionProfile, GestureSemanticInput, MultiTouchGesturePlan, PointerTrajectory, @@ -134,7 +135,15 @@ function buildFlingPlan( ): SinglePointerGesturePlan { if ('preset' in input) { const { from, to } = presetGestureEndpoints(input.preset, viewport); - return buildSinglePointerPlan('fling', from, to, GESTURE_FLING_DURATION_MS, viewport, profile); + return buildSinglePointerPlan( + 'fling', + from, + to, + GESTURE_FLING_DURATION_MS, + viewport, + 'endpoint-hold', + profile, + ); } if ('from' in input) { return buildSinglePointerPlan( @@ -143,6 +152,7 @@ function buildFlingPlan( input.to, GESTURE_FLING_DURATION_MS, viewport, + 'endpoint-hold', profile, ); } @@ -157,6 +167,7 @@ function buildFlingPlan( addPoints(start, gestureDirectionDelta(input.direction, distance)), GESTURE_FLING_DURATION_MS, viewport, + 'endpoint-hold', profile, ); } @@ -173,7 +184,15 @@ function buildPanPlan( ); if ('preset' in input) { const { from, to } = presetGestureEndpoints(input.preset, viewport); - return buildSinglePointerPlan('pan', from, to, durationMs, viewport, profile); + return buildSinglePointerPlan( + 'pan', + from, + to, + durationMs, + viewport, + input.executionProfile ?? 'timed-pan', + profile, + ); } if ((input.pointerCount ?? 1) === 1) { const start = finitePoint(input.origin, 'gesture pan origin'); @@ -184,6 +203,7 @@ function buildPanPlan( addPoints(start, delta), durationMs, viewport, + input.executionProfile ?? 'timed-pan', profile, ); } @@ -210,6 +230,7 @@ function buildSinglePointerPlan( to: Point, durationMs: number, viewport: Rect, + executionProfile: GestureExecutionProfile, profile: GesturePlatformProfile, ): SinglePointerGesturePlan { const start = finitePoint(from, `gesture ${intent} start`); @@ -222,6 +243,7 @@ function buildSinglePointerPlan( return { topology: 'single', intent, + executionProfile, durationMs, viewport, pointers: [{ pointerId: 0, samples }], diff --git a/src/contracts/scroll-gesture.test.ts b/src/contracts/scroll-gesture.test.ts index 9a1eac801..77ba0a54c 100644 --- a/src/contracts/scroll-gesture.test.ts +++ b/src/contracts/scroll-gesture.test.ts @@ -3,11 +3,32 @@ import assert from 'node:assert/strict'; import { AppError } from '../kernel/errors.ts'; import { assertScrollGestureInput, + buildInPageSwipeGesturePlan, buildScrollGesturePlan, clampGestureCoordinate, - pointFromPercentInFrame, } from './scroll-gesture.ts'; +test('buildInPageSwipeGesturePlan applies one inset lane policy in every direction', () => { + const frame = { referenceWidth: 400, referenceHeight: 800 }; + + assert.deepEqual(buildInPageSwipeGesturePlan('left', frame), { + direction: 'left', + x1: 340, + y1: 400, + x2: 60, + y2: 400, + ...frame, + }); + assert.deepEqual(buildInPageSwipeGesturePlan('down', frame), { + direction: 'down', + x1: 200, + y1: 120, + x2: 200, + y2: 680, + ...frame, + }); +}); + // The buildScrollGesturePlan vectors below are the canonical cross-language parity vectors, // mirrored by RunnerTests+ScrollGesture.swift (runnerScrollGesturePlan). If you change the scroll // math, update both this suite and the Swift parity test so the two ports cannot drift silently. @@ -179,14 +200,6 @@ test('assertScrollGestureInput rejects non-positive or non-finite pixels', () => } }); -test('pointFromPercentInFrame preserves authored percentages within valid pixel bounds', () => { - const frame = { referenceWidth: 400, referenceHeight: 800 }; - - assert.deepEqual(pointFromPercentInFrame(frame, 10, 30), { x: 40, y: 240 }); - assert.deepEqual(pointFromPercentInFrame(frame, 100, 0), { x: 399, y: 0 }); - assert.deepEqual(pointFromPercentInFrame(frame, -10, 100), { x: 0, y: 799 }); -}); - test('clampGestureCoordinate rounds values and clamps them into the safe gesture band', () => { assert.equal(clampGestureCoordinate(10.4, 8, 100), 10); assert.equal(clampGestureCoordinate(10.6, 8, 100), 11); diff --git a/src/contracts/scroll-gesture.ts b/src/contracts/scroll-gesture.ts index 1e7f86fff..12aa16c4f 100644 --- a/src/contracts/scroll-gesture.ts +++ b/src/contracts/scroll-gesture.ts @@ -65,6 +65,16 @@ export type SwipePresetGesturePlan = { referenceHeight: number; }; +export type InPageSwipeGesturePlan = { + direction: ScrollDirection; + x1: number; + y1: number; + x2: number; + y2: number; + referenceWidth: number; + referenceHeight: number; +}; + const DEFAULT_SCROLL_AMOUNT = 0.6; const DEFAULT_EDGE_PADDING_FRACTION = 0.05; // Edge presets stay close to the system gesture boundary without emitting edge coordinates. @@ -126,25 +136,26 @@ export function buildSwipePresetGesturePlan( preset: SwipePreset, frame: GestureReferenceFrame, ): SwipePresetGesturePlan { - // Mid-screen keeps in-page swipes on visible content; lower lanes can land in blank pager space. - const horizontalLanePercent = 50; - const inPageStartPercent = 85; - const inPageEndPercent = 15; - const [startPercent, endPercent, yPercent] = - preset === 'left' - ? [inPageStartPercent, inPageEndPercent, horizontalLanePercent] - : preset === 'right' - ? [inPageEndPercent, inPageStartPercent, horizontalLanePercent] - : preset === 'left-edge' - ? [99, 15, 50] - : [1, 85, 50]; + if (preset === 'left' || preset === 'right') { + const plan = buildInPageSwipeGesturePlan(preset, frame); + return { + preset, + x1: plan.x1, + y1: plan.y1, + x2: plan.x2, + y2: plan.y2, + referenceWidth: plan.referenceWidth, + referenceHeight: plan.referenceHeight, + }; + } + const [startPercent, endPercent] = preset === 'left-edge' ? [99, 15] : [1, 85]; const start = clampGesturePoint( - pointFromPercent(frame, startPercent, yPercent), + pointFromPercent(frame, startPercent, 50), frame, SWIPE_PRESET_EDGE_MARGIN_PX, ); const end = clampGesturePoint( - pointFromPercent(frame, endPercent, yPercent), + pointFromPercent(frame, endPercent, 50), frame, SWIPE_PRESET_EDGE_MARGIN_PX, ); @@ -172,6 +183,48 @@ export function gestureDirectionDelta(direction: ScrollDirection, distance: numb } } +/** Plans a centered, edge-inset finger motion that remains inside app content. */ +export function buildInPageSwipeGesturePlan( + direction: ScrollDirection, + frame: GestureReferenceFrame, +): InPageSwipeGesturePlan { + // These insets avoid system-edge gestures while retaining enough travel for pagers. + const startPercent = 85; + const endPercent = 15; + const centerPercent = 50; + const forward = direction === 'left' || direction === 'up'; + const startAxisPercent = forward ? startPercent : endPercent; + const endAxisPercent = forward ? endPercent : startPercent; + const vertical = direction === 'up' || direction === 'down'; + const start = clampGesturePoint( + pointFromPercent( + frame, + vertical ? centerPercent : startAxisPercent, + vertical ? startAxisPercent : centerPercent, + ), + frame, + SWIPE_PRESET_EDGE_MARGIN_PX, + ); + const end = clampGesturePoint( + pointFromPercent( + frame, + vertical ? centerPercent : endAxisPercent, + vertical ? endAxisPercent : centerPercent, + ), + frame, + SWIPE_PRESET_EDGE_MARGIN_PX, + ); + return { + direction, + x1: start.x, + y1: start.y, + x2: end.x, + y2: end.y, + referenceWidth: frame.referenceWidth, + referenceHeight: frame.referenceHeight, + }; +} + export function inferGestureReferenceFrame( nodes: Array>, ): GestureReferenceFrame | undefined { @@ -194,19 +247,6 @@ function pointFromPercent( }; } -export function pointFromPercentInFrame( - frame: GestureReferenceFrame, - xPercent: number, - yPercent: number, -): GesturePoint { - const point = pointFromPercent(frame, xPercent, yPercent); - // Frame dimensions are exclusive upper bounds for zero-based input coordinates. - return { - x: clampToRange(point.x, 0, Math.max(0, Math.round(frame.referenceWidth) - 1)), - y: clampToRange(point.y, 0, Math.max(0, Math.round(frame.referenceHeight) - 1)), - }; -} - function clampGesturePoint( point: GesturePoint, frame: GestureReferenceFrame, diff --git a/src/core/__tests__/dispatch-resolve.test.ts b/src/core/__tests__/dispatch-resolve.test.ts index bd790b2ce..669eff05b 100644 --- a/src/core/__tests__/dispatch-resolve.test.ts +++ b/src/core/__tests__/dispatch-resolve.test.ts @@ -1,10 +1,13 @@ import { beforeEach, test, vi } from 'vitest'; import assert from 'node:assert/strict'; -const { mockFindBootableIosSimulator, mockListAppleDevices } = vi.hoisted(() => ({ - mockFindBootableIosSimulator: vi.fn(), - mockListAppleDevices: vi.fn(), -})); +const { mockFindBootableIosSimulator, mockListAppleDevices, mockListAndroidDevices } = vi.hoisted( + () => ({ + mockFindBootableIosSimulator: vi.fn(), + mockListAppleDevices: vi.fn(), + mockListAndroidDevices: vi.fn(), + }), +); vi.mock('../../platforms/apple/core/devices.ts', async (importOriginal) => { const actual = await importOriginal(); @@ -15,6 +18,14 @@ vi.mock('../../platforms/apple/core/devices.ts', async (importOriginal) => { }; }); +vi.mock('../../platforms/android/devices.ts', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + listAndroidDevices: mockListAndroidDevices, + }; +}); + import { resolveTargetDevice, withDeviceInventoryProvider, @@ -59,10 +70,51 @@ const webDesktop: DeviceInfo = { booted: true, }; +const androidEmulator: DeviceInfo = { + platform: 'android', + id: 'emulator-5554', + name: 'Pixel 9 Pro XL', + kind: 'emulator', + target: 'mobile', + booted: true, +}; + beforeEach(() => { mockFindBootableIosSimulator.mockReset(); mockFindBootableIosSimulator.mockResolvedValue(null); mockListAppleDevices.mockReset(); + mockListAndroidDevices.mockReset(); +}); + +test('resolveTargetDevice narrows local Android discovery to an explicit serial', async () => { + mockListAndroidDevices.mockResolvedValue([androidEmulator]); + + const result = await resolveTargetDevice({ + platform: 'android', + serial: androidEmulator.id, + }); + + assert.equal(result.id, androidEmulator.id); + assert.deepEqual(Array.from(mockListAndroidDevices.mock.calls[0]?.[0]?.serialAllowlist ?? []), [ + androidEmulator.id, + ]); +}); + +test('resolveTargetDevice does not discover an explicit Android serial outside its allowlist', async () => { + mockListAndroidDevices.mockResolvedValue([]); + + await expectDeviceNotFound(() => + resolveTargetDevice({ + platform: 'android', + serial: androidEmulator.id, + androidDeviceAllowlist: 'emulator-5556', + }), + ); + + assert.deepEqual( + Array.from(mockListAndroidDevices.mock.calls[0]?.[0]?.serialAllowlist ?? []), + [], + ); }); test('resolveTargetDevice reuses request-scoped device resolution cache for identical selectors', async () => { diff --git a/src/core/__tests__/gesture-plan.test.ts b/src/core/__tests__/gesture-plan.test.ts index 25f44930d..e9c2bc796 100644 --- a/src/core/__tests__/gesture-plan.test.ts +++ b/src/core/__tests__/gesture-plan.test.ts @@ -22,6 +22,7 @@ describe('single-pointer plans', () => { ); assert.equal(plan.topology, 'single'); assert.equal(plan.intent, 'pan'); + assert.equal(plan.executionProfile, 'timed-pan'); assert.equal(plan.durationMs, 500); assert.deepEqual(plan.pointers[0].samples[0]?.point, { x: 100, y: 200 }); assert.deepEqual(plan.pointers[0].samples.at(-1)?.point, { x: 60, y: 225 }); @@ -36,8 +37,12 @@ describe('single-pointer plans', () => { { intent: 'fling', from: { x: 50, y: 100 }, to: { x: 230, y: 100 } }, PORTRAIT, ); + assert.equal(directional.topology, 'single'); + assert.equal(endpoints.topology, 'single'); assert.equal(directional.durationMs, 100); assert.equal(endpoints.durationMs, 100); + assert.equal(directional.executionProfile, 'endpoint-hold'); + assert.equal(endpoints.executionProfile, 'endpoint-hold'); assert.deepEqual(directional.pointers[0].samples.at(-1)?.point, { x: 50, y: 20 }); assert.deepEqual(endpoints.pointers[0].samples.at(-1)?.point, { x: 230, y: 100 }); }); @@ -53,6 +58,26 @@ describe('single-pointer plans', () => { assert.deepEqual(pan.pointers[0].samples[0]?.point, { x: 80, y: 430 }); assert.deepEqual(pan.pointers[0].samples.at(-1)?.point, { x: 360, y: 430 }); }); + + test('single-pointer pan preserves its execution profile', () => { + const plan = buildGesturePlan( + { + intent: 'pan', + origin: { x: 360, y: 430 }, + delta: { x: -320, y: 0 }, + durationMs: 500, + executionProfile: 'endpoint-hold', + }, + PORTRAIT, + ); + + assert.equal(plan.topology, 'single'); + assert.equal(plan.intent, 'pan'); + assert.equal(plan.executionProfile, 'endpoint-hold'); + assert.equal(plan.durationMs, 500); + assert.deepEqual(plan.pointers[0].samples[0]?.point, { x: 360, y: 430 }); + assert.deepEqual(plan.pointers[0].samples.at(-1)?.point, { x: 40, y: 430 }); + }); }); describe('two-pointer plans', () => { diff --git a/src/core/interactors/register-builtins.ts b/src/core/interactors/register-builtins.ts index ac508b84f..a57360678 100644 --- a/src/core/interactors/register-builtins.ts +++ b/src/core/interactors/register-builtins.ts @@ -7,6 +7,7 @@ import { type DeviceInventoryRequest, } from '../../contracts/device-inventory.ts'; import type { Platform, DeviceInfo } from '../../kernel/device.ts'; +import { resolveAndroidDiscoverySerialAllowlist } from '../platform-inventory.ts'; // The builtin-plugin wiring lives at the interactor seam (src/core/interactors/) — // the one place R3 (see scripts/layering/check.ts) permits a STATIC value import of @@ -56,9 +57,7 @@ const androidPlugin = { discoverDevices: async (request: DeviceInventoryRequest) => { const { listAndroidDevices } = await import('../../platforms/android/devices.ts'); return await listAndroidDevices({ - serialAllowlist: request.androidSerialAllowlist - ? new Set(request.androidSerialAllowlist) - : undefined, + serialAllowlist: resolveAndroidDiscoverySerialAllowlist(request), }); }, } as const satisfies PlatformPlugin; diff --git a/src/core/platform-inventory.ts b/src/core/platform-inventory.ts index 9a9ff68e8..ad0b6a596 100644 --- a/src/core/platform-inventory.ts +++ b/src/core/platform-inventory.ts @@ -26,9 +26,7 @@ export async function listLocalDeviceInventory( if (request.platform === 'android') { const { listAndroidDevices } = await import('../platforms/android/devices.ts'); return await listAndroidDevices({ - serialAllowlist: request.androidSerialAllowlist - ? new Set(request.androidSerialAllowlist) - : undefined, + serialAllowlist: resolveAndroidDiscoverySerialAllowlist(request), }); } @@ -50,3 +48,13 @@ export async function listLocalDeviceInventory( } return devices; } + +export function resolveAndroidDiscoverySerialAllowlist( + request: DeviceInventoryRequest, +): ReadonlySet | undefined { + const policyAllowlist = request.androidSerialAllowlist; + const selectedSerial = request.serial?.trim(); + if (!selectedSerial) return policyAllowlist ? new Set(policyAllowlist) : undefined; + if (!policyAllowlist) return new Set([selectedSerial]); + return new Set(policyAllowlist.includes(selectedSerial) ? [selectedSerial] : []); +} diff --git a/src/daemon/__tests__/direct-ios-selector.test.ts b/src/daemon/__tests__/direct-ios-selector.test.ts index 39600fece..3f6a3d194 100644 --- a/src/daemon/__tests__/direct-ios-selector.test.ts +++ b/src/daemon/__tests__/direct-ios-selector.test.ts @@ -3,11 +3,12 @@ import assert from 'node:assert/strict'; import { AppError } from '../../kernel/errors.ts'; import { isDirectIosSelectorFallbackError } from '../direct-ios-selector.ts'; -test('runner ELEMENT_OFFSCREEN always falls back to tree-based resolution', () => { +test('runner ELEMENT_OFFSCREEN delegates normally but stays typed for Maestro replay', () => { const error = new AppError('ELEMENT_OFFSCREEN', 'element resolved off-screen at (-161, 265)'); assert.equal(isDirectIosSelectorFallbackError(error), true); assert.equal(isDirectIosSelectorFallbackError(error, { allowElementNotFound: false }), true); - assert.equal(isDirectIosSelectorFallbackError(error, { delegateSemanticFailures: false }), true); + assert.equal(isDirectIosSelectorFallbackError(error, { delegateSemanticFailures: true }), true); + assert.equal(isDirectIosSelectorFallbackError(error, { delegateSemanticFailures: false }), false); }); test('runner ELEMENT_NOT_FOUND falls back for query callers that allow it', () => { diff --git a/src/daemon/__tests__/request-router-android-modal.test.ts b/src/daemon/__tests__/request-router-android-modal.test.ts index ca541c70f..45ac8b643 100644 --- a/src/daemon/__tests__/request-router-android-modal.test.ts +++ b/src/daemon/__tests__/request-router-android-modal.test.ts @@ -33,7 +33,7 @@ vi.mock('../../platforms/android/snapshot.ts', async (importOriginal) => { snapshotAndroid: vi.fn(async () => { snapshotCalls += 1; if (snapshotMode === 'throws') { - throw new Error('uiautomator dump did not return XML'); + throw new Error('Android snapshot helper did not return XML'); } if (snapshotCalls === 1) { return { diff --git a/src/daemon/__tests__/runtime-hints.test.ts b/src/daemon/__tests__/runtime-hints.test.ts index 791ac4b6e..e55968a09 100644 --- a/src/daemon/__tests__/runtime-hints.test.ts +++ b/src/daemon/__tests__/runtime-hints.test.ts @@ -282,10 +282,18 @@ test('applyRuntimeHintsToApp also writes the legacy ReactNativeDevPrefs.xml path }); const loggedArgs = await fs.readFile(argsLogPath, 'utf8'); + assert.match( + loggedArgs, + /shell run-as com\.example\.demo cat shared_prefs\/com\.example\.demo_preferences\.xml/, + ); assert.match( loggedArgs, /shell run-as com\.example\.demo cat shared_prefs\/ReactNativeDevPrefs\.xml/, ); + assert.match( + loggedArgs, + /shell run-as com\.example\.demo tee shared_prefs\/com\.example\.demo_preferences\.xml/, + ); assert.match( loggedArgs, /shell run-as com\.example\.demo tee shared_prefs\/ReactNativeDevPrefs\.xml/, @@ -483,7 +491,7 @@ test('applyRuntimeHintsToApp preserves write failures after a successful run-as assert.equal(error.message, 'Failed to write Android runtime hints for com.example.demo'); assert.equal( error.details?.hint, - 'adb run-as succeeded, but writing the React Native dev-server preference file failed. Inspect stderr/details for the failing shell command.', + 'adb run-as succeeded, but writing React Native dev-server preferences failed. Inspect stderr/details for the failing shell command.', ); assert.equal(error.details?.phase, 'write-runtime-hints'); assert.equal(error.details?.exitCode, 1); diff --git a/src/daemon/android-snapshot-timeout-evidence.ts b/src/daemon/android-snapshot-timeout-evidence.ts index 4188faf39..918699aed 100644 --- a/src/daemon/android-snapshot-timeout-evidence.ts +++ b/src/daemon/android-snapshot-timeout-evidence.ts @@ -170,16 +170,12 @@ function hasStringPath(value: unknown): value is { path: string } { function isAndroidSnapshotTimeoutError(error: NormalizedError): boolean { if (error.code !== 'COMMAND_FAILED') return false; return ( - hasKnownAndroidSnapshotTimeoutMessage(error) || - hasHelperTimeoutDetails(error.details?.helper) || - hasUiAutomatorDumpTimeoutDetails(error.details) + hasKnownAndroidSnapshotTimeoutMessage(error) || hasHelperTimeoutDetails(error.details?.helper) ); } function hasKnownAndroidSnapshotTimeoutMessage(error: NormalizedError): boolean { const text = `${error.message}\n${error.hint ?? ''}`; - if (/Android UI hierarchy dump timed out/i.test(text)) return true; - if (/Stock UIAutomator fallback was skipped/i.test(text)) return true; return /Android accessibility snapshots can be blocked/i.test(text); } @@ -190,21 +186,3 @@ function hasHelperTimeoutDetails(helper: unknown): boolean { const message = String(helperRecord.message ?? ''); return /TimeoutException/i.test(errorType) || /timed out/i.test(message); } - -function hasUiAutomatorDumpTimeoutDetails(details: Record | undefined): boolean { - if (!details) return false; - const timeoutMs = details?.timeoutMs; - const cmd = details?.cmd; - const args = normalizeArgList(details?.args); - return ( - typeof timeoutMs === 'number' && - cmd === 'adb' && - args.includes('uiautomator') && - args.includes('dump') - ); -} - -function normalizeArgList(rawArgs: unknown): string[] { - if (Array.isArray(rawArgs)) return rawArgs.map(String); - return typeof rawArgs === 'string' ? rawArgs.split(/\s+/) : []; -} diff --git a/src/daemon/direct-ios-selector.ts b/src/daemon/direct-ios-selector.ts index 0a3c3e8ba..3054446dd 100644 --- a/src/daemon/direct-ios-selector.ts +++ b/src/daemon/direct-ios-selector.ts @@ -54,11 +54,13 @@ export function isDirectIosSelectorFallbackError( return options.delegateSemanticFailures === true || options.allowElementNotFound === true; } if (appError.code === 'AMBIGUOUS_MATCH') return options.delegateSemanticFailures === true; - // The runner refuses to tap a hittable match whose frame is outside the app - // frame (closed drawer, off-viewport carousel). The tree-based path either - // prefers an on-screen candidate or raises the actionable offscreen_selector - // error, so always fall back. - if (appError.code === 'ELEMENT_OFFSCREEN') return true; + // Regular interactions delegate off-screen matches to the shared tree, which + // can prefer an on-screen candidate or raise offscreen_selector. Maestro + // replay keeps the typed runner outcome so its compatibility resolver can + // apply Maestro-specific ranking and tab-strip inference instead. + if (appError.code === 'ELEMENT_OFFSCREEN') { + return options.delegateSemanticFailures !== false; + } if (appError.code !== 'COMMAND_FAILED') return false; // Transport-failure classification stays message-based deliberately: the // sniffed shapes originate at 4+ scattered throw sites (runner-transport diff --git a/src/daemon/handlers/__tests__/session-replay-builtin-vars.test.ts b/src/daemon/handlers/__tests__/session-replay-builtin-vars.test.ts new file mode 100644 index 000000000..f82ddc798 --- /dev/null +++ b/src/daemon/handlers/__tests__/session-replay-builtin-vars.test.ts @@ -0,0 +1,68 @@ +import assert from 'node:assert/strict'; +import { test } from 'vitest'; +import { buildReplayBuiltinVars } from '../session-replay-vars.ts'; + +test('buildReplayBuiltinVars applies request values over metadata', () => { + const builtins = buildReplayBuiltinVars({ + req: { + token: 't', + session: 'session', + command: 'replay', + positionals: [], + flags: { + platform: 'ios', + target: 'desktop', + device: 'Test Device', + serial: 'android-serial', + udid: 'ios-udid', + shardIndex: 1, + shardCount: 4, + artifactsDir: '/tmp/artifacts', + }, + meta: { cwd: '/tmp/replay' }, + }, + sessionName: 'session', + metadata: { platform: 'android', target: 'mobile' }, + resolvedPath: '/tmp/replay/flows/test.ad', + }); + + assert.deepEqual(builtins, { + AD_SESSION: 'session', + AD_FILENAME: 'flows/test.ad', + AD_PLATFORM: 'ios', + AD_TARGET: 'desktop', + AD_DEVICE: 'Test Device', + AD_DEVICE_ID: 'android-serial', + AD_SHARD_INDEX: '1', + AD_SHARD_COUNT: '4', + AD_ARTIFACTS: '/tmp/artifacts', + }); +}); + +test('buildReplayBuiltinVars omits empty optional values', () => { + const builtins = buildReplayBuiltinVars({ + req: { + token: 't', + session: 'session', + command: 'replay', + positionals: [], + flags: { + device: '', + serial: '', + udid: 'ios-udid', + artifactsDir: '', + }, + meta: { cwd: '/tmp/replay' }, + }, + sessionName: 'session', + metadata: { platform: 'android', target: 'mobile' }, + resolvedPath: '/tmp/replay/test.ad', + }); + + assert.deepEqual(builtins, { + AD_SESSION: 'session', + AD_FILENAME: 'test.ad', + AD_PLATFORM: 'android', + AD_TARGET: 'mobile', + }); +}); diff --git a/src/daemon/handlers/__tests__/session-replay-resume.test.ts b/src/daemon/handlers/__tests__/session-replay-resume.test.ts index 4746e94d9..ea4c7c7c8 100644 --- a/src/daemon/handlers/__tests__/session-replay-resume.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-resume.test.ts @@ -10,23 +10,12 @@ function action(overrides: Partial = {}): SessionAction { return { ts: 0, command: 'click', positionals: ['label="Save"'], flags: {}, ...overrides }; } -// --- from === 1: nothing skipped, always allowed --- - -test('from 1 is always allowed, even when step 1 itself is control flow', () => { - const actions: SessionAction[] = [ - action({ - command: 'back', - positionals: [], - replayControl: { kind: 'retry', maxRetries: 2, actions: [action({ command: 'back' })] }, - }), - action({ command: 'click' }), - ]; +test('generic .ad replay allows resuming from the first action', () => { + const actions: SessionAction[] = [action({ command: 'back', positionals: [] }), action()]; assert.deepEqual(evaluateReplayResumePreflight({ from: 1, actions }), { allowed: true }); }); -// --- plain steps: always resumable --- - -test('resuming after only plain, non-control-flow steps is allowed', () => { +test('generic .ad replay allows resuming after earlier actions', () => { const actions: SessionAction[] = [ action({ command: 'open', positionals: ['Demo'] }), action({ command: 'click', positionals: ['label="Continue"'] }), @@ -35,103 +24,16 @@ test('resuming after only plain, non-control-flow steps is allowed', () => { assert.deepEqual(evaluateReplayResumePreflight({ from: 3, actions }), { allowed: true }); }); -// --- control flow in the skipped range --- - -test('rejects when a skipped step is a retry block', () => { - const actions: SessionAction[] = [ - action({ - command: 'back', - positionals: [], - replayControl: { kind: 'retry', maxRetries: 2, actions: [action({ command: 'back' })] }, - }), - action({ command: 'click' }), - ]; - const result = evaluateReplayResumePreflight({ from: 2, actions }); - assert.equal(result.allowed, false); - if (result.allowed) return; - assert.match(result.reason, /control flow/); - assert.match(result.reason, /retry/); -}); - -test('rejects when a skipped step is a maestroRunFlowWhen block', () => { - const actions: SessionAction[] = [ - action({ - command: 'back', - positionals: [], - replayControl: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: 'label="Continue"', - actions: [action({ command: 'back' })], - }, - }), - action({ command: 'click' }), - ]; - const result = evaluateReplayResumePreflight({ from: 2, actions }); - assert.equal(result.allowed, false); - if (result.allowed) return; - assert.match(result.reason, /maestroRunFlowWhen/); -}); - -// --- control flow AS the resume target --- - -test('rejects when the resume target itself is a control-flow block', () => { - const actions: SessionAction[] = [ - action({ command: 'open', positionals: ['Demo'] }), - action({ - command: 'back', - positionals: [], - replayControl: { kind: 'retry', maxRetries: 1, actions: [action({ command: 'back' })] }, - }), - ]; - const result = evaluateReplayResumePreflight({ from: 2, actions }); - assert.equal(result.allowed, false); - if (result.allowed) return; - assert.match(result.reason, /cannot be safely resumed into/); -}); - -// --- outputEnv-producing skipped steps --- - -test('rejects when a skipped step can produce outputEnv values (maestro runScript)', () => { - const actions: SessionAction[] = [ - action({ command: '__maestroRunScript', positionals: ['./setup.js'] }), - action({ command: 'click' }), - ]; - const result = evaluateReplayResumePreflight({ from: 2, actions }); - assert.equal(result.allowed, false); - if (result.allowed) return; - assert.match(result.reason, /outputEnv/); -}); - -test('an outputEnv-producing step AT the resume target itself (not skipped) is fine', () => { - const actions: SessionAction[] = [ - action({ command: 'open', positionals: ['Demo'] }), - action({ command: '__maestroRunScript', positionals: ['./setup.js'] }), - ]; - // Resuming AT the runScript step re-executes it (it is not skipped), so its - // outputEnv is produced fresh, not assumed from a prior run. - assert.deepEqual(evaluateReplayResumePreflight({ from: 2, actions }), { allowed: true }); -}); - -// --- same child index recurring under different parents / repeated plan indices --- -// (Resume addressing is purely by top-level plan index; two structurally -// distinct occurrences of the same command are still distinguished correctly -// because each occupies its own array slot.) - -test('expanded repeats occupy distinct, independently addressable plan indices', () => { +test('repeated generic .ad action lines occupy distinct plan indices', () => { const actions: SessionAction[] = [ action({ command: 'click', positionals: ['label="Item"'] }), action({ command: 'click', positionals: ['label="Item"'] }), action({ command: 'click', positionals: ['label="Item"'] }), ]; - // Resuming at the 3rd occurrence skips the first two identical-looking - // steps; neither is control-flow or outputEnv-producing, so it is allowed. assert.deepEqual(evaluateReplayResumePreflight({ from: 3, actions }), { allowed: true }); }); -// --- buildReplayDivergenceResume: report-facing wrapper --- - -test('buildReplayDivergenceResume reports allowed:true with from/planDigest for a safe failed step', () => { +test('buildReplayDivergenceResume reports a resumable generic .ad failure', () => { const actions: SessionAction[] = [ action({ command: 'open', positionals: ['Demo'] }), action({ command: 'click', positionals: ['label="Save"'] }), @@ -143,24 +45,3 @@ test('buildReplayDivergenceResume reports allowed:true with from/planDigest for }); assert.deepEqual(resume, { allowed: true, from: 2, planDigest: 'abc123' }); }); - -test('buildReplayDivergenceResume reports allowed:false with from/planDigest/reason when unsafe', () => { - const actions: SessionAction[] = [ - action({ - command: 'back', - positionals: [], - replayControl: { kind: 'retry', maxRetries: 1, actions: [action({ command: 'back' })] }, - }), - action({ command: 'click' }), - ]; - const resume = buildReplayDivergenceResume({ - failedIndex: 2, - actions, - planDigest: 'abc123', - }); - assert.equal(resume.allowed, false); - assert.equal(resume.from, 2); - assert.equal(resume.planDigest, 'abc123'); - if (resume.allowed) return; - assert.ok(resume.reason.length > 0); -}); diff --git a/src/daemon/handlers/__tests__/session-replay-runtime-binding.test.ts b/src/daemon/handlers/__tests__/session-replay-runtime-binding.test.ts new file mode 100644 index 000000000..73ca82b14 --- /dev/null +++ b/src/daemon/handlers/__tests__/session-replay-runtime-binding.test.ts @@ -0,0 +1,111 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { beforeEach, expect, test, vi } from 'vitest'; +import { makeIosSession } from '../../../__tests__/test-utils/session-factories.ts'; +import { resolveTargetDevice } from '../../../core/dispatch.ts'; +import { SessionStore } from '../../session-store.ts'; +import { runReplayScriptFile } from '../session-replay-runtime.ts'; +import { baseReplayRequest as baseReq } from './session-replay-runtime.fixtures.ts'; + +vi.mock('../../../core/dispatch.ts', async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, dispatchCommand: vi.fn(async () => ({})), resolveTargetDevice: vi.fn() }; +}); + +const mockResolveTargetDevice = vi.mocked(resolveTargetDevice); + +beforeEach(() => { + mockResolveTargetDevice.mockReset(); + mockResolveTargetDevice.mockResolvedValue(makeIosSession('resolved').device); +}); + +test('typed Maestro does not resolve a device before an explicit-platform flow needs one', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-explicit-platform-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(flowPath, 'appId: com.example.app\n---\n- launchApp\n'); + const invoked: string[] = []; + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { replayBackend: 'maestro', platform: 'android' }, + }), + sessionName: 'default', + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (request) => { + invoked.push(request.command); + return { ok: true, data: {} }; + }, + }); + + expect(response.ok).toBe(true); + expect(mockResolveTargetDevice).not.toHaveBeenCalled(); + expect(invoked).toEqual(['open']); +}); + +test('typed Maestro keeps a port-only runtime digest stable after launch binds the device', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-runtime-device-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(flowPath, 'appId: com.example.app\n---\n- launchApp\n- back\n'); + const runtime = { metroPort: 8083 }; + + const firstAttempt = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + runtime, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (request) => { + if (request.command === 'open') { + expect(request.runtime).toEqual({ + platform: 'ios', + metroHost: '127.0.0.1', + metroPort: 8083, + }); + if (!request.runtime) throw new Error('open must carry effective runtime hints'); + sessionStore.set(sessionName, makeIosSession(sessionName)); + sessionStore.setRuntimeHints(sessionName, request.runtime); + return { ok: true, data: {} }; + } + return { ok: false, error: { code: 'COMMAND_FAILED', message: 'back failed' } }; + }, + }); + expect(firstAttempt.ok).toBe(false); + if (firstAttempt.ok) return; + const divergence = firstAttempt.error.details?.divergence as { + resume: { from: number; planDigest: string }; + }; + expect(divergence.resume.from).toBe(2); + + const invoked: string[] = []; + const resumedAttempt = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { + replayBackend: 'maestro', + platform: 'ios', + replayFrom: divergence.resume.from, + replayPlanDigest: divergence.resume.planDigest, + }, + runtime, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (request) => { + invoked.push(request.command); + return { ok: true, data: {} }; + }, + }); + + expect(resumedAttempt.ok).toBe(true); + expect(invoked).toEqual(['back']); +}); diff --git a/src/daemon/handlers/__tests__/session-replay-runtime-failure.test.ts b/src/daemon/handlers/__tests__/session-replay-runtime-failure.test.ts index 423beb7c6..a5193b12f 100644 --- a/src/daemon/handlers/__tests__/session-replay-runtime-failure.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-runtime-failure.test.ts @@ -249,13 +249,13 @@ test('divergence screen never masks the original cause when the session already expect(screen.reason).toBe('no-session'); }); -// --- Control-flow-wrapped include provenance (reviewer probe scenario) --- +// --- Typed Maestro include provenance --- // // The RN suite's own launch include is retry-wrapped, so the single most // common real failure site (a launch wait timeout inside the include) must // report the INCLUDE's file+line, not the wrapping `retry:`/`runFlow.when:` -// line in the root flow. Regression for the leak where replayControl.actions -// kept the transient replaySource field but the runtime never consulted it. +// line in the root flow. These tests exercise the public typed Maestro replay +// path and its source-aware failure reporting. function writeMaestroInclude(root: string): string { const childPath = path.join(root, 'child.yaml'); @@ -311,6 +311,8 @@ test('a failure inside a retry-wrapped runFlow include reports the include file expect(step.index).toBe(1); expect(step.source.path).toBe(childPath); expect(step.source.line).toBe(3); + expect(divergence.action).toBe('back'); + expect(response.error.details?.action).toBe('back'); // The transport-internal provenance marker is stripped from the flat details. expect(response.error.details?.replaySource).toBeUndefined(); }); @@ -360,5 +362,99 @@ test('a failure inside a runtime runFlow.when-wrapped include reports the includ expect(step.index).toBe(1); expect(step.source.path).toBe(childPath); expect(step.source.line).toBe(3); + expect(divergence.action).toBe('back'); + expect(response.error.details?.action).toBe('back'); expect(response.error.details?.replaySource).toBeUndefined(); }); + +test('typed Maestro failures rank suggestions with Maestro regex selector semantics', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-suggestion-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync( + flowPath, + ['appId: com.example.app', '---', '- tapOn:', ' id: save-.*', ''].join('\n'), + ); + const nodes = [ + { + index: 0, + depth: 0, + type: 'Application', + rect: { x: 0, y: 0, width: 402, height: 874 }, + }, + { + index: 1, + parentIndex: 0, + depth: 1, + type: 'Button', + identifier: 'save-button', + label: 'Save', + rect: { x: 20, y: 40, width: 120, height: 44 }, + hittable: true, + }, + ]; + mockDispatchCommand.mockResolvedValue({ nodes, truncated: false, backend: 'xctest' }); + + const response = await runReplayScriptFile({ + req: baseReq({ positionals: [flowPath], flags: { replayBackend: 'maestro' } }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (req) => { + if (req.command === 'snapshot') return { ok: true, data: { createdAt: 0, nodes } }; + if (req.command === 'click') { + return { ok: false, error: { code: 'COMMAND_FAILED', message: 'tap failed' } }; + } + return { ok: true, data: {} }; + }, + }); + + expect(response.ok).toBe(false); + if (response.ok) return; + const divergence = response.error.details?.divergence as { + suggestionCount: number; + suggestions: Array<{ selector: string; basis: string }>; + }; + expect(divergence.suggestionCount).toBe(1); + expect(divergence.suggestions).toEqual([ + expect.objectContaining({ selector: expect.stringContaining('save-button'), basis: 'id' }), + ]); +}); + +test('typed Maestro failures never serialize input text', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-text-redaction-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const flowPath = path.join(root, 'flow.yaml'); + const typedText = 'highly-sensitive-value'; + fs.writeFileSync( + flowPath, + ['appId: com.example.app', '---', `- inputText: ${typedText}`, ''].join('\n'), + ); + mockDispatchCommand.mockRejectedValue(new Error('no device runner available')); + + const response = await runReplayScriptFile({ + req: baseReq({ positionals: [flowPath], flags: { replayBackend: 'maestro' } }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (req) => + req.command === 'type' + ? { + ok: false, + error: { code: 'COMMAND_FAILED', message: `could not type ${typedText}` }, + } + : { ok: true, data: {} }, + }); + + expect(response.ok).toBe(false); + expect(JSON.stringify(response)).not.toContain(typedText); + if (!response.ok) { + const divergence = response.error.details?.divergence as { action: string }; + expect(divergence.action).toBe('inputText '); + expect(response.error.message).toContain('inputText '); + } +}); diff --git a/src/daemon/handlers/__tests__/session-replay-runtime-plan.test.ts b/src/daemon/handlers/__tests__/session-replay-runtime-plan.test.ts index 1fa90ad78..8696ff3fc 100644 --- a/src/daemon/handlers/__tests__/session-replay-runtime-plan.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-runtime-plan.test.ts @@ -10,7 +10,7 @@ import os from 'node:os'; import path from 'node:path'; import { runReplayScriptFile } from '../session-replay-runtime.ts'; import { SessionStore } from '../../session-store.ts'; -import { dispatchCommand } from '../../../core/dispatch.ts'; +import { dispatchCommand, resolveTargetDevice } from '../../../core/dispatch.ts'; import { makeIosSession } from '../../../__tests__/test-utils/session-factories.ts'; import { baseReplayRequest as baseReq, @@ -18,10 +18,13 @@ import { } from './session-replay-runtime.fixtures.ts'; const mockDispatchCommand = vi.mocked(dispatchCommand); +const mockResolveTargetDevice = vi.mocked(resolveTargetDevice); beforeEach(() => { mockDispatchCommand.mockReset(); mockDispatchCommand.mockResolvedValue({}); + mockResolveTargetDevice.mockReset(); + mockResolveTargetDevice.mockResolvedValue(makeIosSession('resolved').device); }); test('resume skips steps 1..from-1 without invoking them and executes only from the reported step', async () => { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-replay-resume-skip-')); @@ -312,3 +315,139 @@ test('resume rejects resuming past a retry-wrapped step in the skipped range', a expect(resumedAttempt.error.code).toBe('INVALID_ARGS'); expect(resumedAttempt.error.message).toMatch(/control flow/); }); + +test('typed Maestro resume digest binds an inferred session target', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-session-target-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + const session = makeIosSession(sessionName); + sessionStore.set(sessionName, session); + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(flowPath, 'appId: com.example.app\n---\n- back\n'); + + const firstAttempt = await runReplayScriptFile({ + req: baseReq({ positionals: [flowPath], flags: { replayBackend: 'maestro' } }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async () => ({ + ok: false, + error: { code: 'COMMAND_FAILED', message: 'back failed' }, + }), + }); + expect(firstAttempt.ok).toBe(false); + if (firstAttempt.ok) return; + const divergence = firstAttempt.error.details?.divergence as { + resume: { from: number; planDigest: string }; + }; + + sessionStore.set(sessionName, { + ...session, + device: { ...session.device, target: 'tv' }, + }); + const resumedAttempt = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { + replayBackend: 'maestro', + replayFrom: divergence.resume.from, + replayPlanDigest: divergence.resume.planDigest, + }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async () => { + throw new Error('must not execute after the effective session target changed'); + }, + }); + + expect(resumedAttempt.ok).toBe(false); + if (resumedAttempt.ok) return; + expect(resumedAttempt.error.code).toBe('INVALID_ARGS'); + expect(resumedAttempt.error.message).toMatch(/plan digest/); +}); + +test('typed Maestro rejects selectors that conflict with an active session', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-session-conflict-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(flowPath, 'appId: com.example.app\n---\n- back\n'); + const invoke = vi.fn(async () => ({ ok: true as const, data: {} })); + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { replayBackend: 'maestro', platform: 'android' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke, + }); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.code).toBe('INVALID_ARGS'); + expect(response.error.message).toMatch(/already bound.*--platform=android/i); + expect(invoke).not.toHaveBeenCalled(); +}); + +test('typed Maestro resume digest binds effective stored runtime hints', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-session-runtime-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + sessionStore.setRuntimeHints(sessionName, { + platform: 'ios', + metroHost: '127.0.0.1', + metroPort: 8083, + }); + const flowPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(flowPath, 'appId: com.example.app\n---\n- back\n'); + + const firstAttempt = await runReplayScriptFile({ + req: baseReq({ positionals: [flowPath], flags: { replayBackend: 'maestro' } }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async () => ({ + ok: false, + error: { code: 'COMMAND_FAILED', message: 'back failed' }, + }), + }); + expect(firstAttempt.ok).toBe(false); + if (firstAttempt.ok) return; + const divergence = firstAttempt.error.details?.divergence as { + resume: { from: number; planDigest: string }; + }; + + sessionStore.setRuntimeHints(sessionName, { + platform: 'ios', + metroHost: '127.0.0.1', + metroPort: 8084, + }); + const resumedAttempt = await runReplayScriptFile({ + req: baseReq({ + positionals: [flowPath], + flags: { + replayBackend: 'maestro', + replayFrom: divergence.resume.from, + replayPlanDigest: divergence.resume.planDigest, + }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async () => { + throw new Error('must not execute after the effective runtime hints changed'); + }, + }); + + expect(resumedAttempt.ok).toBe(false); + if (resumedAttempt.ok) return; + expect(resumedAttempt.error.code).toBe('INVALID_ARGS'); + expect(resumedAttempt.error.message).toMatch(/plan digest/); +}); diff --git a/src/daemon/handlers/__tests__/session-replay-runtime.test.ts b/src/daemon/handlers/__tests__/session-replay-runtime.test.ts index 79fc8e996..d8b030815 100644 --- a/src/daemon/handlers/__tests__/session-replay-runtime.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-runtime.test.ts @@ -44,6 +44,254 @@ test('a successful replay prints one line with the step count and wall time', as expect(data.replayed).toBe(2); expect(data.message).toMatch(/^Replayed 2 steps in \d+\.\ds$/); }); + +test('Maestro YAML uses the typed engine while .ad remains generic', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-typed-maestro-route-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const yamlPath = path.join(root, 'flow.yaml'); + fs.writeFileSync( + yamlPath, + ['appId: com.example.app', '---', '- launchApp', '- inputText: typed'].join('\n'), + ); + const commands: string[] = []; + const invoke = vi.fn(async (request) => { + if (request.command === 'snapshot') { + return { ok: true as const, data: { createdAt: 0, nodes: [] } }; + } + commands.push(request.command); + return { ok: true as const, data: {} }; + }); + + const yamlResponse = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke, + }); + + expect(yamlResponse).toMatchObject({ ok: true, data: { replayed: 2 } }); + expect(commands).toEqual(['open', 'type']); + + commands.length = 0; + const adPath = writeReplayFile(root, ['open "Generic"']); + const adResponse = await runReplayScriptFile({ + req: baseReq({ + positionals: [adPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke, + }); + expect(adResponse).toMatchObject({ ok: true, data: { replayed: 1 } }); + expect(commands).toEqual(['open']); +}); + +test('typed Maestro nested commands receive the runtime hints bound into the plan', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-runtime-envelope-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + sessionStore.setRuntimeHints(sessionName, { + platform: 'ios', + metroHost: '127.0.0.1', + metroPort: 8083, + }); + const yamlPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(yamlPath, 'appId: com.example.app\n---\n- launchApp\n'); + const requests: Parameters[0]['invoke']>[0][] = []; + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke: async (request) => { + requests.push(request); + return { ok: true, data: {} }; + }, + }); + + expect(response.ok).toBe(true); + expect(requests).toHaveLength(1); + expect(requests[0]).toMatchObject({ + command: 'open', + runtime: { + platform: 'ios', + metroHost: '127.0.0.1', + metroPort: 8083, + }, + }); +}); + +test('typed Maestro writes source-aware redacted step timing traces', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-maestro-step-trace-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const yamlPath = path.join(root, 'flow.yaml'); + const tracePath = path.join(root, 'replay-timing.ndjson'); + fs.writeFileSync(yamlPath, 'appId: com.example.app\n---\n- inputText: highly-sensitive\n'); + fs.writeFileSync(tracePath, ''); + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + tracePath, + invoke: async (request) => + request.command === 'snapshot' + ? { ok: true, data: { createdAt: 0, nodes: [] } } + : { ok: true, data: {} }, + }); + + expect(response.ok).toBe(true); + const events = fs + .readFileSync(tracePath, 'utf8') + .trim() + .split('\n') + .map((line) => JSON.parse(line) as Record); + expect(events).toEqual([ + expect.objectContaining({ + type: 'replay_action_start', + replayPath: yamlPath, + line: 3, + step: 1, + command: 'inputText', + positionals: [''], + }), + expect.objectContaining({ + type: 'replay_action_stop', + replayPath: yamlPath, + line: 3, + step: 1, + command: 'inputText', + ok: true, + durationMs: expect.any(Number), + }), + ]); + expect(fs.readFileSync(tracePath, 'utf8')).not.toContain('highly-sensitive'); +}); + +test('replay trace failures do not change action semantics', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-replay-trace-failure-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const yamlPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(yamlPath, 'appId: com.example.app\n---\n- back\n'); + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + tracePath: root, + invoke: async () => ({ ok: true, data: {} }), + }); + + expect(response.ok).toBe(true); +}); + +test('generic replay traces redact typed text', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-replay-trace-redaction-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const secret = 'highly-sensitive-value'; + const filePath = writeReplayFile(root, [`type "${secret}"`]); + const tracePath = path.join(root, 'replay-timing.ndjson'); + fs.writeFileSync(tracePath, ''); + + const response = await runReplayScriptFile({ + req: baseReq({ positionals: [filePath] }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + tracePath, + invoke: async () => ({ ok: true, data: {} }), + }); + + expect(response.ok).toBe(true); + const trace = fs.readFileSync(tracePath, 'utf8'); + expect(trace).not.toContain(secret); + expect(trace).toContain(''); +}); + +test('Maestro YAML rejects .ad repair recording before executing any command', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-typed-maestro-save-script-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + sessionStore.set(sessionName, makeIosSession(sessionName)); + const yamlPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(yamlPath, 'appId: com.example.app\n---\n- launchApp\n'); + const invoke = vi.fn(async () => ({ ok: true as const, data: {} })); + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios', saveScript: true }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke, + }); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.code).toBe('INVALID_ARGS'); + expect(response.error.message).toMatch(/Maestro YAML.*--save-script.*\.ad scripts/); + expect(invoke).not.toHaveBeenCalled(); + expect(sessionStore.get(sessionName)?.recordSession).not.toBe(true); +}); + +test('Maestro YAML cannot append commands to an active .ad repair session', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-typed-maestro-active-repair-')); + const sessionStore = new SessionStore(path.join(root, 'sessions')); + const sessionName = 'default'; + const session = makeIosSession(sessionName); + session.recordSession = true; + session.saveScriptBoundary = 0; + sessionStore.set(sessionName, session); + const yamlPath = path.join(root, 'flow.yaml'); + fs.writeFileSync(yamlPath, 'appId: com.example.app\n---\n- launchApp\n'); + const invoke = vi.fn(async () => ({ ok: true as const, data: {} })); + + const response = await runReplayScriptFile({ + req: baseReq({ + positionals: [yamlPath], + flags: { replayBackend: 'maestro', platform: 'ios' }, + }), + sessionName, + logPath: path.join(root, 'daemon.log'), + sessionStore, + invoke, + }); + + expect(response.ok).toBe(false); + if (response.ok) return; + expect(response.error.code).toBe('INVALID_ARGS'); + expect(response.error.message).toMatch(/active \.ad --save-script repair run/); + expect(invoke).not.toHaveBeenCalled(); +}); test('replay rejects legacy JSON payload files', async () => { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-replay-json-rejected-')); const sessionStore = new SessionStore(path.join(root, 'sessions')); diff --git a/src/daemon/handlers/__tests__/session-replay-vars.test.ts b/src/daemon/handlers/__tests__/session-replay-vars.test.ts index 6116ccdc3..f0324636c 100644 --- a/src/daemon/handlers/__tests__/session-replay-vars.test.ts +++ b/src/daemon/handlers/__tests__/session-replay-vars.test.ts @@ -62,7 +62,8 @@ async function runReplayFixture(params: { for (const [name, contents] of Object.entries(params.files ?? {})) { fs.writeFileSync(path.join(root, name), contents); } - const scriptPath = path.join(root, 'flow.ad'); + const isMaestro = params.flags?.replayBackend === 'maestro'; + const scriptPath = path.join(root, isMaestro ? 'flow.yaml' : 'flow.ad'); fs.writeFileSync(scriptPath, params.script); const calls: CapturedInvocation[] = []; const invoke = async (req: DaemonRequest): Promise => { @@ -73,6 +74,9 @@ async function runReplayFixture(params: { flags: req.flags, }); if (params.invoke) return await params.invoke(req); + if (isMaestro && req.command === 'snapshot') { + return { ok: true, data: { createdAt: 0, nodes: [] } }; + } return { ok: true, data: {} }; }; const response = await runReplayScriptFile({ @@ -81,7 +85,10 @@ async function runReplayFixture(params: { session: 's', command: 'replay', positionals: [scriptPath], - flags: params.flags ?? {}, + flags: { + ...(params.flags ?? {}), + ...(isMaestro && params.flags?.platform === undefined ? { platform: 'ios' } : {}), + }, meta: { cwd: root }, }, sessionName: 's', @@ -245,38 +252,6 @@ test('resolveReplayAction walks runtime hints', () => { assert.equal(resolved.runtime?.metroHost, '10.0.0.1'); }); -test('resolveReplayAction resolves replay control conditions without pre-resolving nested actions', () => { - const action: SessionAction = { - ts: 0, - command: 'runFlow.when', - positionals: ['visible', '${VISIBLE}'], - flags: {}, - replayControl: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: '${VISIBLE}', - actions: [ - { - ts: 0, - command: 'tap', - positionals: ['${TARGET}'], - flags: {}, - }, - ], - }, - }; - const scope = buildReplayVarScope({ - fileEnv: { VISIBLE: 'Feed', TARGET: '${NEXT}', NEXT: 'Done' }, - }); - const resolved = resolveReplayAction(action, scope, LOC); - assert.equal(resolved.replayControl?.kind, 'maestroRunFlowWhen'); - if (resolved.replayControl?.kind !== 'maestroRunFlowWhen') { - throw new Error('expected runFlow.when control'); - } - assert.equal(resolved.replayControl.selector, 'Feed'); - assert.deepEqual(resolved.replayControl.actions[0]?.positionals, ['${TARGET}']); -}); - test('parseReplayScriptDetailed tracks line numbers', () => { const script = [ '# comment', @@ -647,6 +622,7 @@ test('runReplayScriptFile applies CLI env overrides before Maestro compat mappin ].join('\n'), flags: { replayBackend: 'maestro', + platform: 'android', replayShellEnv: { AD_VAR_BUTTON_ID: 'shell-button' }, replayEnv: ['APP_ID=cli-app'], }, @@ -706,7 +682,11 @@ output.result = SERVER_PATH + ':' + json(res.body).appviewDid assert.equal(response.ok, true); assert.deepEqual( calls.map((call) => [call.command, call.positionals]), - [['type', ['local:did:plc:test']]], + [ + ['type', ['local:did:plc:test']], + ['snapshot', []], + ['snapshot', []], + ], ); }); @@ -759,7 +739,11 @@ output.result = parsed.method + ':' + json(parsed.body).ok assert.equal(response.ok, true); assert.deepEqual( calls.map((call) => [call.command, call.positionals]), - [['type', ['POST:true']]], + [ + ['type', ['POST:true']], + ['snapshot', []], + ['snapshot', []], + ], ); } finally { server.child.kill(); @@ -794,7 +778,11 @@ output.result = [ assert.equal(response.ok, true); assert.deepEqual( calls.map((call) => [call.command, call.positionals]), - [['type', ['false:false:false:2']]], + [ + ['type', ['false:false:false:2']], + ['snapshot', []], + ['snapshot', []], + ], ); }); @@ -850,7 +838,7 @@ test('runReplayScriptFile reports iOS Maestro openLink setup failures before ass assert.equal(response.ok, false); if (!response.ok) { assert.match(response.error.message, /Replay failed at step 1/); - assert.match(response.error.message, /open "demo\.app" "demo:\/\/screen"/); + assert.match(response.error.message, /openLink "demo:\/\/screen"/); assert.match(response.error.message, /Developer mode is disabled/); // The cause's details-borne hint is hoisted onto the error field by the // divergence transport (arbitrary cause details are stripped). @@ -879,7 +867,7 @@ test('runReplayScriptFile explains empty Maestro runScript JSON bodies', async ( if (!response.ok) { assert.match(response.error.message, /Replay failed at step 1/); assert.match(response.error.message, /json\(\) received an empty body/); - assert.match(response.error.message, /setup server output/); + assert.match(response.error.hint ?? '', /setup server output/); } assert.equal(calls.length, 0); }); @@ -906,7 +894,7 @@ test('runReplayScriptFile rejects Maestro runScript output keys containing dots' test('runReplayScriptFile retries Maestro scrollUntilVisible with scroll probes', async () => { const calls: CapturedInvocation[] = []; - let waitAttempts = 0; + let snapshotAttempts = 0; const { response } = await runReplayFixture({ label: 'maestro-scroll-until-visible', script: [ @@ -919,58 +907,23 @@ test('runReplayScriptFile retries Maestro scrollUntilVisible with scroll probes' '', ].join('\n'), flags: { replayBackend: 'maestro' }, - invoke: async (req) => { - calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); - if (req.command === 'scroll') return { ok: true, data: {} }; - if (req.command === 'find') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'find wait timed out' }, - }; - } - waitAttempts += 1; - if (waitAttempts === 3) return { ok: true, data: { waitedMs: 1100 } }; - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'wait timed out' }, - }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - calls.map((call) => [call.command, call.positionals]), - [ - ['wait', ['label="Discover" || text="Discover" || id="Discover"', '500']], - ['find', ['Discover', 'wait', '500']], - ['scroll', ['up']], - ['wait', ['label="Discover" || text="Discover" || id="Discover"', '500']], - ['find', ['Discover', 'wait', '500']], - ['scroll', ['up']], - ['wait', ['label="Discover" || text="Discover" || id="Discover"', '200']], - ], - ); -}); - -test('runReplayScriptFile lets Maestro tapOn use fuzzy visible text matching', async () => { - const calls: CapturedInvocation[] = []; - const { response } = await runReplayFixture({ - label: 'maestro-tap-visible-text-fuzzy', - script: ['appId: demo.app', '---', '- tapOn: Discover', ''].join('\n'), - flags: { replayBackend: 'maestro' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { + snapshotAttempts += 1; return { ok: true, data: { - nodes: [ - { - index: 1, - label: 'Discover people', - rect: { x: 10, y: 600, width: 240, height: 44 }, - }, - ], + nodes: + snapshotAttempts < 3 + ? [] + : [ + { + index: 1, + label: 'Discover', + rect: { x: 10, y: 600, width: 240, height: 44 }, + }, + ], }, }; } @@ -983,10 +936,12 @@ test('runReplayScriptFile lets Maestro tapOn use fuzzy visible text matching', a calls.map((call) => [call.command, call.positionals]), [ ['snapshot', []], - ['click', ['130', '622']], + ['scroll', ['up']], + ['snapshot', []], + ['scroll', ['up']], + ['snapshot', []], ], ); - assert.equal(calls[0]?.flags?.noRecord, true); }); test('runReplayScriptFile promotes Maestro text tapOn to an actionable ancestor', async () => { @@ -1079,7 +1034,7 @@ test('runReplayScriptFile promotes Maestro id tapOn to an actionable ancestor', ); }); -test('runReplayScriptFile captures a fresh Maestro snapshot for tapOn after assertVisible', async () => { +test('runReplayScriptFile reuses the exact observation snapshot for tapOn after assertVisible', async () => { let snapshots = 0; const { response, calls } = await runReplayFixture({ label: 'maestro-assert-visible-tap-fresh-snapshot', @@ -1098,34 +1053,19 @@ test('runReplayScriptFile captures a fresh Maestro snapshot for tapOn after asse return { ok: true, data: { - nodes: - snapshots === 1 - ? [ - { - index: 1, - label: 'Article', - rect: { x: 10, y: 100, width: 160, height: 44 }, - }, - { - index: 2, - label: 'Open feed', - identifier: 'open-feed', - rect: { x: 20, y: 180, width: 180, height: 48 }, - }, - ] - : [ - { - index: 1, - label: 'AppStack.tsx (42:7)', - rect: { x: 28, y: 1304, width: 1025, height: 44 }, - }, - { - index: 2, - label: 'Open feed', - identifier: 'open-feed', - rect: { x: 40, y: 240, width: 200, height: 48 }, - }, - ], + nodes: [ + { + index: 1, + label: 'Article', + rect: { x: 10, y: 100, width: 160, height: 44 }, + }, + { + index: 2, + label: 'Open feed', + identifier: 'open-feed', + rect: { x: 20, y: 180, width: 180, height: 48 }, + }, + ], }, }; } @@ -1134,13 +1074,12 @@ test('runReplayScriptFile captures a fresh Maestro snapshot for tapOn after asse }); assert.equal(response.ok, true); - assert.equal(snapshots, 2); + assert.equal(snapshots, 1); assert.deepEqual( calls.map((call) => [call.command, call.positionals]), [ ['snapshot', []], - ['snapshot', []], - ['click', ['140', '264']], + ['click', ['110', '204']], ], ); }); @@ -1153,9 +1092,6 @@ test('runReplayScriptFile scopes duplicate tap targets after native Maestro asse ), flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { - if (req.command === 'wait') { - return { ok: true, data: { matched: true } }; - } if (req.command === 'snapshot') { return { ok: true, @@ -1209,8 +1145,6 @@ test('runReplayScriptFile scopes duplicate tap targets after native Maestro asse assert.deepEqual( calls.map((call) => [call.command, call.positionals]), [ - ['wait', ['Albums', '17000']], - ['snapshot', []], ['snapshot', []], ['click', ['112', '242']], ], @@ -1238,10 +1172,7 @@ test('runReplayScriptFile treats absent Maestro assertNotVisible targets as pass assert.equal(response.ok, true); assert.deepEqual( calls.map((call) => [call.command, call.positionals]), - [ - ['snapshot', []], - ['snapshot', []], - ], + [['snapshot', []]], ); assert.equal(calls[0]?.flags?.noRecord, true); }); @@ -1305,7 +1236,7 @@ test('runReplayScriptFile waits briefly for Maestro assertNotVisible to stabiliz }); assert.equal(response.ok, true); - assert.equal(calls.length, 3); + assert.equal(calls.length, 2); }); test('runReplayScriptFile treats absent Maestro extendedWaitUntil.notVisible targets as passing', async () => { @@ -1337,123 +1268,6 @@ test('runReplayScriptFile treats absent Maestro extendedWaitUntil.notVisible tar assert.equal(calls[0]?.flags?.noRecord, true); }); -test('runReplayScriptFile treats passed loading extendedWaitUntil as success', async () => { - const { response } = await runReplayFixture({ - label: 'maestro-extended-wait-loading-already-past', - script: [ - 'appId: demo.app', - '---', - '- extendedWaitUntil:', - ' visible: Loading…', - ' timeout: 1', - '', - ].join('\n'), - flags: { replayBackend: 'maestro' }, - invoke: async () => ({ - ok: true, - data: { - createdAt: 1, - nodes: [ - { - index: 1, - label: 'Suspend', - type: 'Button', - rect: { x: 16, y: 120, width: 120, height: 48 }, - visibleToUser: true, - }, - ], - }, - }), - }); - - assert.equal(response.ok, true); -}); - -test('runReplayScriptFile retries Maestro fuzzy tapOn without raw selector fallback', async () => { - const calls: CapturedInvocation[] = []; - let snapshotAttempts = 0; - const { response } = await runReplayFixture({ - label: 'maestro-tap-visible-text-fuzzy-retry', - script: ['appId: demo.app', '---', '- tapOn: Discover', ''].join('\n'), - flags: { replayBackend: 'maestro' }, - invoke: async (req) => { - calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); - if (req.command === 'snapshot') { - snapshotAttempts += 1; - return { - ok: true, - data: { - nodes: - snapshotAttempts === 1 - ? [] - : [ - { - index: 1, - label: 'Discover people', - rect: { x: 10, y: 600, width: 240, height: 44 }, - }, - ], - }, - }; - } - return { ok: true, data: {} }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - calls.map((call) => [call.command, call.positionals]), - [ - ['snapshot', []], - ['snapshot', []], - ['click', ['130', '622']], - ], - ); -}); - -test('runReplayScriptFile lets optional Maestro fuzzy tapOn click first visible match', async () => { - const calls: CapturedInvocation[] = []; - const { response } = await runReplayFixture({ - label: 'maestro-tap-visible-text-optional-first-match', - script: [ - 'appId: demo.app', - '---', - '- tapOn:', - ' text: Later', - ' optional: true', - '', - ].join('\n'), - flags: { replayBackend: 'maestro' }, - invoke: async (req) => { - calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); - if (req.command === 'snapshot') { - return { - ok: true, - data: { - nodes: [ - { - index: 1, - label: 'Maybe Later', - rect: { x: 100, y: 700, width: 240, height: 44 }, - }, - ], - }, - }; - } - return { ok: true, data: {} }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - calls.map((call) => [call.command, call.positionals]), - [ - ['snapshot', []], - ['click', ['220', '722']], - ], - ); -}); - test('runReplayScriptFile resolves Maestro percentage point taps from snapshot size', async () => { const calls: CapturedInvocation[] = []; const { response } = await runReplayFixture({ @@ -1497,7 +1311,7 @@ test('runReplayScriptFile retries Maestro id tapOn through snapshot coordinates' const { response } = await runReplayFixture({ label: 'maestro-tap-on-retry', script: ['appId: demo.app', '---', '- tapOn:', ' id: delayedButton', ''].join('\n'), - flags: { replayBackend: 'maestro' }, + flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { @@ -1516,10 +1330,7 @@ test('runReplayScriptFile retries Maestro id tapOn through snapshot coordinates' }, }; } - return { - ok: false, - error: { code: 'ELEMENT_NOT_FOUND', message: 'element not found' }, - }; + return { ok: true, data: { nodes: [] } }; } if (req.command === 'click') return { ok: true, data: {} }; return { @@ -1615,7 +1426,7 @@ test('runReplayScriptFile lets snapshot id tap handle Maestro one-point edge con const { response } = await runReplayFixture({ label: 'maestro-tap-edge-rect', script: ['appId: demo.app', '---', '- tapOn:', ' id: hiddenTestLogin', ''].join('\n'), - flags: { replayBackend: 'maestro' }, + flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { @@ -1646,7 +1457,7 @@ test('runReplayScriptFile lets snapshot id tap handle Maestro one-point edge con ); }); -test('runReplayScriptFile coalesces Maestro text-entry tapOn into native fill', async () => { +test('runReplayScriptFile resolves a text-entry target once before typing', async () => { const calls: CapturedInvocation[] = []; const { response } = await runReplayFixture({ label: 'maestro-tap-input-text-snapshot', @@ -1659,7 +1470,7 @@ test('runReplayScriptFile coalesces Maestro text-entry tapOn into native fill', '- pressKey: Enter', '', ].join('\n'), - flags: { replayBackend: 'maestro' }, + flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { @@ -1684,12 +1495,14 @@ test('runReplayScriptFile coalesces Maestro text-entry tapOn into native fill', assert.deepEqual( calls.map((call) => [call.command, call.positionals]), [ - ['wait', ['id="editableNameInput"', '30000']], - ['fill', ['id="editableNameInput"', 'Saved list']], + ['snapshot', []], + ['click', ['120', '120']], + ['type', ['Saved list']], + ['snapshot', []], + ['snapshot', []], ['keyboard', ['enter']], ], ); - assert.equal(calls[1]?.flags?.maestro?.allowNonHittableCoordinateFallback, true); }); test('runReplayScriptFile resolves Maestro swipe.label from a labeled element rect', async () => { @@ -1987,55 +1800,6 @@ test('runReplayScriptFile falls back to newline type when keyboard enter is unsu ); }); -test('runReplayScriptFile skips Maestro runFlow.when.visible commands when absent', async () => { - const calls: CapturedInvocation[] = []; - const { response } = await runReplayFixture({ - label: 'maestro-run-flow-when-visible-skip', - script: [ - 'appId: demo.app', - '---', - '- runFlow:', - ' when:', - ' visible: Continue', - ' commands:', - ' - tapOn: Continue', - '', - ].join('\n'), - flags: { replayBackend: 'maestro' }, - invoke: async (req) => { - calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); - if (req.command === 'snapshot') { - return { - ok: true, - data: { - nodes: [ - { - index: 0, - type: 'application', - rect: { x: 0, y: 0, width: 390, height: 844 }, - }, - ], - }, - }; - } - return { - ok: false, - error: { - code: 'COMMAND_FAILED', - message: 'not visible', - details: { command: 'is', reason: 'selector_not_found' }, - }, - }; - }, - }); - - assert.equal(response.ok, true); - assert.deepEqual( - calls.map((call) => [call.command, call.positionals]), - Array.from({ length: 13 }, () => ['snapshot', []]), - ); -}); - test('runReplayScriptFile retries Maestro retry commands until they pass', async () => { const calls: CapturedInvocation[] = []; let openAttempts = 0; @@ -2092,8 +1856,8 @@ test('runReplayScriptFile retries Maestro retry commands until they pass', async assert.deepEqual( calls.filter((call) => call.command === 'open').map((call) => [call.command, call.positionals]), [ - ['open', ['demo://details']], - ['open', ['demo://details']], + ['open', ['demo.app', 'demo://details']], + ['open', ['demo.app', 'demo://details']], ], ); assert.equal(calls.filter((call) => call.command === 'snapshot').length > 1, true); @@ -2126,10 +1890,11 @@ test('runReplayScriptFile propagates Maestro runFlow.when runtime errors', async assert.equal(response.error.code, 'REPLAY_DIVERGENCE'); assert.match(response.error.message, /fetch failed/); const divergence = response.error.details?.divergence as - | { cause: { code: string; message: string } } + | { cause: { code: string; message: string }; repairHint: string } | undefined; assert.equal(divergence?.cause.code, 'UNKNOWN'); assert.match(divergence?.cause.message ?? '', /fetch failed/); + assert.equal(divergence?.repairHint, 'manual'); } }); @@ -2147,7 +1912,7 @@ test('runReplayScriptFile runs Maestro runFlow.when.visible commands when presen ' - tapOn: Continue', '', ].join('\n'), - flags: { replayBackend: 'maestro' }, + flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { @@ -2172,12 +1937,6 @@ test('runReplayScriptFile runs Maestro runFlow.when.visible commands when presen }, }; } - if (req.command === 'click') { - return { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'Selector did not match' }, - }; - } return { ok: true, data: {} }; }, }); @@ -2186,10 +1945,8 @@ test('runReplayScriptFile runs Maestro runFlow.when.visible commands when presen assert.deepEqual( calls.map((call) => [call.command, call.positionals]), [ - ['snapshot', []], ['snapshot', []], ['click', ['76', '122']], - ['find', ['Continue', 'click']], ], ); assert.equal( @@ -2200,15 +1957,11 @@ test('runReplayScriptFile runs Maestro runFlow.when.visible commands when presen calls.find((call) => call.command === 'click')?.flags?.postGestureStabilization, true, ); - assert.equal(calls.find((call) => call.command === 'find')?.flags?.interactionOutcome, undefined); - assert.equal( - calls.find((call) => call.command === 'find')?.flags?.postGestureStabilization, - true, - ); }); test('runReplayScriptFile runs nested Maestro runtime commands inside runFlow.when', async () => { const calls: CapturedInvocation[] = []; + let snapshots = 0; const { response } = await runReplayFixture({ label: 'maestro-run-flow-when-nested-runtime', script: [ @@ -2228,6 +1981,7 @@ test('runReplayScriptFile runs nested Maestro runtime commands inside runFlow.wh invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { + snapshots += 1; return { ok: true, data: { @@ -2245,11 +1999,22 @@ test('runReplayScriptFile runs nested Maestro runtime commands inside runFlow.wh label: 'Feed', rect: { x: 16, y: 100, width: 120, height: 24 }, }, + ...(snapshots < 3 + ? [] + : [ + { + index: 2, + depth: 1, + parentIndex: 0, + type: 'statictext', + label: 'Done', + rect: { x: 16, y: 300, width: 120, height: 24 }, + }, + ]), ], }, }; } - if (req.command === 'wait') return { ok: true, data: { found: true } }; return { ok: true, data: {} }; }, }); @@ -2259,7 +2024,9 @@ test('runReplayScriptFile runs nested Maestro runtime commands inside runFlow.wh calls.map((call) => [call.command, call.positionals]), [ ['snapshot', []], - ['wait', ['label="Done" || text="Done" || id="Done"', '500']], + ['snapshot', []], + ['scroll', ['down']], + ['snapshot', []], ], ); }); @@ -2282,7 +2049,7 @@ test('runReplayScriptFile resolves nested Maestro runFlow.when command variables ' - tapOn: ${TARGET_LABEL}', '', ].join('\n'), - flags: { replayBackend: 'maestro' }, + flags: { replayBackend: 'maestro', platform: 'android' }, invoke: async (req) => { calls.push({ command: req.command, positionals: req.positionals, flags: req.flags }); if (req.command === 'snapshot') { @@ -2308,7 +2075,7 @@ test('runReplayScriptFile resolves nested Maestro runFlow.when command variables depth: 1, parentIndex: 0, type: 'button', - label: '${FINAL_LABEL}', + label: 'Done', rect: { x: 100, y: 300, width: 80, height: 40 }, }, ], @@ -2323,7 +2090,6 @@ test('runReplayScriptFile resolves nested Maestro runFlow.when command variables assert.deepEqual( calls.map((call) => [call.command, call.positionals]), [ - ['snapshot', []], ['snapshot', []], ['click', ['140', '320']], ], diff --git a/src/daemon/handlers/__tests__/session-test-fail-fast.test.ts b/src/daemon/handlers/__tests__/session-test-fail-fast.test.ts new file mode 100644 index 000000000..0a438bb83 --- /dev/null +++ b/src/daemon/handlers/__tests__/session-test-fail-fast.test.ts @@ -0,0 +1,39 @@ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { expect, test } from 'vitest'; +import { makeSessionStore } from '../../../__tests__/test-utils/store-factory.ts'; +import { handleSessionCommands } from '../session.ts'; + +test('test --fail-fast continues after passing scripts', async () => { + const root = fs.mkdtempSync(path.join(os.tmpdir(), 'agent-device-test-fail-fast-pass-')); + fs.writeFileSync(path.join(root, '01-first.ad'), 'context platform=ios\nopen "Demo"\n'); + fs.writeFileSync(path.join(root, '02-second.ad'), 'context platform=ios\nopen "Demo"\n'); + + const invokedPaths: string[] = []; + const response = await handleSessionCommands({ + req: { + token: 't', + session: 'default', + command: 'test', + positionals: [root], + meta: { cwd: root, requestId: 'suite-fail-fast-pass' }, + flags: { failFast: true }, + }, + sessionName: 'default', + logPath: path.join(os.tmpdir(), 'daemon.log'), + sessionStore: makeSessionStore('agent-device-test-fail-fast-pass-store-'), + invoke: async (request) => { + invokedPaths.push(String(request.positionals?.[0])); + return { ok: true, data: {} }; + }, + }); + + expect(response?.ok).toBe(true); + if (!response?.ok) throw new Error('Expected successful daemon response.'); + expect(invokedPaths).toHaveLength(2); + expect(response.data?.total).toBe(2); + expect(response.data?.executed).toBe(2); + expect(response.data?.passed).toBe(2); + expect(response.data?.notRun).toBe(0); +}); diff --git a/src/daemon/handlers/__tests__/snapshot-handler.test.ts b/src/daemon/handlers/__tests__/snapshot-handler.test.ts index 5196a88ff..7c39b93cb 100644 --- a/src/daemon/handlers/__tests__/snapshot-handler.test.ts +++ b/src/daemon/handlers/__tests__/snapshot-handler.test.ts @@ -154,10 +154,10 @@ function mockAndroidTimeoutEvidenceDispatch(): void { function androidSnapshotTimeoutError(): AppError { return new AppError( 'COMMAND_FAILED', - 'Android UI hierarchy dump timed out while waiting for the UI to become idle.', + 'Android snapshot helper timed out while waiting for the UI to become idle.', { cmd: 'adb', - args: ['exec-out', 'uiautomator', 'dump', '/dev/tty'], + args: ['shell', 'am', 'instrument'], timeoutMs: 8000, hint: 'Android accessibility snapshots can be blocked by busy or continuously changing app UI. Use screenshot as visual truth after this timeout.', }, @@ -169,7 +169,7 @@ function expectAndroidTimeoutEvidence( ) { if (!response) throw new Error('Expected snapshot response'); if (response.ok) throw new Error('Expected snapshot timeout failure'); - expect(response.error.message).toMatch(/UI hierarchy dump timed out/i); + expect(response.error.message).toMatch(/snapshot helper timed out/i); expect(response.error.hint).toMatch(/Use screenshot as visual truth/i); assertAndroidTimeoutEvidencePayload(response.error.details?.androidSnapshotTimeoutScreenshot); } diff --git a/src/daemon/handlers/interaction-gesture.ts b/src/daemon/handlers/interaction-gesture.ts index e057f8e73..33860df75 100644 --- a/src/daemon/handlers/interaction-gesture.ts +++ b/src/daemon/handlers/interaction-gesture.ts @@ -47,7 +47,7 @@ export async function dispatchGestureViaRuntime( const input = readGesturePayload(params.req.input); const normalized = normalizePublicGesture(input); requireGestureSupported(normalized.gesture, session.device); - const result = await createInteractionRuntime(params).interactions.gesture({ + const result = await createGestureRuntime(params).interactions.gesture({ session: params.sessionName, requestId: params.req.meta?.requestId, gesture: normalized.gesture, @@ -70,7 +70,7 @@ export async function dispatchSwipeViaRuntime( const count = input.count ?? 1; const pauseMs = input.pauseMs ?? 0; const pattern = input.pattern ?? 'one-way'; - const runtime = createInteractionRuntime(params); + const runtime = createGestureRuntime(params); const result = await runSwipeRepetitions(runtime, params, input, count, pauseMs, pattern); return { positionals: swipeReplayPositionals(input), @@ -92,6 +92,13 @@ export async function dispatchSwipeViaRuntime( }); } +function createGestureRuntime(params: GestureHandlerParams) { + return createInteractionRuntime({ + ...params, + pairedGestureViewport: params.req.internal?.gestureViewport, + }); +} + async function dispatchGestureInteraction( params: GestureHandlerParams, command: 'gesture' | 'swipe', diff --git a/src/daemon/handlers/interaction-runtime.ts b/src/daemon/handlers/interaction-runtime.ts index 50d83044f..84a8c07cd 100644 --- a/src/daemon/handlers/interaction-runtime.ts +++ b/src/daemon/handlers/interaction-runtime.ts @@ -20,12 +20,14 @@ import { createDaemonRuntimeSessionStore } from '../runtime-session.ts'; import { resolveWebProvider, type WebProvider } from '../../platforms/web/provider.ts'; import { stripAtPrefix } from './interaction-touch-targets.ts'; import { NO_ACTIVE_SESSION_MESSAGE } from './response.ts'; +import type { Rect } from '../../kernel/snapshot.ts'; -export function createInteractionRuntime( - params: InteractionHandlerParams & { - captureSnapshotForSession: CaptureSnapshotForSession; - }, -) { +type InteractionRuntimeParams = InteractionHandlerParams & { + captureSnapshotForSession: CaptureSnapshotForSession; + pairedGestureViewport?: Rect; +}; + +export function createInteractionRuntime(params: InteractionRuntimeParams) { const session = params.sessionStore.get(params.sessionName); if (!session) throw new AppError('SESSION_NOT_FOUND', NO_ACTIVE_SESSION_MESSAGE); return createAgentDevice({ @@ -45,9 +47,7 @@ export function createInteractionRuntime( } function createInteractionBackend( - params: InteractionHandlerParams & { session: SessionState } & { - captureSnapshotForSession: CaptureSnapshotForSession; - }, + params: InteractionRuntimeParams & { session: SessionState }, ): AgentDeviceBackend { const { req, session } = params; const webProvider = resolveNativeWebInteractionProvider(session); @@ -66,10 +66,11 @@ function createInteractionBackend( ), }), resolveGestureViewport: async () => - await dispatchGestureViewport( + params.pairedGestureViewport ?? + (await dispatchGestureViewport( session.device, params.contextFromFlags(req.flags, session.appBundleId, session.trace?.outPath), - ), + )), tap: async (_context, point): Promise => toBackendActionResult( await dispatchCommand( diff --git a/src/daemon/handlers/session-replay-action-runtime.ts b/src/daemon/handlers/session-replay-action-runtime.ts index 8a699fa30..c541676d2 100644 --- a/src/daemon/handlers/session-replay-action-runtime.ts +++ b/src/daemon/handlers/session-replay-action-runtime.ts @@ -1,28 +1,17 @@ -import fs from 'node:fs'; import type { CommandFlags } from '../../core/dispatch.ts'; -import { - mergeReplayVarScopeValues, - resolveReplayAction, - type ReplayVarScope, -} from '../../replay/vars.ts'; +import { resolveReplayAction, type ReplayVarScope } from '../../replay/vars.ts'; import type { DaemonInvokeFn, DaemonRequest, DaemonResponse, SessionAction } from '../types.ts'; import { mergeParentFlags } from '../../core/batch.ts'; -import { invokeMaestroRuntimeCommand } from '../../compat/maestro/runtime.ts'; -import { invokeMaestroRunFlowWhenControl } from '../../compat/maestro/runtime-flow.ts'; -import { - invokeReplayRetryBlock, - type ReplayActionBlockInvoker, -} from '../../replay/control-flow-runtime.ts'; import { AppError, normalizeError } from '../../kernel/errors.ts'; import { gesturePayloadFromPositionals, swipePayloadFromPositionals, } from '../../contracts/gesture-normalization.ts'; +import { buildDisplayPositionals } from '../session-event-action.ts'; +import { appendReplayTraceEvent } from './session-replay-trace.ts'; type ReplayBaseRequest = Omit; -type ReplayActionInvoker = ReplayActionBlockInvoker; - export async function invokeReplayAction(params: { req: DaemonRequest; sessionName: string; @@ -39,20 +28,6 @@ export async function invokeReplayAction(params: { const { req, sessionName, action, scope, filePath, line, step, sourcePath, tracePath, invoke } = params; const resolved = resolveReplayAction(action, scope, { file: sourcePath ?? filePath, line }); - const invokeNestedReplayAction: ReplayActionInvoker = (nested) => - invokeReplayAction({ - req, - sessionName, - action: nested.action, - scope, - filePath, - line: nested.line, - step: nested.step, - // No recorded source on a nested action = same file as its wrapper. - sourcePath: nested.sourcePath ?? sourcePath, - tracePath, - invoke, - }); const startedAt = Date.now(); appendReplayTraceEvent(tracePath, { type: 'replay_action_start', @@ -62,18 +37,13 @@ export async function invokeReplayAction(params: { line, step, command: resolved.command, - positionals: resolved.positionals ?? [], + positionals: buildDisplayPositionals(resolved) ?? [], }); // A raw dispatch failure (e.g. a selector-miss during press/click/fill/ - // longpress) can THROW an AppError instead of resolving to `{ok:false}` — - // every caller reachable from here (the top-level replay loop, retry - // blocks, and nested Maestro runFlow invocations) only ever branches on - // `response.ok`, so this is the single place every replay action dispatch - // funnels through, regardless of nesting depth. Normalizing here means the - // top-level loop's existing `if (!response.ok)` divergence wrapping (and - // retry/control-flow's own `response.ok` checks) apply uniformly instead of - // the throw escaping unwrapped to the outer catch. + // longpress) can throw an AppError instead of resolving to `{ok:false}`. + // Normalize it at the generic replay dispatch boundary so the top-level + // loop's existing divergence wrapping applies uniformly. let response: DaemonResponse; try { response = await invokeResolvedReplayAction({ @@ -84,7 +54,6 @@ export async function invokeReplayAction(params: { line, step, invoke, - invokeReplayAction: invokeNestedReplayAction, }); } catch (dispatchErr) { // Only an expected AppError dispatch failure (e.g. a selector-miss) gets @@ -142,9 +111,8 @@ async function invokeResolvedReplayAction(params: { line: number; step: number; invoke: DaemonInvokeFn; - invokeReplayAction: ReplayActionInvoker; }): Promise { - const { req, sessionName, resolved, scope, line, step, invoke, invokeReplayAction } = params; + const { req, sessionName, resolved, invoke } = params; const flags = buildReplayActionFlags(req.flags, resolved.flags); const baseReq: ReplayBaseRequest = { token: req.token, @@ -154,31 +122,7 @@ async function invokeResolvedReplayAction(params: { meta: req.meta, internal: req.internal, }; - const response = - (await invokeReplayControl({ - control: resolved.replayControl, - baseReq, - line, - step, - invoke, - invokeReplayAction, - })) ?? - (await invokeMaestroRuntimeCommand({ - command: resolved.command, - baseReq, - positionals: resolved.positionals ?? [], - scope, - line, - step, - invoke, - invokeReplayAction, - })) ?? - (await invoke(buildReplayInteractionRequest(baseReq, resolved))); - if (response.ok) { - const outputEnv = readReplayOutputEnv(response.data); - if (outputEnv) mergeReplayVarScopeValues(scope, outputEnv); - } - return response; + return await invoke(buildReplayInteractionRequest(baseReq, resolved)); } function buildReplayInteractionRequest( @@ -209,50 +153,6 @@ function buildReplayInteractionRequest( return { ...baseReq, command: action.command, positionals }; } -async function invokeReplayControl(params: { - control: SessionAction['replayControl'] | undefined; - baseReq: ReplayBaseRequest; - line: number; - step: number; - invoke: DaemonInvokeFn; - invokeReplayAction: ReplayActionInvoker; -}): Promise { - const { control, baseReq, line, step, invoke, invokeReplayAction } = params; - if (!control) return undefined; - switch (control.kind) { - case 'retry': - return await invokeReplayRetryBlock({ - actions: control.actions, - actionSources: control.actionSources, - maxRetries: control.maxRetries, - line, - step, - invokeReplayAction, - }); - case 'maestroRunFlowWhen': - return await invokeMaestroRunFlowWhenControl({ - baseReq, - control, - line, - step, - invoke, - invokeReplayAction, - }); - } - const _exhaustive: never = control; - return _exhaustive; -} - -function readReplayOutputEnv(data: unknown): Record | null { - if (!data || typeof data !== 'object') return null; - const raw = (data as { outputEnv?: unknown }).outputEnv; - if (!raw || typeof raw !== 'object' || Array.isArray(raw)) return null; - const entries = Object.entries(raw).filter( - (entry): entry is [string, string] => typeof entry[1] === 'string', - ); - return entries.length > 0 ? Object.fromEntries(entries) : null; -} - function readResponseTiming(data: unknown): Record | undefined { if (!data || typeof data !== 'object' || Array.isArray(data)) return undefined; const timing = (data as { timing?: unknown }).timing; @@ -265,14 +165,6 @@ function readResponseTiming(data: unknown): Record | undefined ); } -function appendReplayTraceEvent( - tracePath: string | undefined, - event: Record, -): void { - if (!tracePath) return; - fs.appendFileSync(tracePath, `${JSON.stringify(event)}\n`); -} - function buildReplayActionFlags( parentFlags: CommandFlags | undefined, actionFlags: SessionAction['flags'] | undefined, diff --git a/src/daemon/handlers/session-replay-divergence.ts b/src/daemon/handlers/session-replay-divergence.ts index c9c076d5d..c01675456 100644 --- a/src/daemon/handlers/session-replay-divergence.ts +++ b/src/daemon/handlers/session-replay-divergence.ts @@ -324,7 +324,7 @@ function classifySuggestionBasis(selector: Selector): ReplayDivergenceSuggestion * then document order; the same-scrollRegion tier awaits decision 3's * recorded evidence (migration step 4). */ -function collectReplayDivergenceSuggestions(params: { +export function collectReplayDivergenceSuggestions(params: { action: SessionAction; session: SessionState; nodes: SnapshotNode[]; @@ -411,27 +411,43 @@ function resolveSuggestionCandidate(params: { disambiguateAmbiguous: matching.allowDisambiguation, }); if (!resolved) return undefined; - - const selectorChain = buildSelectorChainForNode(resolved.node, session.device.platform, { - action: - action.command === 'fill' ? 'fill' : isTouchTargetCommand(action.command) ? 'click' : 'get', - }); const basis = classifySuggestionBasis(resolved.selector); - const role = formatRole(resolved.node.type ?? 'Element'); - const label = displayLabel(resolved.node, role); return { - suggestion: { - selector: sanitize(selectorChain.join(' || ')), + suggestion: buildReplayDivergenceSuggestionForNode({ + node: resolved.node, + session, + action, basis, - ...(resolved.node.ref ? { ref: resolved.node.ref } : {}), - role: sanitize(role), - ...(label ? { label: sanitize(label) } : {}), - }, + sanitize, + }), basisRank: BASIS_RANK[basis], nodeIndex: resolved.node.index, }; } +export function buildReplayDivergenceSuggestionForNode(params: { + node: SnapshotNode; + session: SessionState; + action: SessionAction; + basis: ReplayDivergenceSuggestionBasis; + sanitize: DivergenceFieldSanitizer; +}): ReplayDivergenceSuggestion { + const { node, session, action, basis, sanitize } = params; + const selectorChain = buildSelectorChainForNode(node, session.device.platform, { + action: + action.command === 'fill' ? 'fill' : isTouchTargetCommand(action.command) ? 'click' : 'get', + }); + const role = formatRole(node.type ?? 'Element'); + const label = displayLabel(node, role); + return { + selector: sanitize(selectorChain.join(' || ')), + basis, + ...(node.ref ? { ref: node.ref } : {}), + role: sanitize(role), + ...(label ? { label: sanitize(label) } : {}), + }; +} + function writeReplayDivergenceArtifact( sessionStore: SessionStore, sessionName: string, diff --git a/src/daemon/handlers/session-replay-maestro-failure.ts b/src/daemon/handlers/session-replay-maestro-failure.ts new file mode 100644 index 000000000..6355fa8e3 --- /dev/null +++ b/src/daemon/handlers/session-replay-maestro-failure.ts @@ -0,0 +1,314 @@ +import type { MaestroEngineEvent } from '../../compat/maestro/engine-types.ts'; +import { formatMaestroCommandProgress } from '../../compat/maestro/progress.ts'; +import type { MaestroCommand, MaestroSelector } from '../../compat/maestro/program-ir.ts'; +import { evaluateMaestroReplayResume } from '../../compat/maestro/replay-plan.ts'; +import type { MaestroReplayPlan } from '../../compat/maestro/replay-plan-types.ts'; +import { resolveMaestroTargetFromSnapshot } from '../../compat/maestro/runtime-targets.ts'; +import { getSnapshotReferenceFrame } from '../touch-reference-frame.ts'; +import type { DaemonError } from '../../kernel/contracts.ts'; +import type { SnapshotNode } from '../../kernel/snapshot.ts'; +import { + REPLAY_DIVERGENCE_SUGGESTION_LIMIT, + createReplayDivergenceSanitizer, + type ReplayDivergence, + type ReplayVarScrubEntry, +} from '../../replay/divergence.ts'; +import { formatScriptArg } from '../../replay/script-utils.ts'; +import type { SnapshotDiagnosticsSummary } from '../../snapshot-diagnostics.ts'; +import { SessionStore } from '../session-store.ts'; +import type { DaemonRequest, DaemonResponse, SessionAction, SessionState } from '../types.ts'; +import { + boundReplayDivergenceForSession, + buildReplayDivergenceSuggestionForNode, + buildDivergenceScreen, + captureDivergenceObservation, + collectReplayDivergenceSuggestions, + toReplayRepairHintCapture, + type DivergenceFieldSanitizer, +} from './session-replay-divergence.ts'; +import { computeReplayRepairHint } from './session-replay-repair-hint.ts'; +import { + buildReplayDivergenceFailureResponseFromDescriptor, + hoistReplayFailureCauseDiagnosticMeta, +} from './session-replay-runtime-failure-response.ts'; + +export type MaestroFailedEngineEvent = MaestroEngineEvent & { + readonly durationMs: number; + readonly error: unknown; + readonly artifactPaths: readonly string[]; + readonly expandedVariables: Readonly>; +}; + +export async function buildTypedMaestroFailureResponse(params: { + readonly error: DaemonError; + readonly event: MaestroFailedEngineEvent; + readonly plan: MaestroReplayPlan; + readonly replayPath: string; + readonly req: DaemonRequest; + readonly sessionName: string; + readonly sessionStore: SessionStore; + readonly logPath: string; + readonly snapshotDiagnostics?: SnapshotDiagnosticsSummary; +}): Promise { + const { event, plan, replayPath, req, sessionName, sessionStore, logPath } = params; + const cause = hoistReplayFailureCauseDiagnosticMeta(params.error); + const scrubVars = [ + ...collectExpandedScrubVars(event.expandedVariables), + ...collectMaestroTextScrubVars(event.command), + ].sort((left, right) => right.value.length - left.value.length); + const sanitize = createReplayDivergenceSanitizer(scrubVars); + const safeCause = { + ...cause, + message: sanitize(cause.message), + ...(cause.hint ? { hint: sanitize(cause.hint) } : {}), + }; + const diagnosticAction = diagnosticActionForEvent(event, req); + const session = sessionStore.get(sessionName); + const observation = session + ? await captureDivergenceObservation({ + session, + sessionName, + sessionStore, + logPath, + action: diagnosticAction, + }) + : { + state: 'unavailable' as const, + reason: 'no-session', + hint: 'The session closed before a post-failure screen could be captured.', + }; + const suggestions = + session && observation.state === 'available' + ? collectTypedMaestroSuggestions({ + command: event.command, + platform: plan.platform, + action: diagnosticAction, + session, + nodes: observation.nodes, + sanitize, + }) + : []; + const resume = evaluateMaestroReplayResume(plan, { + from: event.stepIndex, + planDigest: plan.digest, + }); + const progress = formatMaestroCommandProgress(event.command); + const actionLabel = [event.command.kind, formatMaestroActionValue(progress.value)] + .filter(Boolean) + .join(' '); + const divergence: ReplayDivergence = { + version: 1, + kind: 'action-failure', + step: { + index: event.stepIndex, + source: { + path: sanitize(event.source.path ?? replayPath), + line: event.source.line, + }, + }, + action: sanitize(actionLabel), + cause: { + code: safeCause.code, + message: safeCause.message, + ...(safeCause.hint ? { hint: safeCause.hint } : {}), + }, + screen: buildDivergenceScreen(observation, sanitize), + suggestions: suggestions.slice(0, REPLAY_DIVERGENCE_SUGGESTION_LIMIT), + suggestionCount: suggestions.length, + resume: resume.allowed + ? { allowed: true, from: event.stepIndex, planDigest: plan.digest } + : { + allowed: false, + from: event.stepIndex, + planDigest: plan.digest, + reason: resume.reason, + }, + repairHint: computeReplayRepairHint({ + kind: 'action-failure', + targetEvidence: diagnosticAction.targetEvidence, + capture: toReplayRepairHintCapture(observation), + }), + }; + const bounded = boundReplayDivergenceForSession({ + sessionStore, + sessionName, + divergence, + responseLevel: req.meta?.responseLevel, + }); + return buildReplayDivergenceFailureResponseFromDescriptor({ + error: safeCause, + actionLabel, + action: event.command.kind, + positionals: safeProgressPositionals(event.command.kind, progress.value), + step: event.stepIndex, + replayPath, + artifactPaths: [...event.artifactPaths], + snapshotDiagnostics: params.snapshotDiagnostics, + divergence: bounded, + scrubVars, + }); +} + +function formatMaestroActionValue(value: string | undefined): string { + if (!value || value === '') return value ?? ''; + return formatScriptArg(value); +} + +function diagnosticActionForEvent(event: MaestroEngineEvent, req: DaemonRequest): SessionAction { + const progress = formatMaestroCommandProgress(event.command); + const command = diagnosticCommand(event.command.kind); + return { + ts: Date.now(), + command, + positionals: safeProgressPositionals(event.command.kind, progress.value), + flags: req.flags ?? {}, + }; +} + +function collectTypedMaestroSuggestions(params: { + command: MaestroCommand; + platform: MaestroReplayPlan['platform']; + action: SessionAction; + session: SessionState; + nodes: SnapshotNode[]; + sanitize: DivergenceFieldSanitizer; +}) { + const query = typedSuggestionQuery(params.command); + if (!query || (params.platform !== 'android' && params.platform !== 'ios')) { + return collectReplayDivergenceSuggestions({ + action: params.action, + session: params.session, + nodes: params.nodes, + sanitize: params.sanitize, + }); + } + const snapshot = { createdAt: Date.now(), nodes: params.nodes }; + const resolution = resolveMaestroTargetFromSnapshot( + snapshot, + query, + params.platform, + getSnapshotReferenceFrame(snapshot), + { requireOnScreen: true, promoteTapTarget: query.promoteTapTarget }, + ); + if (!resolution.ok) return []; + return [ + buildReplayDivergenceSuggestionForNode({ + node: resolution.node, + session: params.session, + action: params.action, + basis: suggestionBasis(query.selector), + sanitize: params.sanitize, + }), + ]; +} + +type TypedSuggestionQuery = { + selector: MaestroSelector; + index?: number; + childOf?: MaestroSelector; + promoteTapTarget?: boolean; +}; + +function typedSuggestionQuery(command: MaestroCommand): TypedSuggestionQuery | undefined { + if (isTargetInteraction(command)) return targetInteractionSuggestion(command); + if (isObservationCommand(command)) return observationSuggestion(command); + if (command.kind === 'swipe' && command.gesture.kind === 'target') { + return { selector: command.gesture.from }; + } + return undefined; +} + +type TargetInteractionCommand = Extract< + MaestroCommand, + { kind: 'tapOn' | 'doubleTapOn' | 'longPressOn' } +>; + +function isTargetInteraction(command: MaestroCommand): command is TargetInteractionCommand { + return ( + command.kind === 'tapOn' || command.kind === 'doubleTapOn' || command.kind === 'longPressOn' + ); +} + +function targetInteractionSuggestion( + command: TargetInteractionCommand, +): TypedSuggestionQuery | undefined { + if (command.target.space !== 'target') return undefined; + if (command.kind === 'tapOn') { + return { + selector: command.target.selector, + index: command.index, + childOf: command.childOf, + promoteTapTarget: true, + }; + } + return { selector: command.target.selector, promoteTapTarget: true }; +} + +type ObservationCommand = Extract< + MaestroCommand, + { kind: 'assertVisible' | 'assertNotVisible' | 'extendedWaitUntil' | 'scrollUntilVisible' } +>; + +function isObservationCommand(command: MaestroCommand): command is ObservationCommand { + return ( + command.kind === 'assertVisible' || + command.kind === 'assertNotVisible' || + command.kind === 'extendedWaitUntil' || + command.kind === 'scrollUntilVisible' + ); +} + +function observationSuggestion(command: ObservationCommand): TypedSuggestionQuery | undefined { + switch (command.kind) { + case 'assertVisible': + case 'assertNotVisible': + return { selector: command.target }; + case 'extendedWaitUntil': + return command.visible + ? { selector: command.visible } + : command.notVisible + ? { selector: command.notVisible } + : undefined; + case 'scrollUntilVisible': + return { selector: command.element }; + } +} + +function suggestionBasis(selector: MaestroSelector): 'id' | 'label' | 'other' { + if (selector.id !== undefined) return 'id'; + if (selector.text !== undefined || selector.label !== undefined) return 'label'; + return 'other'; +} + +function diagnosticCommand(command: string): string { + if (command === 'tapOn' || command === 'doubleTapOn') return 'click'; + if (command === 'longPressOn') return 'longpress'; + if ( + command === 'assertVisible' || + command === 'assertNotVisible' || + command === 'extendedWaitUntil' || + command === 'scrollUntilVisible' + ) { + return 'wait'; + } + return command; +} + +function safeProgressPositionals(command: string, value: string | undefined): string[] { + if (!value || command === 'inputText' || command === 'pasteText') return []; + return [value]; +} + +function collectExpandedScrubVars(values: Readonly>): ReplayVarScrubEntry[] { + return Object.entries(values) + .filter(([, value]) => value.length > 0) + .map(([name, value]) => ({ name, value })) + .sort((left, right) => right.value.length - left.value.length); +} + +function collectMaestroTextScrubVars(command: MaestroCommand): ReplayVarScrubEntry[] { + if ((command.kind !== 'inputText' && command.kind !== 'pasteText') || command.text.length === 0) { + return []; + } + return [{ name: `${command.kind}.text`, value: command.text }]; +} diff --git a/src/daemon/handlers/session-replay-maestro-runtime.ts b/src/daemon/handlers/session-replay-maestro-runtime.ts new file mode 100644 index 000000000..546a3082c --- /dev/null +++ b/src/daemon/handlers/session-replay-maestro-runtime.ts @@ -0,0 +1,456 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { setTimeout as sleep } from 'node:timers/promises'; +import { AppError, normalizeError } from '../../kernel/errors.ts'; +import { resolveTargetDevice } from '../../core/dispatch.ts'; +import { getRequestSignal } from '../../request/cancel.ts'; +import { emitRequestProgress, readReplayTestActionProgress } from '../../request/progress.ts'; +import { + collectReplayShellEnv, + parseReplayCliEnvEntries, + readReplayCliEnvEntries, + readReplayShellEnvSource, +} from '../../replay/vars.ts'; +import { summarizeSnapshotTimingSamples } from '../../snapshot-diagnostics.ts'; +import { createDaemonMaestroRuntimePort } from '../../compat/maestro/daemon-runtime-port.ts'; +import { executeMaestroPlan } from '../../compat/maestro/engine.ts'; +import { formatMaestroCommandProgress } from '../../compat/maestro/progress.ts'; +import { parseMaestroProgram } from '../../compat/maestro/program-ir-parser.ts'; +import { createMaestroProgramLoader } from '../../compat/maestro/program-loader.ts'; +import { + compileMaestroReplayPlan, + resolveMaestroReplayStartIndex, +} from '../../compat/maestro/replay-plan.ts'; +import type { + MaestroEngineEvent, + MaestroEngineObserver, +} from '../../compat/maestro/engine-types.ts'; +import type { MaestroPlatform, MaestroProgram } from '../../compat/maestro/program-ir.ts'; +import type { MaestroReplayPlan } from '../../compat/maestro/replay-plan-types.ts'; +import type { DeviceInfo } from '../../kernel/device.ts'; +import type { ReplayCommandResult } from '../../contracts/replay.ts'; +import type { DaemonInvokeFn, DaemonRequest, DaemonResponse } from '../types.ts'; +import { assertSessionSelectorMatches } from '../session-selector.ts'; +import { SessionStore } from '../session-store.ts'; +import { errorResponse } from './response.ts'; +import { buildReplayBuiltinVars } from './session-replay-vars.ts'; +import { + buildTypedMaestroFailureResponse, + type MaestroFailedEngineEvent, +} from './session-replay-maestro-failure.ts'; +import { resolveEffectiveOpenRuntimeHints } from './session-runtime.ts'; +import { appendReplayTraceEvent } from './session-replay-trace.ts'; + +type TypedMaestroReplayParams = { + req: DaemonRequest; + sessionName: string; + logPath: string; + sessionStore: SessionStore; + tracePath?: string; + invoke: DaemonInvokeFn; +}; + +type TypedMaestroReplayState = { + failedEvent?: MaestroFailedEngineEvent; + plan?: MaestroReplayPlan; + snapshotStart: number; +}; + +type TypedMaestroReplayContext = { + filePath: string; + program: MaestroProgram; + platform: Extract; + target: string; + runtimeHints: ReturnType; + defaults: Record; + env: Record; + signal: AbortSignal | undefined; + loadProgram: ReturnType; +}; + +export async function runTypedMaestroReplayFile( + params: TypedMaestroReplayParams, +): Promise { + const { req } = params; + const requestedPath = req.positionals?.[0]; + if (!requestedPath) return errorResponse('INVALID_ARGS', 'replay requires a path'); + if (req.flags?.saveScript !== undefined) { + return errorResponse( + 'INVALID_ARGS', + 'Maestro YAML does not support --save-script; ADR 0012 repair recording applies only to .ad scripts.', + ); + } + const startedAt = Date.now(); + const state: TypedMaestroReplayState = { snapshotStart: 0 }; + try { + return await executeTypedMaestroReplay({ + ...params, + requestedPath, + startedAt, + state, + }); + } catch (error) { + return await buildTypedMaestroReplayErrorResponse({ + ...params, + requestedPath, + state, + error, + }); + } +} + +async function executeTypedMaestroReplay( + params: TypedMaestroReplayParams & { + requestedPath: string; + startedAt: number; + state: TypedMaestroReplayState; + }, +): Promise { + const { req, sessionName, sessionStore, tracePath, invoke, state } = params; + const context = await prepareTypedMaestroReplay(params); + const plan = await compileMaestroReplayPlan(context.program, { + defaults: context.defaults, + env: context.env, + platform: context.platform, + target: context.target, + runtimeHints: context.runtimeHints, + loadProgram: context.loadProgram, + signal: context.signal, + }); + state.plan = plan; + const startIndex = resolveMaestroReplayStartIndex(plan, { + from: req.flags?.replayFrom, + planDigest: req.flags?.replayPlanDigest, + }); + const port = createMaestroReplayPort({ + req, + invoke, + platform: context.platform, + runtimeHints: context.runtimeHints, + sourcePath: context.filePath, + }); + state.snapshotStart = sessionStore.get(sessionName)?.snapshotDiagnostics?.samples.length ?? 0; + const result = await executeMaestroPlan(plan, port, { + defaults: context.defaults, + env: context.env, + platform: context.platform, + target: context.target, + loadProgram: context.loadProgram, + signal: context.signal, + startIndex, + observer: createMaestroReplayObserver({ + filePath: context.filePath, + tracePath, + onFailure: (event) => { + state.failedEvent = event; + }, + }), + }); + return buildTypedMaestroSuccessResponse({ + result, + plan, + startIndex, + startedAt: params.startedAt, + sessionName, + sessionStore, + snapshotStart: state.snapshotStart, + }); +} + +async function prepareTypedMaestroReplay( + params: TypedMaestroReplayParams & { requestedPath: string }, +): Promise { + const { req, requestedPath, sessionName, sessionStore } = params; + const filePath = SessionStore.expandHome(requestedPath, req.meta?.cwd); + const program = parseMaestroProgram(fs.readFileSync(filePath, 'utf8'), { + sourcePath: filePath, + }); + const session = sessionStore.get(sessionName); + if (session) assertSessionSelectorMatches(session, req.flags); + const device = + session?.device ?? + (maestroReplayNeedsResolvedDevice(req, sessionStore.getRuntimeHints(sessionName)) + ? await resolveTargetDevice(req.flags ?? {}) + : undefined); + const platform = resolveMaestroPlatform(req, device); + const target = resolveMaestroTarget(req, device); + const runtimeHints = resolveEffectiveOpenRuntimeHints({ + req, + sessionStore, + sessionName, + device, + platform, + }); + return { + filePath, + program, + platform, + target, + runtimeHints, + defaults: buildTypedMaestroDefaults({ + req, + sessionName, + filePath, + platform, + target, + }), + env: buildTypedMaestroEnv(req), + signal: getRequestSignal(req.meta?.requestId), + loadProgram: createMaestroProgramLoader(path.dirname(filePath)), + }; +} + +function maestroReplayNeedsResolvedDevice( + req: DaemonRequest, + storedRuntime: ReturnType, +): boolean { + if (req.flags?.platform !== 'android' && req.flags?.platform !== 'ios') return true; + const runtime = req.runtime ?? storedRuntime; + return ( + runtime?.metroPort !== undefined && !runtime.metroHost?.trim() && !runtime.bundleUrl?.trim() + ); +} + +function buildTypedMaestroDefaults(params: { + req: DaemonRequest; + sessionName: string; + filePath: string; + platform: Extract; + target: string; +}): Record { + return { + ...buildReplayBuiltinVars({ + req: params.req, + sessionName: params.sessionName, + metadata: {}, + resolvedPath: params.filePath, + }), + AD_PLATFORM: params.platform, + AD_TARGET: params.target, + }; +} + +function buildTypedMaestroEnv(req: DaemonRequest): Record { + return { + ...collectReplayShellEnv(readReplayShellEnvSource(req.flags?.replayShellEnv)), + ...parseReplayCliEnvEntries(readReplayCliEnvEntries(req.flags?.replayEnv)), + }; +} + +function createMaestroReplayPort(params: { + req: DaemonRequest; + invoke: DaemonInvokeFn; + platform: Extract; + runtimeHints: ReturnType; + sourcePath: string; +}) { + const { req, invoke, platform, runtimeHints, sourcePath } = params; + const { command: _command, positionals: _positionals, ...requestBase } = req; + const baseReq = + runtimeHints === undefined ? requestBase : { ...requestBase, runtime: runtimeHints }; + return createDaemonMaestroRuntimePort({ + baseReq, + invoke, + platform, + sourcePath, + dependencies: { + now: Date.now, + sleep: async (milliseconds, abortSignal) => { + await sleep(milliseconds, undefined, { signal: abortSignal }); + }, + }, + }); +} + +function createMaestroReplayObserver(params: { + filePath: string; + tracePath: string | undefined; + onFailure: (event: MaestroFailedEngineEvent) => void; +}): MaestroEngineObserver { + const { filePath, tracePath, onFailure } = params; + return { + commandStarted: (event) => { + emitMaestroProgress(filePath, event); + appendMaestroTraceStart(tracePath, filePath, event); + }, + commandCompleted: (event) => { + appendMaestroTraceStop(tracePath, filePath, event, true); + }, + commandFailed: (event) => { + onFailure(event); + appendMaestroTraceStop(tracePath, filePath, event, false); + }, + }; +} + +function buildTypedMaestroSuccessResponse(params: { + result: { artifactPaths: string[] }; + plan: MaestroReplayPlan; + startIndex: number; + startedAt: number; + sessionName: string; + sessionStore: SessionStore; + snapshotStart: number; +}): DaemonResponse { + const { result, plan, startIndex, startedAt, sessionName, sessionStore, snapshotStart } = params; + const snapshotDiagnostics = readTypedMaestroSnapshotDiagnostics( + sessionStore, + sessionName, + snapshotStart, + ); + const replayed = plan.total - startIndex; + return { + ok: true, + data: { + replayed, + healed: 0, + session: sessionName, + artifactPaths: result.artifactPaths, + ...(snapshotDiagnostics ? { snapshotDiagnostics } : {}), + message: replaySuccessMessage(replayed, Date.now() - startedAt), + } satisfies ReplayCommandResult, + }; +} + +async function buildTypedMaestroReplayErrorResponse( + params: TypedMaestroReplayParams & { + requestedPath: string; + state: TypedMaestroReplayState; + error: unknown; + }, +): Promise { + const normalizedError = normalizeError(params.error); + const { failedEvent, plan } = params.state; + if (failedEvent && plan) { + return await buildTypedMaestroFailureResponse({ + error: normalizedError, + event: failedEvent, + plan, + replayPath: SessionStore.expandHome(params.requestedPath, params.req.meta?.cwd), + req: params.req, + sessionName: params.sessionName, + sessionStore: params.sessionStore, + logPath: params.logPath, + snapshotDiagnostics: readTypedMaestroSnapshotDiagnostics( + params.sessionStore, + params.sessionName, + params.state.snapshotStart, + ), + }); + } + return errorResponse(normalizedError.code, normalizedError.message, { + ...(normalizedError.details ?? {}), + ...buildTypedMaestroErrorDetails(failedEvent), + }); +} + +function readTypedMaestroSnapshotDiagnostics( + sessionStore: SessionStore, + sessionName: string, + snapshotStart: number, +) { + const samples = + sessionStore.get(sessionName)?.snapshotDiagnostics?.samples.slice(snapshotStart) ?? []; + return summarizeSnapshotTimingSamples(samples); +} + +function buildTypedMaestroErrorDetails( + failedEvent: MaestroFailedEngineEvent | undefined, +): Record { + if (!failedEvent) return {}; + return { + replaySource: failedEvent.source, + replayStep: failedEvent.stepIndex, + replayStepTotal: failedEvent.stepTotal, + }; +} + +export function isTypedMaestroReplay(req: DaemonRequest, filePath: string): boolean { + return ( + req.flags?.replayBackend === 'maestro' && + (path.extname(filePath) === '.yaml' || path.extname(filePath) === '.yml') + ); +} + +function resolveMaestroPlatform( + req: DaemonRequest, + sessionDevice: DeviceInfo | undefined, +): Extract { + const platform = req.flags?.platform; + if (platform === 'android' || platform === 'ios') return platform; + if (sessionDevice?.platform === 'android') return 'android'; + if (sessionDevice?.platform === 'apple' && sessionDevice.appleOs === 'ios') return 'ios'; + throw new AppError( + 'INVALID_ARGS', + 'Maestro replay requires --platform android|ios or an active mobile session.', + ); +} + +function resolveMaestroTarget(req: DaemonRequest, sessionDevice: DeviceInfo | undefined): string { + return typeof req.flags?.target === 'string' + ? req.flags.target + : (sessionDevice?.target ?? 'mobile'); +} + +function emitMaestroProgress(file: string, event: MaestroEngineEvent): void { + const progress = readReplayTestActionProgress(); + if (!progress) return; + const formatted = formatMaestroCommandProgress(event.command); + emitRequestProgress({ + type: 'replay-test', + ...progress, + file: progress.file || file, + status: 'progress', + stepIndex: event.stepIndex, + stepTotal: event.stepTotal, + stepCommand: formatted.command, + ...(formatted.value ? { stepValue: formatted.value } : {}), + }); +} + +function appendMaestroTraceStart( + tracePath: string | undefined, + replayPath: string, + event: MaestroEngineEvent, +): void { + const formatted = formatMaestroCommandProgress(event.command); + appendReplayTraceEvent(tracePath, { + type: 'replay_action_start', + ts: new Date().toISOString(), + replayPath, + ...(event.source.path && event.source.path !== replayPath + ? { sourcePath: event.source.path } + : {}), + line: event.source.line, + step: event.stepIndex, + command: formatted.command, + positionals: formatted.value ? [formatted.value] : [], + }); +} + +function appendMaestroTraceStop( + tracePath: string | undefined, + replayPath: string, + event: MaestroEngineEvent & { durationMs: number; error?: unknown }, + ok: boolean, +): void { + appendReplayTraceEvent(tracePath, { + type: 'replay_action_stop', + ts: new Date().toISOString(), + replayPath, + ...(event.source.path && event.source.path !== replayPath + ? { sourcePath: event.source.path } + : {}), + line: event.source.line, + step: event.stepIndex, + command: formatMaestroCommandProgress(event.command).command, + ok, + durationMs: event.durationMs, + ...(!ok && event.error instanceof AppError ? { errorCode: event.error.code } : {}), + }); +} + +function replaySuccessMessage(replayed: number, wallClockMs: number): string { + const noun = replayed === 1 ? 'step' : 'steps'; + return `Replayed ${replayed} ${noun} in ${(wallClockMs / 1_000).toFixed(1)}s`; +} diff --git a/src/daemon/handlers/session-replay-resume.ts b/src/daemon/handlers/session-replay-resume.ts index e88ccfb1e..9e3f7f888 100644 --- a/src/daemon/handlers/session-replay-resume.ts +++ b/src/daemon/handlers/session-replay-resume.ts @@ -1,65 +1,17 @@ -import { MAESTRO_RUNTIME_COMMAND } from '../../compat/maestro/runtime-commands.ts'; import type { SessionAction } from '../types.ts'; import type { ReplayDivergenceResume } from '../../replay/divergence.ts'; -/** - * ADR 0012 decision 4 / migration step 5: resume does not reconstruct - * execution state. For a target `from` (1-based, into the SAME top-level - * `actions` array the replay runtime iterates) greater than 1, preflight - * rejects when any SKIPPED step (`1..from-1`) can produce `outputEnv` - * values a later step might consume, or when the skipped range or the - * resume target itself is a runtime control-flow wrapper (`retry` or - * `maestroRunFlowWhen` — evaluated dynamically, never expanded into - * individually addressable plan indices, so skipping into/over one cannot be - * proven safe). `from === 1` skips nothing and is always allowed. - */ +/** Generic `.ad` actions have no runtime variable producers or control wrappers. */ export type ReplayResumePreflightResult = { allowed: true } | { allowed: false; reason: string }; -// The only outputEnv producer today (`invokeMaestroRunScript`, -// `compat/maestro/runtime.ts`); a plain command's response is never merged -// into replay var scope (`readReplayOutputEnv`, -// `session-replay-action-runtime.ts`). -const OUTPUT_ENV_PRODUCING_COMMAND: string = MAESTRO_RUNTIME_COMMAND.runScript; - export function evaluateReplayResumePreflight(params: { from: number; actions: SessionAction[]; }): ReplayResumePreflightResult { - const { from, actions } = params; - if (from <= 1) return { allowed: true }; - - for (let index = 0; index <= from - 2; index += 1) { - const action = actions[index]; - if (!action) continue; - if (action.replayControl) { - return { - allowed: false, - reason: `step ${index + 1} is inside runtime control flow (${action.replayControl.kind}); skipping it without executing it cannot be proven safe.`, - }; - } - if (producesOutputEnv(action)) { - return { - allowed: false, - reason: `step ${index + 1} (${action.command}) can produce outputEnv values a later step may consume; skipping it without executing it cannot be proven safe.`, - }; - } - } - - const target = actions[from - 1]; - if (target?.replayControl) { - return { - allowed: false, - reason: `step ${from} is inside runtime control flow (${target.replayControl.kind}); it cannot be safely resumed into.`, - }; - } - + void params; return { allowed: true }; } -function producesOutputEnv(action: SessionAction): boolean { - return action.command === OUTPUT_ENV_PRODUCING_COMMAND; -} - /** Builds the `resume` object attached to every divergence report. */ export function buildReplayDivergenceResume(params: { failedIndex: number; // 1-based diff --git a/src/daemon/handlers/session-replay-runtime-failure-response.ts b/src/daemon/handlers/session-replay-runtime-failure-response.ts index e45f23f99..2f44e64b6 100644 --- a/src/daemon/handlers/session-replay-runtime-failure-response.ts +++ b/src/daemon/handlers/session-replay-runtime-failure-response.ts @@ -37,12 +37,50 @@ export function buildReplayDivergenceFailureResponse(params: { divergence, scrubVars, } = params; + return buildReplayDivergenceFailureResponseFromDescriptor({ + error, + actionLabel: formatDivergenceActionLabel(action), + action: action.command, + positionals: buildDisplayPositionals(action) ?? [], + step, + replayPath, + artifactPaths, + snapshotDiagnostics, + divergence, + scrubVars, + }); +} + +export function buildReplayDivergenceFailureResponseFromDescriptor(params: { + error: ReplayFailureCause; + actionLabel: string; + action: string; + positionals: string[]; + step: number; + replayPath: string; + artifactPaths: string[]; + snapshotDiagnostics?: SnapshotDiagnosticsSummary; + divergence: unknown; + scrubVars: ReplayVarScrubEntry[]; +}): DaemonResponse { + const { + error, + actionLabel, + action, + positionals, + step, + replayPath, + artifactPaths, + snapshotDiagnostics, + divergence, + scrubVars, + } = params; return { ok: false, error: { code: 'REPLAY_DIVERGENCE', message: scrubReplayVarValues( - `Replay failed at step ${step} (${formatDivergenceActionLabel(action)}): ${error.message}`, + `Replay failed at step ${step} (${actionLabel}): ${error.message}`, scrubVars, ), hint: error.hint === undefined ? undefined : scrubReplayVarValues(error.hint, scrubVars), @@ -54,8 +92,8 @@ export function buildReplayDivergenceFailureResponse(params: { ...pickSafeCauseDetails(error.details), replayPath, step, - action: action.command, - positionals: buildDisplayPositionals(action) ?? [], + action, + positionals, artifactPaths, ...(snapshotDiagnostics ? { snapshotDiagnostics } : {}), divergence, diff --git a/src/daemon/handlers/session-replay-runtime.ts b/src/daemon/handlers/session-replay-runtime.ts index bc6603486..a2cbf530b 100644 --- a/src/daemon/handlers/session-replay-runtime.ts +++ b/src/daemon/handlers/session-replay-runtime.ts @@ -10,7 +10,6 @@ import { } from '../../request/progress.ts'; import { SessionStore } from '../session-store.ts'; import { expandSessionPath } from '../session-paths.ts'; -import { type ReplayScriptMetadata } from '../../replay/script.ts'; import { computeReplayPlanDigest } from '../../replay/plan-digest.ts'; import { errorResponse } from './response.ts'; import { invokeReplayAction } from './session-replay-action-runtime.ts'; @@ -41,6 +40,11 @@ import { isReplayTargetGuardMismatchResponse, verifyReplayActionTarget, } from './session-replay-target-verification.ts'; +import { buildReplayBuiltinVars } from './session-replay-vars.ts'; +import { + isTypedMaestroReplay, + runTypedMaestroReplayFile, +} from './session-replay-maestro-runtime.ts'; /** Per-run invariants for a single replay step (ADR 0012 step 4 verify + dispatch + guard). */ type ReplayStepContext = { @@ -151,6 +155,22 @@ export async function runReplayScriptFile(params: { const artifactPaths = new Set(); try { resolved = SessionStore.expandHome(filePath, req.meta?.cwd); + if (isTypedMaestroReplay(req, resolved)) { + if (sessionStore.get(sessionName)?.saveScriptBoundary !== undefined) { + return errorResponse( + 'INVALID_ARGS', + 'This session has an active .ad --save-script repair run; finish it with replay --from or close before running Maestro YAML.', + ); + } + return await runTypedMaestroReplayFile({ + req, + sessionName, + logPath, + sessionStore, + tracePath, + invoke, + }); + } const script = fs.readFileSync(resolved, 'utf8'); const firstNonWhitespace = script.trimStart()[0]; if (firstNonWhitespace === '{' || firstNonWhitespace === '[') { @@ -160,7 +180,7 @@ export async function runReplayScriptFile(params: { ); } - const parsed = parseReplayInput(script, req.flags, { sourcePath: resolved }); + const parsed = parseReplayInput(script, req.flags); const metadata = parsed.metadata; const replayReq = metadata.platform || metadata.target @@ -295,43 +315,6 @@ export async function runReplayScriptFile(params: { } } -// fallow-ignore-next-line complexity -function buildReplayBuiltinVars(params: { - req: DaemonRequest; - sessionName: string; - metadata: ReplayScriptMetadata; - resolvedPath: string; -}): Record { - const { req, sessionName, metadata, resolvedPath } = params; - const flags = req.flags ?? {}; - const cwd = req.meta?.cwd ?? process.cwd(); - const filename = path.relative(cwd, resolvedPath) || resolvedPath; - const builtins: Record = { - AD_SESSION: sessionName, - AD_FILENAME: filename, - }; - const platform = (flags.platform as string | undefined) ?? metadata.platform; - if (platform) builtins.AD_PLATFORM = platform; - const target = (flags.target as string | undefined) ?? metadata.target; - if (target) builtins.AD_TARGET = target; - const device = flags.device; - if (typeof device === 'string' && device.length > 0) builtins.AD_DEVICE = device; - const deviceId = typeof flags.serial === 'string' ? flags.serial : flags.udid; - if (typeof deviceId === 'string' && deviceId.length > 0) { - builtins.AD_DEVICE_ID = deviceId; - } - if (typeof flags.shardIndex === 'number') { - const shardIndex = String(flags.shardIndex); - builtins.AD_SHARD_INDEX = shardIndex; - } - if (typeof flags.shardCount === 'number') builtins.AD_SHARD_COUNT = String(flags.shardCount); - const artifactsDir = flags.artifactsDir; - if (typeof artifactsDir === 'string' && artifactsDir.length > 0) { - builtins.AD_ARTIFACTS = artifactsDir; - } - return builtins; -} - function emitReplayTestActionProgress( file: string, actionIndex: number, @@ -355,26 +338,17 @@ function formatReplayTestActionProgress( action: SessionAction, ): Pick { return { - stepCommand: formatReplayTestProgressCommand(action.command), + stepCommand: action.command, ...formatReplayTestProgressValue(action), }; } -function formatReplayTestProgressCommand(command: string): string { - if (!command.startsWith('__maestro')) return command; - const name = command.slice('__maestro'.length); - return name.length > 0 ? name[0]!.toLowerCase() + name.slice(1) : command; -} - function formatReplayTestProgressValue( action: SessionAction, ): Pick { const positionals = action.positionals ?? []; const selectorValue = readSelectorDisplayValue(positionals[0]); if (selectorValue) return { stepValue: selectorValue }; - if (action.command === '__maestroTapPointPercent' && positionals.length >= 2) { - return { stepValue: `${positionals[0]},${positionals[1]}%` }; - } if (positionals.length === 0) return {}; return { stepValue: positionals.join(' ') }; } diff --git a/src/daemon/handlers/session-replay-trace.ts b/src/daemon/handlers/session-replay-trace.ts new file mode 100644 index 000000000..3e85a4e48 --- /dev/null +++ b/src/daemon/handlers/session-replay-trace.ts @@ -0,0 +1,22 @@ +import fs from 'node:fs'; +import { redactDiagnosticData } from '../../kernel/redaction.ts'; +import { emitDiagnostic } from '../../utils/diagnostics.ts'; + +export function appendReplayTraceEvent( + tracePath: string | undefined, + event: Record, +): void { + if (!tracePath) return; + try { + fs.appendFileSync(tracePath, `${JSON.stringify(redactDiagnosticData(event))}\n`); + } catch (error) { + emitDiagnostic({ + level: 'warn', + phase: 'replay_trace_write_failed', + data: { + path: tracePath, + error: error instanceof Error ? error.message : String(error), + }, + }); + } +} diff --git a/src/daemon/handlers/session-replay-vars.ts b/src/daemon/handlers/session-replay-vars.ts new file mode 100644 index 000000000..2a76678d1 --- /dev/null +++ b/src/daemon/handlers/session-replay-vars.ts @@ -0,0 +1,50 @@ +import path from 'node:path'; +import type { DaemonRequest } from '../types.ts'; +import type { ReplayScriptMetadata } from '../../replay/script.ts'; + +export function buildReplayBuiltinVars(params: { + req: DaemonRequest; + sessionName: string; + metadata: ReplayScriptMetadata; + resolvedPath: string; +}): Record { + const { req, sessionName, metadata, resolvedPath } = params; + const flags = req.flags ?? {}; + const builtins: Record = { + AD_SESSION: sessionName, + AD_FILENAME: resolveReplayFilename(req, resolvedPath), + }; + addReplayStringBuiltin(builtins, 'AD_PLATFORM', flags.platform ?? metadata.platform); + addReplayStringBuiltin(builtins, 'AD_TARGET', flags.target ?? metadata.target); + addReplayStringBuiltin(builtins, 'AD_DEVICE', flags.device); + addReplayStringBuiltin(builtins, 'AD_DEVICE_ID', resolveReplayDeviceId(flags)); + addReplayNumberBuiltin(builtins, 'AD_SHARD_INDEX', flags.shardIndex); + addReplayNumberBuiltin(builtins, 'AD_SHARD_COUNT', flags.shardCount); + addReplayStringBuiltin(builtins, 'AD_ARTIFACTS', flags.artifactsDir); + return builtins; +} + +function resolveReplayFilename(req: DaemonRequest, resolvedPath: string): string { + const cwd = req.meta?.cwd ?? process.cwd(); + return path.relative(cwd, resolvedPath) || resolvedPath; +} + +function resolveReplayDeviceId(flags: NonNullable): unknown { + return typeof flags.serial === 'string' ? flags.serial : flags.udid; +} + +function addReplayStringBuiltin( + builtins: Record, + key: string, + value: unknown, +): void { + if (typeof value === 'string' && value.length > 0) builtins[key] = value; +} + +function addReplayNumberBuiltin( + builtins: Record, + key: string, + value: unknown, +): void { + if (typeof value === 'number') builtins[key] = String(value); +} diff --git a/src/daemon/handlers/session-runtime.ts b/src/daemon/handlers/session-runtime.ts index 1a7b292fa..881b10d48 100644 --- a/src/daemon/handlers/session-runtime.ts +++ b/src/daemon/handlers/session-runtime.ts @@ -187,11 +187,12 @@ function resolveSessionRuntimeHints( sessionStore: SessionStore, sessionName: string, device?: DeviceInfo, + platform?: RuntimePlatform, ): SessionRuntimeHints | undefined { const runtime = sessionStore.getRuntimeHints(sessionName); if (!runtime) return undefined; const boundPlatform = device ? publicPlatformString(device) : undefined; - const deviceRuntimePlatform = toRuntimePlatform(boundPlatform); + const deviceRuntimePlatform = toRuntimePlatform(boundPlatform) ?? platform; if (runtime.platform && device && !deviceRuntimePlatform) { throw new AppError( 'INVALID_ARGS', @@ -214,41 +215,53 @@ function resolveOpenRuntimeHints(params: { req: DaemonRequest; sessionStore: SessionStore; sessionName: string; - device: DeviceInfo; + device?: DeviceInfo; + platform?: RuntimePlatform; }): { runtime: SessionRuntimeHints | undefined; previousRuntime: SessionRuntimeHints | undefined; replacedStoredRuntime: boolean; } { const { req, sessionStore, sessionName, device } = params; + const runtimePlatform = device + ? toRuntimePlatform(publicPlatformString(device)) + : params.platform; const previousRuntime = sessionStore.getRuntimeHints(sessionName); const explicitRuntime = normalizeExplicitRuntimeHints({ runtime: req.runtime, sessionName, - platform: toRuntimePlatform(publicPlatformString(device)), + platform: runtimePlatform, }); if (req.runtime === undefined) { + const storedRuntime = resolveSessionRuntimeHints( + sessionStore, + sessionName, + device, + runtimePlatform, + ); return { - runtime: applyDeviceDefaultMetroHost( - resolveSessionRuntimeHints(sessionStore, sessionName, device), - device, - ), + runtime: device ? applyDeviceDefaultMetroHost(storedRuntime, device) : storedRuntime, previousRuntime, replacedStoredRuntime: false, }; } + const selectedRuntime = + explicitRuntime && countConfiguredRuntimeHints(explicitRuntime) > 0 + ? explicitRuntime + : undefined; return { - runtime: applyDeviceDefaultMetroHost( - explicitRuntime && countConfiguredRuntimeHints(explicitRuntime) > 0 - ? explicitRuntime - : undefined, - device, - ), + runtime: device ? applyDeviceDefaultMetroHost(selectedRuntime, device) : selectedRuntime, previousRuntime, replacedStoredRuntime: true, }; } +export function resolveEffectiveOpenRuntimeHints( + params: Parameters[0], +): SessionRuntimeHints | undefined { + return resolveOpenRuntimeHints(params).runtime; +} + export function tryResolveOpenRuntimeHints( params: Parameters[0], ): { ok: true; data: ReturnType } | DaemonFailureResponse { diff --git a/src/daemon/handlers/session-test-discovery.ts b/src/daemon/handlers/session-test-discovery.ts index 051fa31ca..d6d63fcf8 100644 --- a/src/daemon/handlers/session-test-discovery.ts +++ b/src/daemon/handlers/session-test-discovery.ts @@ -5,7 +5,7 @@ import { isApplePlatform, type PlatformSelector } from '../../kernel/device.ts'; import { resolveRequestTrackingId } from '../../request/cancel.ts'; import { SessionStore } from '../session-store.ts'; import { readReplayScriptMetadata, type ReplayScriptMetadata } from '../../replay/script.ts'; -import { readMaestroFlowName } from '../../compat/maestro/replay-flow.ts'; +import { parseMaestroProgram } from '../../compat/maestro/program-ir-parser.ts'; const GLOB_PATTERN_CHARS = /[*?[\]{}]/; @@ -288,7 +288,7 @@ function readReplayTestTitle( replayBackend: string | undefined, ): string | undefined { return isMaestroReplayBackend(replayBackend) && path.extname(filePath) !== '.ad' - ? readMaestroFlowName(script) + ? parseMaestroProgram(script, { sourcePath: filePath }).config.name : undefined; } diff --git a/src/daemon/handlers/session-test.ts b/src/daemon/handlers/session-test.ts index 1f123cc0d..f315a584e 100644 --- a/src/daemon/handlers/session-test.ts +++ b/src/daemon/handlers/session-test.ts @@ -393,7 +393,7 @@ function shouldStopReplayTestExecution( ): boolean { return ( isRequestCanceled(requestId) || - flags?.failFast === true || + (flags?.failFast === true && result.status === 'failed') || isReplayInfrastructureFailure(result) ); } diff --git a/src/daemon/runtime-hints.ts b/src/daemon/runtime-hints.ts index f875f1e1f..5b51cba74 100644 --- a/src/daemon/runtime-hints.ts +++ b/src/daemon/runtime-hints.ts @@ -1,6 +1,6 @@ import { isIosFamily, type DeviceInfo } from '../kernel/device.ts'; import { AppError, asAppError } from '../kernel/errors.ts'; -import { execFailureDetails } from '../utils/exec.ts'; +import { execFailureDetails, type ExecResult } from '../utils/exec.ts'; import type { SessionRuntimeHints } from './types.ts'; import { resolveRuntimeTransportHints, @@ -28,7 +28,7 @@ const IOS_PACKAGER_SCHEME_KEY = 'RCT_packager_scheme'; const ANDROID_RUN_AS_HINT = 'React Native runtime hints require adb run-as access to the app sandbox. Verify the app is debuggable and the selected package/device are correct.'; const ANDROID_WRITE_HINT = - 'adb run-as succeeded, but writing the React Native dev-server preference file failed. Inspect stderr/details for the failing shell command.'; + 'adb run-as succeeded, but writing React Native dev-server preferences failed. Inspect stderr/details for the failing shell command.'; const ANDROID_PROBE_HINT = 'adb shell run-as probe failed. Check adb connectivity and that the device is reachable. Inspect stderr/details for more information.'; const DEFAULT_ANDROID_PREFS_XML = [ @@ -92,29 +92,32 @@ async function applyAndroidRuntimeHints( transport: ResolvedRuntimeTransport, ): Promise { assertAndroidRuntimePackageName(packageName); + const files: Array<{ path: string; xml: string }> = []; for (const prefsPath of androidDevPrefsPaths(packageName)) { const currentXml = await readAndroidDevPrefs(device, packageName, prefsPath); - let nextXml = upsertAndroidStringPref( + let xml = upsertAndroidStringPref( currentXml, ANDROID_DEBUG_HOST_KEY, `${transport.host}:${transport.port}`, ); - nextXml = upsertAndroidBooleanPref(nextXml, ANDROID_HTTPS_KEY, transport.scheme === 'https'); - await writeAndroidDevPrefs(device, packageName, prefsPath, nextXml); + xml = upsertAndroidBooleanPref(xml, ANDROID_HTTPS_KEY, transport.scheme === 'https'); + files.push({ path: prefsPath, xml }); } + await writeAndroidDevPrefs(device, packageName, files); } async function clearAndroidRuntimeHints(device: DeviceInfo, packageName: string): Promise { assertAndroidRuntimePackageName(packageName); + const files: Array<{ path: string; xml: string }> = []; for (const prefsPath of androidDevPrefsPaths(packageName)) { const currentXml = await readAndroidDevPrefs(device, packageName, prefsPath); const withoutHost = removeAndroidPrefEntry(currentXml, ANDROID_DEBUG_HOST_KEY); - const withoutHttps = removeAndroidPrefEntry(withoutHost, ANDROID_HTTPS_KEY); - if (withoutHttps === currentXml) continue; - await writeAndroidDevPrefs(device, packageName, prefsPath, withoutHttps); + const xml = removeAndroidPrefEntry(withoutHost, ANDROID_HTTPS_KEY); + if (xml !== currentXml) files.push({ path: prefsPath, xml }); } + if (files.length === 0) return; + await writeAndroidDevPrefs(device, packageName, files); } - async function readAndroidDevPrefs( device: DeviceInfo, packageName: string, @@ -130,55 +133,81 @@ async function readAndroidDevPrefs( async function writeAndroidDevPrefs( device: DeviceInfo, packageName: string, - prefsPath: string, - xml: string, + files: Array<{ path: string; xml: string }>, +): Promise { + await assertAndroidAppSandboxAccessible(device, packageName); + try { + await writeAndroidDevPrefsFiles(device, packageName, files); + } catch (error) { + throw androidRuntimeHintsWriteError(error, packageName); + } +} + +async function assertAndroidAppSandboxAccessible( + device: DeviceInfo, + packageName: string, ): Promise { const probeArgs = ['shell', 'run-as', packageName, 'id']; const probeResult = await runAndroidAdb(device, probeArgs, { allowFailure: true }); - if (probeResult.exitCode !== 0) { - const runAsDenied = isAndroidRunAsDeniedOutput(probeResult.stdout, probeResult.stderr); - throw new AppError( - 'COMMAND_FAILED', - runAsDenied - ? `Failed to access Android app sandbox for ${packageName}` - : `Failed to probe Android app sandbox for ${packageName}`, - execFailureDetails(probeResult, { - package: packageName, - cmd: 'adb', - args: probeArgs, - hint: runAsDenied ? ANDROID_RUN_AS_HINT : ANDROID_PROBE_HINT, - }), - ); - } + if (probeResult.exitCode === 0) return; + throw androidRuntimeHintsProbeError(probeResult, packageName, probeArgs); +} - try { - await runAndroidAdb(device, ['shell', 'run-as', packageName, 'mkdir', '-p', 'shared_prefs']); - await runAndroidAdb(device, ['shell', 'run-as', packageName, 'tee', prefsPath], { - stdin: xml.trimEnd(), +function androidRuntimeHintsProbeError( + result: ExecResult, + packageName: string, + args: string[], +): AppError { + const runAsDenied = isAndroidRunAsDeniedOutput(result.stdout, result.stderr); + return new AppError( + 'COMMAND_FAILED', + runAsDenied + ? `Failed to access Android app sandbox for ${packageName}` + : `Failed to probe Android app sandbox for ${packageName}`, + execFailureDetails(result, { + package: packageName, + cmd: 'adb', + args, + hint: runAsDenied ? ANDROID_RUN_AS_HINT : ANDROID_PROBE_HINT, + }), + ); +} + +async function writeAndroidDevPrefsFiles( + device: DeviceInfo, + packageName: string, + files: Array<{ path: string; xml: string }>, +): Promise { + await runAndroidAdb(device, ['shell', 'run-as', packageName, 'mkdir', '-p', 'shared_prefs']); + for (const file of files) { + await runAndroidAdb(device, ['shell', 'run-as', packageName, 'tee', file.path], { + stdin: file.xml.trimEnd(), }); - } catch (error) { - const appErr = asAppError(error); - if (appErr.code === 'TOOL_MISSING') throw appErr; - const stdout = typeof appErr.details?.stdout === 'string' ? appErr.details.stdout : ''; - const stderr = typeof appErr.details?.stderr === 'string' ? appErr.details.stderr : ''; - const runAsDenied = isAndroidRunAsDeniedOutput(stdout, stderr); - throw new AppError( - 'COMMAND_FAILED', - runAsDenied - ? `Failed to access Android app sandbox for ${packageName}` - : `Failed to write Android runtime hints for ${packageName}`, - { - ...(appErr.details ?? {}), - package: packageName, - cmd: 'adb', - phase: 'write-runtime-hints', - hint: runAsDenied ? ANDROID_RUN_AS_HINT : ANDROID_WRITE_HINT, - }, - appErr, - ); } } +function androidRuntimeHintsWriteError(error: unknown, packageName: string): AppError { + const appErr = asAppError(error); + if (appErr.code === 'TOOL_MISSING') return appErr; + const stdout = typeof appErr.details?.stdout === 'string' ? appErr.details.stdout : ''; + const stderr = typeof appErr.details?.stderr === 'string' ? appErr.details.stderr : ''; + const runAsDenied = isAndroidRunAsDeniedOutput(stdout, stderr); + return new AppError( + 'COMMAND_FAILED', + runAsDenied + ? `Failed to access Android app sandbox for ${packageName}` + : `Failed to write Android runtime hints for ${packageName}`, + { + ...(appErr.details ?? {}), + package: packageName, + cmd: 'adb', + phase: 'write-runtime-hints', + hint: runAsDenied ? ANDROID_RUN_AS_HINT : ANDROID_WRITE_HINT, + }, + appErr, + ); +} + async function applyIosSimulatorRuntimeHints( device: DeviceInfo, bundleId: string, diff --git a/src/daemon/types.ts b/src/daemon/types.ts index d8261ca0b..799c17b76 100644 --- a/src/daemon/types.ts +++ b/src/daemon/types.ts @@ -17,7 +17,7 @@ import type { RecordingExportQuality } from '../core/recording-export-quality.ts import type { RecordingScope } from '../contracts/recording-scope.ts'; import type { DeviceInfo, Platform, PlatformSelector } from '../kernel/device.ts'; import type { ExecBackgroundResult, ExecResult } from '../utils/exec.ts'; -import type { SnapshotState } from '../kernel/snapshot.ts'; +import type { Rect, SnapshotState } from '../kernel/snapshot.ts'; import type { TargetAnnotationV1 } from '../replay/target-identity.ts'; import type { ReplayTargetGuardDenotation } from '../replay/target-identity-node.ts'; import type { AppLogFailure, AppLogState } from './app-log-process.ts'; @@ -57,6 +57,8 @@ export type DaemonOpenLifecycle = { type DaemonRequestInternal = { openLifecycle?: DaemonOpenLifecycle; admittedLease?: DeviceLease; + /** Provider-owned viewport already resolved while normalizing a nested gesture command. */ + gestureViewport?: Rect; /** * ADR 0012 step 4 post-resolution guard: the verified target member's * normalized local identity AND structural denotation (document order + @@ -334,35 +336,11 @@ export type SessionState = { appLogFailure?: AppLogFailure; }; -/** - * Per-nested-action source provenance for control-flow wrappers, parallel to - * `actions`; `undefined` at an index means "the wrapping control action's own - * file". The runtime block invoker reads it so a failure inside a wrapped - * `runFlow` include reports the include's file+line, not the wrapper's. - */ -export type ReplayControlActionSource = { path: string; line: number }; - -export type SessionReplayControl = - | { - kind: 'maestroRunFlowWhen'; - mode: 'visible' | 'notVisible'; - selector: string; - actions: SessionAction[]; - actionSources?: (ReplayControlActionSource | undefined)[]; - } - | { - kind: 'retry'; - maxRetries: number; - actions: SessionAction[]; - actionSources?: (ReplayControlActionSource | undefined)[]; - }; - export type SessionAction = { ts: number; command: string; positionals: string[]; runtime?: SessionRuntimeHints; - replayControl?: SessionReplayControl; flags: Partial & { snapshotInteractiveOnly?: boolean; snapshotDepth?: number; diff --git a/src/platforms/android/__tests__/android-helper-artifact.fixtures.ts b/src/platforms/android/__tests__/android-helper-artifact.fixtures.ts deleted file mode 100644 index 634072fbd..000000000 --- a/src/platforms/android/__tests__/android-helper-artifact.fixtures.ts +++ /dev/null @@ -1,8 +0,0 @@ -import { fileURLToPath } from 'node:url'; - -export const ANDROID_HELPER_FIXTURE_APK_PATH = fileURLToPath( - new URL('./fixtures/helper-apk.fixture', import.meta.url), -); - -export const ANDROID_HELPER_FIXTURE_APK_SHA256 = - 'a5f6a2fba1163bba2f13026bd3a192f52ba2816524b7cfa83c6b7ca568f6710a'; diff --git a/src/platforms/android/__tests__/ime-lifecycle.test.ts b/src/platforms/android/__tests__/ime-lifecycle.test.ts index 1af7071fd..e79eb3e35 100644 --- a/src/platforms/android/__tests__/ime-lifecycle.test.ts +++ b/src/platforms/android/__tests__/ime-lifecycle.test.ts @@ -12,16 +12,16 @@ const PENDING_DIR = 'android-test-ime-pending'; // the suite passes on a fresh checkout that hasn't packaged android-ime-helper/dist (CI Coverage). vi.mock('../ime-helper.ts', async (importOriginal) => { const actual = await importOriginal(); - const fixture = await import('./android-helper-artifact.fixtures.ts'); + const fixture = await import('../../../__tests__/test-utils/android-snapshot-helper.ts'); return { ...actual, resolveAndroidImeHelperArtifact: vi.fn(async () => ({ - apkPath: fixture.ANDROID_HELPER_FIXTURE_APK_PATH, + apkPath: fixture.ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT.apkPath, manifest: { name: 'android-ime-helper' as const, version: '0.0.0', assetName: 'helper.apk', - sha256: fixture.ANDROID_HELPER_FIXTURE_APK_SHA256, + sha256: fixture.ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT.manifest.sha256, packageName: 'com.callstack.agentdevice.imehelper', versionCode: 1, serviceComponent: HELPER_SERVICE, diff --git a/src/platforms/android/__tests__/input-actions-fill.test.ts b/src/platforms/android/__tests__/input-actions-fill.test.ts index 4828bef50..60db310a9 100644 --- a/src/platforms/android/__tests__/input-actions-fill.test.ts +++ b/src/platforms/android/__tests__/input-actions-fill.test.ts @@ -1,6 +1,10 @@ import { test } from 'vitest'; import assert from 'node:assert/strict'; -import { ANDROID_EMULATOR } from '../../../__tests__/test-utils/index.ts'; +import { + ANDROID_EMULATOR, + ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, + createAndroidSnapshotHelperExecutor, +} from '../../../__tests__/test-utils/index.ts'; import { AppError } from '../../../kernel/errors.ts'; import { fillAndroid, typeAndroid } from '../input-actions.ts'; import { withAndroidAdbProvider, type AndroidAdbExecutor } from '../adb-executor.ts'; @@ -14,11 +18,16 @@ import { test('fillAndroid reports when the IME captures input instead of the app field', async () => { const calls: string[][] = []; let imeText = ''; + let snapshotCount = 0; await withFillAdb( async (args) => { calls.push(args); if (isTextInput(args)) imeText = args[3] ?? ''; - return adbResult(args[0] === 'exec-out' ? imeCaptureHierarchy(imeText) : ''); + return adbResult(''); + }, + () => { + snapshotCount += 1; + return imeCaptureHierarchy(imeText); }, async () => { await assert.rejects( @@ -48,12 +57,13 @@ test('fillAndroid reports when the IME captures input instead of the app field', false, ); assert.equal(calls.filter(isTextInput).length, 1); - assert.equal(calls.filter((args) => args[0] === 'exec-out').length, 1); + assert.equal(snapshotCount, 1); }); test('fillAndroid detects unknown active IME package during verification', async () => { const calls: string[][] = []; let imeText = ''; + let snapshotCount = 0; await withFillAdb( async (args) => { calls.push(args); @@ -61,7 +71,11 @@ test('fillAndroid detects unknown active IME package during verification', async return adbResult(vendorImeWithAppFocusInputMethodDump()); } if (isTextInput(args)) imeText = args[3] ?? ''; - return adbResult(args[0] === 'exec-out' ? vendorImeCaptureHierarchy(imeText) : ''); + return adbResult(''); + }, + () => { + snapshotCount += 1; + return vendorImeCaptureHierarchy(imeText); }, async () => { await assert.rejects( @@ -82,7 +96,7 @@ test('fillAndroid detects unknown active IME package during verification', async ); assert.equal(calls.filter(isTextInput).length, 1); - assert.equal(calls.filter((args) => args[0] === 'exec-out').length, 1); + assert.equal(snapshotCount, 1); }); test('typeAndroid rejects unicode text without provider-native injection', async () => { @@ -167,12 +181,13 @@ test('fillAndroid delegates target replacement to provider-native text injection let value = ''; await withAndroidAdbProvider( { - exec: async (args) => { - if (args[0] === 'exec-out') { - return adbResult(androidInputXml({ text: value })); - } - throw new Error(`unexpected adb call: ${args.join(' ')}`); - }, + snapshotHelperArtifact: ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, + exec: createAndroidSnapshotHelperExecutor({ + exec: async (args) => { + throw new Error(`unexpected adb call: ${args.join(' ')}`); + }, + captureXml: () => androidInputXml({ text: value }), + }), text: async (request) => { calls.push(request); value = request.text; @@ -205,13 +220,13 @@ test('fillAndroid waits for settled app text before reporting success', async () inputCount += 1; typed += args[3] ?? ''; } - if (args[0] === 'exec-out') { - dumpCount += 1; - const visibleText = dumpCount <= 1 ? typed : dumpCount <= 3 ? 'file' : 'filed'; - return adbResult(androidInputXml({ text: visibleText })); - } return adbResult(''); }, + () => { + dumpCount += 1; + const visibleText = dumpCount <= 1 ? typed : dumpCount <= 3 ? 'file' : 'filed'; + return androidInputXml({ text: visibleText }); + }, async () => { await fillAndroid(ANDROID_EMULATOR, 10, 10, 'filed'); }, @@ -313,8 +328,9 @@ test('fillAndroid accepts matching-length masked password verification', async ( async (args) => { if (isDeleteKey(args)) typed = ''; if (isTextInput(args)) typed = args[3] ?? ''; - return adbResult(args[0] === 'exec-out' ? passwordHierarchy(maskBullets(typed)) : ''); + return adbResult(''); }, + () => passwordHierarchy(maskBullets(typed)), async () => { await fillAndroid(ANDROID_EMULATOR, 10, 10, 'Test@123'); }, @@ -344,8 +360,31 @@ test('readAndroidTextAtPointInHierarchy prefers focused edit text over point fal const IME_RESOURCE_ID = 'com.google.android.inputmethod.latin:id/0_resource_name_obfuscated'; -async function withFillAdb(exec: AndroidAdbExecutor, fn: () => Promise): Promise { - await withAndroidAdbProvider({ exec }, { serial: ANDROID_EMULATOR.id }, fn); +async function withFillAdb(exec: AndroidAdbExecutor, fn: () => Promise): Promise; +async function withFillAdb( + exec: AndroidAdbExecutor, + captureXml: () => string | Promise, + fn: () => Promise, +): Promise; +async function withFillAdb( + exec: AndroidAdbExecutor, + captureXmlOrFn: (() => string | Promise) | (() => Promise), + maybeFn?: () => Promise, +): Promise { + const captureXml = maybeFn + ? (captureXmlOrFn as () => string | Promise) + : () => { + throw new Error('snapshot helper capture was not expected'); + }; + const fn = maybeFn ?? (captureXmlOrFn as () => Promise); + await withAndroidAdbProvider( + { + exec: createAndroidSnapshotHelperExecutor({ exec, captureXml }), + snapshotHelperArtifact: ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, + }, + { serial: ANDROID_EMULATOR.id }, + fn, + ); } function adbResult(stdout: string) { diff --git a/src/platforms/android/__tests__/input-actions-test-ime.test.ts b/src/platforms/android/__tests__/input-actions-test-ime.test.ts index e3100bf26..b9038d2a9 100644 --- a/src/platforms/android/__tests__/input-actions-test-ime.test.ts +++ b/src/platforms/android/__tests__/input-actions-test-ime.test.ts @@ -25,7 +25,11 @@ vi.mock('../ime-helper.ts', async (importOriginal) => { }; }); -import { ANDROID_EMULATOR } from '../../../__tests__/test-utils/index.ts'; +import { + ANDROID_EMULATOR, + ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, + createAndroidSnapshotHelperExecutor, +} from '../../../__tests__/test-utils/index.ts'; import { fillAndroid, typeAndroid } from '../input-actions.ts'; import { withAndroidAdbProvider, type AndroidAdbExecutor } from '../adb-executor.ts'; import { @@ -80,29 +84,33 @@ test('fillAndroid clears then commits non-ASCII text through the test IME and ve setAndroidTestImeActiveForTests(ANDROID_EMULATOR, true); let currentText = ''; const calls: string[][] = []; - const adb: AndroidAdbExecutor = async (args) => { - calls.push(args); - if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'broadcast') { - const action = args[args.indexOf('-a') + 1]; - if (action === 'com.callstack.agentdevice.imehelper.ACTION_CLEAR_TEXT') { - currentText = ''; - } else if (action === 'com.callstack.agentdevice.imehelper.ACTION_INPUT_TEXT_B64') { - const textIndex = args.indexOf('text'); - const payload = textIndex >= 0 ? args[textIndex + 1] : undefined; - currentText += Buffer.from(payload ?? '', 'base64').toString('utf8'); + const adb: AndroidAdbExecutor = createAndroidSnapshotHelperExecutor({ + exec: async (args) => { + calls.push(args); + if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'broadcast') { + const action = args[args.indexOf('-a') + 1]; + if (action === 'com.callstack.agentdevice.imehelper.ACTION_CLEAR_TEXT') { + currentText = ''; + } else if (action === 'com.callstack.agentdevice.imehelper.ACTION_INPUT_TEXT_B64') { + const textIndex = args.indexOf('text'); + const payload = textIndex >= 0 ? args[textIndex + 1] : undefined; + currentText += Buffer.from(payload ?? '', 'base64').toString('utf8'); + } + return { exitCode: 0, stdout: '', stderr: '' }; } return { exitCode: 0, stdout: '', stderr: '' }; - } - if (args[0] === 'exec-out') { - return { exitCode: 0, stdout: androidInputXml({ text: currentText }), stderr: '' }; - } - return { exitCode: 0, stdout: '', stderr: '' }; - }; - - await withAndroidAdbProvider(adb, { serial: ANDROID_EMULATOR.id }, async () => { - await fillAndroid(ANDROID_EMULATOR, 10, 10, 'Café ☕ 🎉 你好'); + }, + captureXml: () => androidInputXml({ text: currentText }), }); + await withAndroidAdbProvider( + { exec: adb, snapshotHelperArtifact: ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT }, + { serial: ANDROID_EMULATOR.id }, + async () => { + await fillAndroid(ANDROID_EMULATOR, 10, 10, 'Café ☕ 🎉 你好'); + }, + ); + assert.equal(currentText, 'Café ☕ 🎉 你好'); assert.equal( calls.some((args) => args[0] === 'shell' && args[1] === 'input' && args[2] === 'text'), diff --git a/src/platforms/android/__tests__/input-actions.test.ts b/src/platforms/android/__tests__/input-actions.test.ts index 8220f4e95..007dc484d 100644 --- a/src/platforms/android/__tests__/input-actions.test.ts +++ b/src/platforms/android/__tests__/input-actions.test.ts @@ -236,11 +236,6 @@ test('fillAndroid uses chunk-safe shell input and retries when verification stil ' fi', ' exit 0', 'fi', - 'if [ "$1" = "exec-out" ] && [ "$2" = "uiautomator" ] && [ "$3" = "dump" ] && [ "$4" = "/dev/tty" ]; then', - ' text="$(cat "$STATE_FILE" 2>/dev/null)"', - ' printf "" "$text"', - ' exit 0', - 'fi', 'echo "unexpected args: $@" >&2', 'exit 1', '', @@ -283,11 +278,6 @@ test('fillAndroid keeps delayed typing in typed-input mode', async () => { ' printf "%s" "$4" >> "$STATE_FILE"', ' exit 0', 'fi', - 'if [ "$1" = "exec-out" ] && [ "$2" = "uiautomator" ] && [ "$3" = "dump" ] && [ "$4" = "/dev/tty" ]; then', - ' text="$(cat "$STATE_FILE" 2>/dev/null)"', - ' printf "" "$text"', - ' exit 0', - 'fi', 'echo "unexpected args: $@" >&2', 'exit 1', '', @@ -316,6 +306,16 @@ test('fillAndroid tolerates delayed React Native text verification', async () => ' shift', ' shift', 'fi', + ...androidSnapshotHelperStateFileScript([ + 'count="$(cat "$DUMP_COUNT_FILE" 2>/dev/null || echo 0)"', + 'count=$((count + 1))', + 'printf "%s" "$count" > "$DUMP_COUNT_FILE"', + 'if [ "$count" -eq 1 ]; then', + ' text="sent the updat"', + 'else', + ' text="$(cat "$STATE_FILE" 2>/dev/null)"', + 'fi', + ]), 'if [ "$1" = "shell" ] && [ "$2" = "input" ] && [ "$3" = "tap" ]; then', ' exit 0', 'fi', @@ -331,18 +331,6 @@ test('fillAndroid tolerates delayed React Native text verification', async () => ' printf "%s" "$text" >> "$STATE_FILE"', ' exit 0', 'fi', - 'if [ "$1" = "exec-out" ] && [ "$2" = "uiautomator" ] && [ "$3" = "dump" ] && [ "$4" = "/dev/tty" ]; then', - ' count="$(cat "$DUMP_COUNT_FILE" 2>/dev/null || echo 0)"', - ' count=$((count + 1))', - ' printf "%s" "$count" > "$DUMP_COUNT_FILE"', - ' if [ "$count" -eq 1 ]; then', - ' text="sent the updat"', - ' else', - ' text="$(cat "$STATE_FILE" 2>/dev/null)"', - ' fi', - ' printf "" "$text"', - ' exit 0', - 'fi', 'echo "unexpected args: $@" >&2', 'exit 1', '', @@ -389,14 +377,16 @@ test('typeAndroid reports clear error when unicode input is unsupported', async ); }); -function androidSnapshotHelperStateFileScript(): string[] { +function androidSnapshotHelperStateFileScript( + resolveText: readonly string[] = ['text="$(cat "$STATE_FILE" 2>/dev/null)"'], +): string[] { return [ 'if [ "$1" = "shell" ] && [ "$2" = "cmd" ] && [ "$3" = "package" ] && [ "$4" = "list" ] && [ "$5" = "packages" ] && [ "$6" = "--show-versioncode" ] && [ "$7" = "com.callstack.agentdevice.snapshothelper" ]; then', ' printf "package:com.callstack.agentdevice.snapshothelper versionCode:999999\\n"', ' exit 0', 'fi', 'if [ "$1" = "shell" ] && [ "$2" = "am" ] && [ "$3" = "instrument" ]; then', - ' text="$(cat "$STATE_FILE" 2>/dev/null)"', + ...resolveText.map((line) => ` ${line}`), ' xml="$(printf "" "$text")"', ' payload="$(printf "%s" "$xml" | base64 | tr -d "\\n")"', ' printf "INSTRUMENTATION_STATUS: agentDeviceProtocol=android-snapshot-helper-v1\\n"', diff --git a/src/platforms/android/__tests__/snapshot-content-recovery.test.ts b/src/platforms/android/__tests__/snapshot-content-recovery.test.ts index 468c3219d..ca42b9901 100644 --- a/src/platforms/android/__tests__/snapshot-content-recovery.test.ts +++ b/src/platforms/android/__tests__/snapshot-content-recovery.test.ts @@ -39,7 +39,7 @@ test('keeps known IME blocking windows instead of falling back to covered app co expect(decision).toBeUndefined(); }); -test('falls back when helper output has only one meaningful IME node', () => { +test('rejects helper output with only one meaningful IME node', () => { const decision = classifyAndroidHelperContentRecovery( helperXml([ node({ @@ -67,7 +67,7 @@ test('falls back when helper output has only one meaningful IME node', () => { expect(decision?.diagnostics.helperInputMethodMeaningfulNodeCount).toBe(1); }); -test('falls back when helper output has no foreground app or IME content', () => { +test('rejects helper output with no foreground app or IME content', () => { const decision = classifyAndroidHelperContentRecovery( helperXml([ node({ @@ -95,6 +95,145 @@ test('falls back when helper output has no foreground app or IME content', () => expect(decision?.diagnostics.helperInputMethodMeaningfulNodeCount).toBe(0); }); +test('keeps a meaningful foreground application surface owned by another package', () => { + const decision = classifyAndroidHelperContentRecovery( + helperXml([ + node({ + windowType: 1, + packageName: 'com.google.android.permissioncontroller', + className: 'android.widget.FrameLayout', + }), + node({ + text: 'Allow access to photos?', + packageName: 'com.google.android.permissioncontroller', + className: 'android.widget.TextView', + }), + node({ + text: 'Allow', + resourceId: 'com.android.permissioncontroller:id/permission_allow_button', + packageName: 'com.google.android.permissioncontroller', + className: 'android.widget.Button', + }), + ]), + { + backend: 'android-helper', + nodeCount: 3, + rootPresent: true, + windowCount: 1, + captureMode: 'interactive-windows', + }, + { foregroundAppPackage: 'org.reactnavigation.playground' }, + ); + + expect(decision).toBeUndefined(); +}); + +test('rejects a status-bar-only capture without window metadata', () => { + const decision = classifyAndroidHelperContentRecovery( + helperXml([ + node({ + text: '7:52', + resourceId: 'com.android.systemui:id/clock', + packageName: 'com.android.systemui', + className: 'android.widget.TextView', + }), + node({ + text: 'Battery 100 percent', + resourceId: 'com.android.systemui:id/battery', + packageName: 'com.android.systemui', + className: 'android.widget.LinearLayout', + }), + ]), + { + backend: 'android-helper', + nodeCount: 2, + rootPresent: true, + windowCount: 1, + captureMode: 'interactive-windows', + }, + ); + + expect(decision?.reason).toBe('system-window-only'); + expect(decision?.diagnostics.helperWindowRootCount).toBe(0); + expect(decision?.diagnostics.helperSystemUiNodeCount).toBe(2); + expect(decision?.diagnostics.helperNonSystemMeaningfulNodeCount).toBe(0); +}); + +test('keeps a recognized system dialog without window metadata', () => { + const decision = classifyAndroidHelperContentRecovery( + helperXml([ + node({ + text: "Demo isn't responding", + resourceId: 'android:id/alertTitle', + packageName: 'com.android.systemui', + className: 'android.widget.TextView', + }), + node({ + text: 'Do you want to close it?', + resourceId: 'android:id/message', + packageName: 'com.android.systemui', + className: 'android.widget.TextView', + }), + node({ + text: 'Close app', + resourceId: 'android:id/button2', + packageName: 'com.android.systemui', + className: 'android.widget.Button', + }), + node({ + text: 'Wait', + resourceId: 'android:id/button1', + packageName: 'com.android.systemui', + className: 'android.widget.Button', + }), + ]), + { + backend: 'android-helper', + nodeCount: 4, + rootPresent: true, + windowCount: 1, + captureMode: 'interactive-windows', + }, + { foregroundAppPackage: 'com.pagerviewexample' }, + ); + + expect(decision).toBeUndefined(); +}); + +test('rejects system chrome inherited under an empty application window', () => { + const decision = classifyAndroidHelperContentRecovery( + helperXml([ + node({ + windowType: 1, + packageName: 'com.pagerviewexample', + className: 'android.widget.FrameLayout', + }), + node({ + text: '5:25', + packageName: 'com.android.systemui', + className: 'android.widget.TextView', + }), + node({ + text: 'Battery 100 percent', + packageName: 'com.android.systemui', + className: 'android.widget.ImageView', + }), + ]), + { + backend: 'android-helper', + nodeCount: 3, + rootPresent: true, + windowCount: 2, + captureMode: 'interactive-windows', + }, + { foregroundAppPackage: 'com.pagerviewexample' }, + ); + + expect(decision?.reason).toBe('content-poor-app-window'); + expect(decision?.diagnostics.helperApplicationMeaningfulNodeCount).toBe(0); + expect(decision?.diagnostics.helperNonSystemMeaningfulNodeCount).toBe(0); +}); + function helperXml(nodes: string[]): string { return `${nodes.join('')}`; } diff --git a/src/platforms/android/__tests__/snapshot.test.ts b/src/platforms/android/__tests__/snapshot.test.ts index ace0e45d6..3e6752a15 100644 --- a/src/platforms/android/__tests__/snapshot.test.ts +++ b/src/platforms/android/__tests__/snapshot.test.ts @@ -17,7 +17,7 @@ vi.mock('../adb.ts', async (importOriginal) => { }); import { screenshotAndroid } from '../screenshot.ts'; -import { dumpUiHierarchy, snapshotAndroid } from '../snapshot.ts'; +import { snapshotAndroid } from '../snapshot.ts'; import { buildUiHierarchySnapshot, parseUiHierarchyTree } from '../ui-hierarchy.ts'; import type { DeviceInfo } from '../../../kernel/device.ts'; import { flushDiagnosticsToSessionFile, withDiagnosticsScope } from '../../../utils/diagnostics.ts'; @@ -26,16 +26,13 @@ import { runCmd } from '../../../utils/exec.ts'; import { sleep } from '../adb.ts'; import { resetAndroidSnapshotHelperInstallCache } from '../snapshot-helper-install.ts'; import { resetAndroidSnapshotHelperSessions } from '../snapshot-helper-session.ts'; -import { type AndroidAdbExecutor, type AndroidSnapshotHelperManifest } from '../snapshot-helper.ts'; +import { type AndroidAdbExecutor } from '../snapshot-helper.ts'; +import { ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT } from '../../../__tests__/test-utils/index.ts'; import { withAndroidAdbProvider, type AndroidAdbProcess, type AndroidAdbProvider, } from '../adb-executor.ts'; -import { - ANDROID_HELPER_FIXTURE_APK_PATH, - ANDROID_HELPER_FIXTURE_APK_SHA256, -} from './android-helper-artifact.fixtures.ts'; const VALID_PNG = Buffer.from( 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+b9xkAAAAASUVORK5CYII=', @@ -43,7 +40,6 @@ const VALID_PNG = Buffer.from( ); const mockRunCmd = vi.mocked(runCmd); const mockSleep = vi.mocked(sleep); -const localAdbExecOptions = { detached: process.platform !== 'win32' }; const device: DeviceInfo = { platform: 'android', @@ -53,25 +49,7 @@ const device: DeviceInfo = { booted: true, }; -const helperManifest: AndroidSnapshotHelperManifest = { - name: 'android-snapshot-helper', - version: '0.13.3', - apkUrl: null, - sha256: ANDROID_HELPER_FIXTURE_APK_SHA256, - packageName: 'com.callstack.agentdevice.snapshothelper', - versionCode: 13003, - instrumentationRunner: 'com.callstack.agentdevice.snapshothelper/.SnapshotInstrumentation', - minSdk: 23, - targetSdk: 36, - outputFormat: 'uiautomator-xml', - statusProtocol: 'android-snapshot-helper-v1', - installArgs: ['install', '-r', '-t'], -}; - -const helperArtifact = { - apkPath: ANDROID_HELPER_FIXTURE_APK_PATH, - manifest: helperManifest, -}; +const helperArtifact = ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT; const installedHelperProbe = { exitCode: 0, stdout: 'package:com.callstack.agentdevice.snapshothelper versionCode:13004', @@ -93,23 +71,33 @@ function snapshotAndroidWithHelper( } function createHelperAdb( - handlers: Partial>, + handlers: Partial>, ): AndroidAdbExecutor { return async (args, options) => { - if (args.includes('--show-versioncode')) return installedHelperProbe; - if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'force-stop') { + if (isHelperVersionProbe(args)) return installedHelperProbe; + if (isHelperRuntimeReset(args)) { return { exitCode: 0, stdout: '', stderr: '' }; } - if (args.includes('instrument') && handlers.instrument) { - return await handlers.instrument(args, options); - } - if (args.includes('exec-out') && handlers.stock) { - return await handlers.stock(args, options); - } + const operation = helperAdbOperation(args); + const handler = operation ? handlers[operation] : undefined; + if (handler) return await handler(args, options); throw new Error(`unexpected helper adb args: ${args.join(' ')}`); }; } +function isHelperVersionProbe(args: string[]): boolean { + return args.includes('--show-versioncode'); +} + +function isHelperRuntimeReset(args: string[]): boolean { + return args[0] === 'shell' && args[1] === 'am' && args[2] === 'force-stop'; +} + +function helperAdbOperation(args: string[]): 'instrument' | 'activity' | undefined { + if (args.includes('instrument')) return 'instrument'; + return args.includes('dumpsys') && args.includes('activity') ? 'activity' : undefined; +} + function createPersistentSnapshotHelperProvider(options: { calls: string[][]; spawnArgs: string[][]; @@ -367,18 +355,6 @@ function androidSystemWindowOnlyXml(): string { ].join('\n'); } -function androidFabricAppXml(): string { - return [ - '', - '', - ' ', - ' ', - ' ', - ' ', - '', - ].join('\n'); -} - function androidContentPoorFabricAppWindowXml(): string { return [ '', @@ -445,18 +421,12 @@ async function withTempScreenshot( } } -function mockAndroidSnapshotXml(xml: string, activityDump = ''): void { - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (isAndroidSdkVersionCommand(args)) { - return { exitCode: 0, stdout: '35', stderr: '' }; - } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - if (args.includes('dumpsys') && args.includes('activity') && args.includes('top')) { - return { exitCode: 0, stdout: activityDump, stderr: '' }; - } - throw new Error(`unexpected args: ${args.join(' ')}`); +function androidSnapshotHelperAdb(xml: string, activityDump?: string): AndroidAdbExecutor { + return createHelperAdb({ + instrument: async () => ({ exitCode: 0, stdout: helperOutput(xml), stderr: '' }), + ...(activityDump === undefined + ? {} + : { activity: async () => ({ exitCode: 0, stdout: activityDump, stderr: '' }) }), }); } @@ -466,14 +436,6 @@ function isAndroidSdkVersionCommand(args: string[]): boolean { ); } -function adbTimeout(args: string[]): AppError { - return new AppError('COMMAND_FAILED', 'adb timed out after 8000ms', { - cmd: 'adb', - args, - timeoutMs: 8000, - }); -} - async function captureDiagnostics( scope: Parameters[0], callback: () => Promise, @@ -489,29 +451,7 @@ async function captureDiagnostics( } } -test('dumpUiHierarchy returns streamed XML even when exec-out exits non-zero', async () => { - const xml = - ''; - - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (args.includes('exec-out')) { - return { exitCode: 1, stdout: xml, stderr: 'theme warning' }; - } - throw new Error('fallback should not run'); - }); - - const result = await dumpUiHierarchy(device); - - assert.equal(result, xml); - assert.equal(mockRunCmd.mock.calls.length, 1); - assert.deepEqual(mockRunCmd.mock.calls[0]?.[2], { - allowFailure: true, - timeoutMs: 8000, - ...localAdbExecOptions, - }); -}); - -test('snapshotAndroid uses injected helper artifact before stock uiautomator', async () => { +test('snapshotAndroid uses the injected helper artifact', async () => { const timeouts: Array = []; const helperAdb: AndroidAdbExecutor = async (args, options) => { timeouts.push(options?.timeoutMs); @@ -645,6 +585,7 @@ test('snapshotAndroid emits helper phase diagnostics', async () => { test('snapshotAndroid resolves helper adb through scoped provider', async () => { const adbCalls: string[][] = []; const provider: AndroidAdbProvider = { + snapshotHelperArtifact: helperArtifact, exec: async (args) => { adbCalls.push(args); if (args.includes('--show-versioncode')) { @@ -674,13 +615,12 @@ test('snapshotAndroid resolves helper adb through scoped provider', async () => }; const result = await withAndroidAdbProvider(provider, { serial: device.id }, async () => - snapshotAndroid(device, { - helperArtifact, - }), + snapshotAndroid(device), ); assert.equal(result.nodes[0]?.label, 'provider-helper'); assert.equal(result.androidSnapshot.backend, 'android-helper'); + assert.equal(result.androidSnapshot.helperVersion, helperArtifact.manifest.version); assert.deepEqual( adbCalls.map((args) => args[0]), ['shell', 'shell'], @@ -754,10 +694,8 @@ test('snapshotAndroid keeps daemon-session helper alive for reuse until session ); }); -test('snapshotAndroid falls back to stock uiautomator when helper fails', async () => { +test('snapshotAndroid fails closed when the helper fails', async () => { const adbCalls: string[][] = []; - const stockXml = - ''; const helperAdb: AndroidAdbExecutor = async (args) => { adbCalls.push(args); if (args.includes('--show-versioncode')) { @@ -767,37 +705,26 @@ test('snapshotAndroid falls back to stock uiautomator when helper fails', async stderr: '', }; } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: stockXml, stderr: '' }; - } if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'force-stop') { return { exitCode: 0, stdout: '', stderr: '' }; } return { exitCode: 1, stdout: '', stderr: 'instrumentation failed' }; }; - const result = await snapshotAndroid(device, { - helperAdb, - helperArtifact, - }); - - assert.equal(result.nodes[0]?.label, 'stock'); - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.match( - result.androidSnapshot.fallbackReason ?? '', - /failed before returning parseable output/, + await assert.rejects( + () => snapshotAndroid(device, { helperAdb, helperArtifact }), + /Android snapshot helper failed.*failed before returning parseable output/, ); - assert.deepEqual( - adbCalls.map((args) => args[0]), - ['shell', 'shell', 'shell', 'exec-out'], + assert.equal( + adbCalls.some((args) => args.includes('exec-out')), + false, ); assert.equal(mockRunCmd.mock.calls.length, 0); }); -test('snapshotAndroid falls back to stock uiautomator when helper returns only system windows', async () => { +test('snapshotAndroid fails closed when helper returns only system windows', async () => { const adbCalls: string[][] = []; const helperXml = androidSystemWindowOnlyXml(); - const stockXml = androidFabricAppXml(); const helperAdb: AndroidAdbExecutor = async (args) => { adbCalls.push(args); if (args.includes('--show-versioncode')) return installedHelperProbe; @@ -807,26 +734,12 @@ test('snapshotAndroid falls back to stock uiautomator when helper returns only s if (args.includes('instrument')) { return { exitCode: 0, stdout: helperOutput(helperXml, { nodeCount: 3 }), stderr: '' }; } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: stockXml, stderr: '' }; - } throw new Error(`unexpected helper adb args: ${args.join(' ')}`); }; - const result = await snapshotAndroidWithHelper(helperAdb); - - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.equal( - result.androidSnapshot.fallbackReason, - 'Android snapshot helper returned only non-application windows', - ); - assert.equal( - result.nodes.some((node) => node.label === 'Fabric dashboard'), - true, - ); - assert.equal( - result.nodes.some((node) => node.label === 'Open details'), - true, + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb), + /Android snapshot helper returned only non-application windows/, ); assert.equal( adbCalls.some( @@ -836,97 +749,64 @@ test('snapshotAndroid falls back to stock uiautomator when helper returns only s ); assert.equal( adbCalls.some((args) => args.includes('exec-out')), - true, + false, ); }); -test('snapshotAndroid falls back to stock uiautomator when helper returns no nodes', async () => { +test('snapshotAndroid fails closed when helper returns no nodes', async () => { const helperXml = ''; - const stockXml = androidFabricAppXml(); const helperAdb = createHelperAdb({ instrument: async () => ({ exitCode: 0, stdout: helperOutput(helperXml, { nodeCount: 0 }), stderr: '', }), - stock: async () => ({ exitCode: 0, stdout: stockXml, stderr: '' }), }); - const result = await snapshotAndroidWithHelper(helperAdb); - - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.equal( - result.androidSnapshot.fallbackReason, - 'Android snapshot helper returned no accessibility nodes', - ); - assert.equal( - result.nodes.some((node) => node.label === 'Fabric dashboard'), - true, + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb), + /Android snapshot helper returned no accessibility nodes/, ); }); -test('snapshotAndroid falls back to stock uiautomator when foreground app window lacks content', async () => { +test('snapshotAndroid fails closed when foreground app window lacks content', async () => { const helperXml = androidContentPoorFabricAppWindowXml(); - const stockXml = androidFabricAppXml(); const helperAdb = createHelperAdb({ instrument: async () => ({ exitCode: 0, stdout: helperOutput(helperXml, { nodeCount: 4, windowCount: 2 }), stderr: '', }), - stock: async () => ({ exitCode: 0, stdout: stockXml, stderr: '' }), - }); - - const result = await snapshotAndroidWithHelper(helperAdb, { - appBundleId: 'io.example.fabric', }); - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.equal( - result.androidSnapshot.fallbackReason, - 'Android snapshot helper returned insufficient foreground app content', - ); - assert.equal( - result.nodes.some((node) => node.label === 'Fabric dashboard'), - true, - ); - assert.equal( - result.nodes.some((node) => node.label === 'Open details'), - true, + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb, { appBundleId: 'io.example.fabric' }), + (error: unknown) => { + assert(error instanceof AppError); + assert.match(error.message, /insufficient foreground app content/); + assert.equal(error.details?.retriable, true); + return true; + }, ); }); -test('snapshotAndroid falls back to stock uiautomator when standalone helper sees only an app overlay', async () => { +test('snapshotAndroid fails closed when standalone helper sees only an app overlay', async () => { const helperXml = androidContentPoorExpoToolsOverlayXml(); - const stockXml = androidFabricAppXml(); const helperAdb = createHelperAdb({ instrument: async () => ({ exitCode: 0, stdout: helperOutput(helperXml, { nodeCount: 4, windowCount: 2 }), stderr: '', }), - stock: async () => ({ exitCode: 0, stdout: stockXml, stderr: '' }), }); - const result = await snapshotAndroidWithHelper(helperAdb); - - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.equal( - result.androidSnapshot.fallbackReason, - 'Android snapshot helper returned insufficient application window content', - ); - assert.equal( - result.nodes.some((node) => node.label === 'Fabric dashboard'), - true, - ); - assert.equal( - result.nodes.some((node) => node.label === 'Open details'), - true, + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb), + /Android snapshot helper returned insufficient application window content/, ); }); test('snapshotAndroid keeps helper output when application and system windows are both present', async () => { - let stockAttempted = false; const helperXml = [ '', '', @@ -945,10 +825,6 @@ test('snapshotAndroid keeps helper output when application and system windows ar stdout: helperOutput(helperXml, { nodeCount: 4 }), stderr: '', }), - stock: async () => { - stockAttempted = true; - throw new Error('stock fallback should not run'); - }, }); const result = await snapshotAndroidWithHelper(helperAdb, { @@ -956,17 +832,13 @@ test('snapshotAndroid keeps helper output when application and system windows ar }); assert.equal(result.androidSnapshot.backend, 'android-helper'); - assert.equal(result.androidSnapshot.fallbackReason, undefined); assert.equal( result.nodes.some((node) => node.label === 'Fabric dashboard'), true, ); - assert.equal(stockAttempted, false); }); -test('snapshotAndroid emits fallback and stock capture diagnostics', async () => { - const stockXml = - ''; +test('snapshotAndroid emits helper failure diagnostics', async () => { const helperAdb: AndroidAdbExecutor = async (args) => { if (args.includes('--show-versioncode')) { return { @@ -975,37 +847,24 @@ test('snapshotAndroid emits fallback and stock capture diagnostics', async () => stderr: '', }; } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: stockXml, stderr: '' }; - } + if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'force-stop') + return { exitCode: 0, stdout: '', stderr: '' }; return { exitCode: 1, stdout: '', stderr: 'helper unavailable' }; }; const diagnostics = await captureDiagnostics( - { session: 'snapshot-fallback', requestId: 'req-2', command: 'snapshot', debug: true }, + { session: 'snapshot-failure', requestId: 'req-2', command: 'snapshot', debug: true }, async () => { - await snapshotAndroid(device, { - helperAdb, - helperArtifact, - }); + await assert.rejects(() => snapshotAndroid(device, { helperAdb, helperArtifact })); return flushDiagnosticsToSessionFile({ force: true }); }, ); - assert.match(diagnostics, /android_snapshot_helper_fallback/); - assert.match(diagnostics, /android_snapshot_stock_capture/); + assert.match(diagnostics, /android_snapshot_helper_failed/); assert.match(diagnostics, /helper unavailable/); }); test('snapshotAndroid emits unavailable diagnostics when helper artifact is missing', async () => { - const stockXml = - ''; - const helperAdb: AndroidAdbExecutor = async (args) => { - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: stockXml, stderr: '' }; - } - throw new Error(`unexpected adb args: ${args.join(' ')}`); - }; const accessSpy = vi.spyOn(fs, 'access').mockRejectedValueOnce(new Error('helper missing')); try { @@ -1017,9 +876,10 @@ test('snapshotAndroid emits unavailable diagnostics when helper artifact is miss debug: true, }, async () => { - const result = await snapshotAndroid(device, { helperAdb }); - assert.equal(result.nodes[0]?.label, 'stock'); - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); + await assert.rejects( + () => snapshotAndroid(device), + /Android snapshot helper is unavailable/, + ); return flushDiagnosticsToSessionFile({ force: true }); }, ); @@ -1027,20 +887,16 @@ test('snapshotAndroid emits unavailable diagnostics when helper artifact is miss assert.match(diagnostics, /android_snapshot_helper_artifact_resolution/); assert.match(diagnostics, /android_snapshot_helper_unavailable/); assert.match(diagnostics, /artifact_not_found/); - assert.match(diagnostics, /android_snapshot_stock_capture/); } finally { accessSpy.mockRestore(); } }); -test('snapshotAndroid emits timeout fallback diagnostics when helper capture times out', async () => { - const stockXml = - ''; +test('snapshotAndroid emits timeout diagnostics when helper capture times out', async () => { const helperAdb = createHelperAdb({ instrument: async () => { throw new AppError('COMMAND_FAILED', 'helper capture timed out'); }, - stock: async () => ({ exitCode: 0, stdout: stockXml, stderr: '' }), }); const diagnostics = await captureDiagnostics( @@ -1051,20 +907,19 @@ test('snapshotAndroid emits timeout fallback diagnostics when helper capture tim debug: true, }, async () => { - const result = await snapshotAndroidWithHelper(helperAdb); - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.match(result.androidSnapshot.fallbackReason ?? '', /helper capture timed out/); + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb), + /Android snapshot helper failed: helper capture timed out/, + ); return flushDiagnosticsToSessionFile({ force: true }); }, ); - assert.match(diagnostics, /android_snapshot_helper_fallback/); + assert.match(diagnostics, /android_snapshot_helper_failed/); assert.match(diagnostics, /helper capture timed out/); - assert.match(diagnostics, /android_snapshot_stock_capture/); }); -test('snapshotAndroid skips stock fallback after structured helper timeout', async () => { - let stockAttempted = false; +test('snapshotAndroid preserves structured helper timeout guidance', async () => { const helperAdb = createHelperAdb({ instrument: async () => ({ exitCode: 1, @@ -1079,17 +934,13 @@ test('snapshotAndroid skips stock fallback after structured helper timeout', asy ].join('\n'), stderr: '', }), - stock: async () => { - stockAttempted = true; - throw new Error('stock fallback should not run'); - }, }); await assert.rejects( () => snapshotAndroidWithHelper(helperAdb), (error) => { assert.match((error as Error).message, /Timed out waiting for accessibility root/); - assert.match((error as Error).message, /Stock UIAutomator fallback was skipped/); + assert.match((error as Error).message, /Android snapshot helper failed/); assert.equal( (error as { details?: Record }).details?.hint, 'Android accessibility snapshots can be blocked by busy or continuously changing app UI. Use screenshot as visual truth after this timeout and report the busy UI if it persists.', @@ -1097,17 +948,11 @@ test('snapshotAndroid skips stock fallback after structured helper timeout', asy return true; }, ); - assert.equal(stockAttempted, false); }); -test('snapshotAndroid skips stock fallback after killed helper instrumentation', async () => { - let stockAttempted = false; +test('snapshotAndroid preserves killed helper instrumentation details', async () => { const helperAdb = createHelperAdb({ instrument: async () => ({ exitCode: 137, stdout: '', stderr: '' }), - stock: async () => { - stockAttempted = true; - throw new Error('stock fallback should not run'); - }, }); await assert.rejects( @@ -1117,17 +962,14 @@ test('snapshotAndroid skips stock fallback after killed helper instrumentation', (error as Error).message, /Android snapshot helper failed before returning parseable output/, ); - assert.match((error as Error).message, /Stock UIAutomator fallback was skipped/); + assert.match((error as Error).message, /Android snapshot helper failed/); assert.equal((error as { details?: Record }).details?.exitCode, 137); return true; }, ); - assert.equal(stockAttempted, false); }); -test('snapshotAndroid falls back to stock dump after unparseable helper output', async () => { - const stockXml = - ''; +test('snapshotAndroid fails closed after unparseable helper output', async () => { const calls: string[][] = []; const helperAdb: AndroidAdbExecutor = async (args) => { calls.push(args); @@ -1136,16 +978,12 @@ test('snapshotAndroid falls back to stock dump after unparseable helper output', if (args[0] === 'shell' && args[1] === 'am' && args[2] === 'force-stop') { return { exitCode: 0, stdout: '', stderr: '' }; } - if (args.includes('exec-out')) return { exitCode: 0, stdout: stockXml, stderr: '' }; throw new Error(`unexpected helper adb args: ${args.join(' ')}`); }; - const result = await snapshotAndroidWithHelper(helperAdb); - - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.match( - result.androidSnapshot.fallbackReason ?? '', - /Android snapshot helper output could not be parsed/, + await assert.rejects( + () => snapshotAndroidWithHelper(helperAdb), + /Android snapshot helper failed.*output could not be parsed/, ); assert.equal( calls.some((args) => args.includes('instrument')), @@ -1159,14 +997,12 @@ test('snapshotAndroid falls back to stock dump after unparseable helper output', ); assert.equal( calls.some((args) => args.includes('exec-out')), - true, + false, ); assert.equal(mockSleep.mock.calls.at(-1)?.[0], 150); }); -test('snapshotAndroid falls back to stock dump after helper adb timeout', async () => { - const stockXml = - ''; +test('snapshotAndroid fails closed after helper adb timeout', async () => { const helperAdb = createHelperAdb({ instrument: async (args) => { throw new AppError('COMMAND_FAILED', 'adb timed out after 8000ms', { @@ -1174,46 +1010,14 @@ test('snapshotAndroid falls back to stock dump after helper adb timeout', async timeoutMs: 8000, }); }, - stock: async () => ({ exitCode: 0, stdout: stockXml, stderr: '' }), }); - const result = await snapshotAndroidWithHelper(helperAdb); - - assert.equal(result.androidSnapshot.backend, 'uiautomator-dump'); - assert.match(result.androidSnapshot.fallbackReason ?? '', /adb timed out after 8000ms/); -}); - -test('snapshotAndroid preserves helper failure reason when stock fallback fails', async () => { - const helperAdb: AndroidAdbExecutor = async (args) => { - if (args.includes('--show-versioncode')) { - return { - exitCode: 0, - stdout: 'package:com.callstack.agentdevice.snapshothelper versionCode:13004', - stderr: '', - }; - } - if (args.includes('exec-out')) { - throw new AppError('COMMAND_FAILED', 'stock dump timed out', { hint: 'stock hint' }); - } - return { exitCode: 1, stdout: '', stderr: 'instrumentation failed' }; - }; - await assert.rejects( - () => - snapshotAndroid(device, { - helperAdb, - helperArtifact, - }), + () => snapshotAndroidWithHelper(helperAdb), (error) => { assert.ok(error instanceof AppError); - assert.match(error.message, /stock dump timed out/); - assert.match(error.message, /Android snapshot helper failed before stock fallback/); - assert.match(error.message, /failed before returning parseable output/); - assert.match( - String(error.details?.androidSnapshotHelperFallbackReason), - /Android snapshot helper failed before returning parseable output/, - ); - assert.equal(error.details?.hint, 'stock hint'); + assert.match(error.message, /Android snapshot helper failed: adb timed out after 8000ms/); + assert.equal(error.details?.androidSnapshotHelperFailureReason, 'adb timed out after 8000ms'); return true; }, ); @@ -1242,190 +1046,21 @@ test('snapshotAndroid re-probes helper install after helper capture failure', as stderr: '', }; } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: stockXml, stderr: '' }; - } throw new Error(`unexpected helper adb args: ${args.join(' ')}`); }; - const stockXml = - ''; const helperOptions = { helperAdb, helperArtifact, }; - const fallback = await snapshotAndroid(device, helperOptions); + await assert.rejects(() => snapshotAndroid(device, helperOptions)); const helper = await snapshotAndroid(device, helperOptions); - assert.equal(fallback.androidSnapshot.backend, 'uiautomator-dump'); assert.equal(helper.androidSnapshot.backend, 'android-helper'); assert.equal(helper.nodes[0]?.label, 'helper'); assert.equal(versionProbeCount, 2); }); -test('dumpUiHierarchy reads fallback XML when dump exits non-zero', async () => { - const xml = - ''; - - mockRunCmd.mockImplementation(async (_cmd, args, options) => { - if (args.includes('exec-out')) { - return { exitCode: 1, stdout: '', stderr: 'stream unavailable' }; - } - if ( - args.includes('uiautomator') && - args.includes('dump') && - args.includes('/sdcard/window_dump.xml') - ) { - if (options?.allowFailure !== true) { - throw new AppError('COMMAND_FAILED', 'adb exited with code 1', { - stderr: 'theme engine error', - }); - } - return { - exitCode: 1, - stdout: 'UI hierarchy dumped to: /sdcard/window_dump.xml', - stderr: 'theme engine error', - }; - } - if (args.includes('cat') && args.includes('/sdcard/window_dump.xml')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - const result = await dumpUiHierarchy(device); - const dumpCall = mockRunCmd.mock.calls.find(([, args]) => - args.includes('/sdcard/window_dump.xml'), - ); - const catCall = mockRunCmd.mock.calls.find( - ([, args]) => args.includes('cat') && args.includes('/sdcard/window_dump.xml'), - ); - - assert.equal(result, xml); - assert.deepEqual(dumpCall?.[2], { - allowFailure: true, - timeoutMs: 8000, - ...localAdbExecOptions, - }); - assert.deepEqual(catCall?.[2], localAdbExecOptions); -}); - -test('dumpUiHierarchy does not read a stale fallback file when dump fails without a path', async () => { - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (args.includes('exec-out')) { - return { exitCode: 137, stdout: 'Killed', stderr: '' }; - } - if ( - args.includes('uiautomator') && - args.includes('dump') && - args.includes('/sdcard/window_dump.xml') - ) { - return { exitCode: 137, stdout: 'Killed', stderr: '' }; - } - if (args.includes('cat') && args.includes('/sdcard/window_dump.xml')) { - throw new Error('cat should not read a stale dump file'); - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - await assert.rejects( - dumpUiHierarchy(device), - (error: unknown) => - error instanceof AppError && - error.code === 'COMMAND_FAILED' && - error.message.includes('did not return XML') && - error.details?.reason === 'missing_fresh_dump', - ); -}); - -test('dumpUiHierarchy retries when fallback dump file is temporarily missing', async () => { - const xml = ''; - let catAttempts = 0; - - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (args.includes('exec-out')) { - return { exitCode: 1, stdout: '', stderr: 'stream unavailable' }; - } - if ( - args.includes('uiautomator') && - args.includes('dump') && - args.includes('/sdcard/window_dump.xml') - ) { - return { - exitCode: 0, - stdout: 'UI hierarchy dumped to: /sdcard/window_dump.xml', - stderr: '', - }; - } - if (args.includes('cat') && args.includes('/sdcard/window_dump.xml')) { - catAttempts += 1; - if (catAttempts === 1) { - throw new AppError('COMMAND_FAILED', 'adb exited with code 1', { - stderr: 'cat: /sdcard/window_dump.xml: No such file or directory', - }); - } - return { exitCode: 0, stdout: xml, stderr: '' }; - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - const result = await dumpUiHierarchy(device); - - assert.equal(result, xml); - assert.equal(catAttempts, 2); - assert.equal( - mockRunCmd.mock.calls.filter( - ([, args]) => args.includes('uiautomator') && args.includes('/sdcard/window_dump.xml'), - ).length, - 2, - ); -}); - -test('dumpUiHierarchy explains timeout on looping Android animations', async () => { - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (args.includes('uiautomator')) { - throw adbTimeout(args); - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - await assert.rejects( - dumpUiHierarchy(device), - (error: unknown) => - error instanceof AppError && - error.message.includes('Android UI hierarchy dump timed out') && - typeof error.details?.hint === 'string' && - error.details.hint.includes('Android accessibility snapshots can be blocked') && - error.details.hint.includes('Use screenshot as visual truth after this timeout'), - ); -}); - -test('dumpUiHierarchy does not attach animation hint to non-dump timeouts', async () => { - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: '', stderr: '' }; - } - if (args.includes('uiautomator')) { - return { exitCode: 0, stdout: 'UI hierarchy dumped to: /sdcard/window_dump.xml', stderr: '' }; - } - if (args.includes('cat')) { - throw adbTimeout(args); - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - await assert.rejects( - dumpUiHierarchy(device), - (error: unknown) => - error instanceof AppError && - error.message === 'adb timed out after 8000ms' && - // Non-dump timeouts keep the generic classified adb-timeout hint instead - // of the looping-animation explanation reserved for uiautomator dump. - !String(error.details?.hint).includes('Android accessibility snapshots can be blocked') && - error.details?.adbFailure === 'timeout', - ); -}); - test('snapshotAndroid preserves hidden scroll content hints in interactive snapshots', async () => { const xml = ` @@ -1447,9 +1082,9 @@ test('snapshotAndroid preserves hidden scroll content hints in interactive snaps ' com.facebook.react.views.view.ReactViewGroup{c V.E...... ........ 0,636-390,804 #3}', ].join('\n'); - mockAndroidSnapshotXml(xml, dump); - - const result = await snapshotAndroid(device, { interactiveOnly: true }); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml, dump), { + interactiveOnly: true, + }); const scrollArea = result.nodes.find((node) => node.type === 'android.widget.ScrollView'); assert.ok(scrollArea); @@ -1470,9 +1105,9 @@ test('snapshotAndroid keeps generic-id scroll containers in interactive snapshot `; - mockAndroidSnapshotXml(xml); - - const result = await snapshotAndroid(device, { interactiveOnly: true }); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml, ''), { + interactiveOnly: true, + }); const scrollArea = result.nodes.find( (node) => node.type === 'android.widget.ScrollView' && @@ -1490,21 +1125,10 @@ test('snapshotAndroid skips activity dump when snapshot has no scrollable nodes' `; - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (isAndroidSdkVersionCommand(args)) { - return { exitCode: 0, stdout: '35', stderr: '' }; - } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - if (args.includes('dumpsys') && args.includes('activity') && args.includes('top')) { - throw new Error('dumpsys activity top should not run without scrollable nodes'); - } - throw new Error(`unexpected args: ${args.join(' ')}`); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml), { + interactiveOnly: true, }); - const result = await snapshotAndroid(device, { interactiveOnly: true }); - assert.equal(result.nodes.length, 1); assert.equal(result.nodes[0]?.label, 'Continue'); }); @@ -1519,21 +1143,10 @@ test('snapshotAndroid skips hidden content hints when disabled', async () => { `; - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (isAndroidSdkVersionCommand(args)) { - return { exitCode: 0, stdout: '35', stderr: '' }; - } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - if (args.includes('dumpsys') && args.includes('activity') && args.includes('top')) { - throw new Error('dumpsys activity top should not run when hints are disabled'); - } - throw new Error(`unexpected args: ${args.join(' ')}`); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml), { + includeHiddenContentHints: false, }); - const result = await snapshotAndroid(device, { includeHiddenContentHints: false }); - assert.equal( result.nodes.some((node) => node.type === 'android.widget.ScrollView'), true, @@ -1552,20 +1165,7 @@ test('snapshotAndroid uses helper scroll action hints without activity dump', as `; - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (isAndroidSdkVersionCommand(args)) { - return { exitCode: 0, stdout: '35', stderr: '' }; - } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - if (args.includes('dumpsys') && args.includes('activity') && args.includes('top')) { - throw new Error('dumpsys activity top should not run when helper action hints exist'); - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - const result = await snapshotAndroid(device); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml)); const scrollArea = result.nodes.find((node) => node.type === 'android.widget.ScrollView'); assert.ok(scrollArea); @@ -1585,20 +1185,7 @@ test('snapshotAndroid does not convert horizontal helper scroll action to vertic `; - mockRunCmd.mockImplementation(async (_cmd, args) => { - if (isAndroidSdkVersionCommand(args)) { - return { exitCode: 0, stdout: '35', stderr: '' }; - } - if (args.includes('exec-out')) { - return { exitCode: 0, stdout: xml, stderr: '' }; - } - if (args.includes('dumpsys') && args.includes('activity') && args.includes('top')) { - throw new Error('dumpsys activity top should not run when helper action hints exist'); - } - throw new Error(`unexpected args: ${args.join(' ')}`); - }); - - const result = await snapshotAndroid(device); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml)); const scrollArea = result.nodes.find( (node) => node.type === 'android.widget.HorizontalScrollView', ); @@ -1621,9 +1208,9 @@ test('snapshotAndroid derives hidden content hints for interactive snapshots fro `; - mockAndroidSnapshotXml(xml); - - const result = await snapshotAndroid(device, { interactiveOnly: true }); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml), { + interactiveOnly: true, + }); const scrollArea = result.nodes.find((node) => node.type === 'android.widget.ScrollView'); assert.ok(scrollArea); @@ -1652,9 +1239,9 @@ test('snapshotAndroid omits zero-area interactive nodes from interactive snapsho `; - mockAndroidSnapshotXml(xml); - - const result = await snapshotAndroid(device, { interactiveOnly: true }); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml), { + interactiveOnly: true, + }); assert.equal( result.nodes.some((node) => node.label === 'Visible action'), @@ -1689,9 +1276,9 @@ test('snapshotAndroid preserves bottomed-out hidden-above hints in interactive s ' com.facebook.react.views.view.ReactViewGroup{c V.E...... ........ 0,636-390,804 #3}', ].join('\n'); - mockAndroidSnapshotXml(xml, dump); - - const result = await snapshotAndroid(device, { interactiveOnly: true }); + const result = await snapshotAndroidWithHelper(androidSnapshotHelperAdb(xml, dump), { + interactiveOnly: true, + }); const scrollArea = result.nodes.find( (node) => node.hiddenContentAbove === true || node.hiddenContentBelow === true, ); diff --git a/src/platforms/android/__tests__/touch-executor-helper.test.ts b/src/platforms/android/__tests__/touch-executor-helper.test.ts index 83cb1cceb..ae51f529e 100644 --- a/src/platforms/android/__tests__/touch-executor-helper.test.ts +++ b/src/platforms/android/__tests__/touch-executor-helper.test.ts @@ -19,15 +19,15 @@ vi.mock('../snapshot-helper.ts', async (importOriginal) => ({ vi.mock('../helper-package-install.ts', async (importOriginal) => { const actual = await importOriginal(); - const fixture = await import('./android-helper-artifact.fixtures.ts'); + const fixture = await import('../../../__tests__/test-utils/android-snapshot-helper.ts'); const multitouchFixture = await import('./multitouch-helper.fixtures.ts'); return { ...actual, resolveAndroidHelperArtifact: vi.fn(async () => ({ - apkPath: fixture.ANDROID_HELPER_FIXTURE_APK_PATH, + apkPath: fixture.ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT.apkPath, manifest: { ...multitouchFixture.ANDROID_MULTITOUCH_HELPER_MANIFEST, - sha256: fixture.ANDROID_HELPER_FIXTURE_APK_SHA256, + sha256: fixture.ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT.manifest.sha256, }, })), }; diff --git a/src/platforms/android/adb-executor.ts b/src/platforms/android/adb-executor.ts index 0531f14a6..3ae326501 100644 --- a/src/platforms/android/adb-executor.ts +++ b/src/platforms/android/adb-executor.ts @@ -2,6 +2,7 @@ import { AsyncLocalStorage } from 'node:async_hooks'; import type { Readable, Writable } from 'node:stream'; import type { DeviceInfo } from '../../kernel/device.ts'; import type { Rect } from '../../kernel/snapshot.ts'; +import type { AndroidSnapshotHelperArtifact } from './snapshot-helper-types.ts'; import type { AndroidProviderTouchPlan } from './touch-plan.ts'; import { coerceExecResult, @@ -143,6 +144,7 @@ type AndroidAdbProviderBase = { install?: AndroidAdbInstaller; installBundle?: AndroidBundleInstaller; text?: AndroidTextInjector; + snapshotHelperArtifact?: AndroidSnapshotHelperArtifact; }; type AndroidTouchCapabilities = diff --git a/src/platforms/android/alert-detection.ts b/src/platforms/android/alert-detection.ts index a68649b6d..ca1548678 100644 --- a/src/platforms/android/alert-detection.ts +++ b/src/platforms/android/alert-detection.ts @@ -42,6 +42,15 @@ const ACCEPT_LABEL_PATTERN = const DISMISS_LABEL_PATTERN = /^(?:cancel|deny|don.t allow|don’t allow|not now|no|dismiss|close|close app|later|skip)$/i; +export function classifyAndroidAlertIdentifier( + identifier: string | null, +): 'button' | 'content' | undefined { + if (identifier === null) return undefined; + if (ANDROID_ALERT_BUTTON_ID_PATTERN.test(identifier)) return 'button'; + if (ANDROID_ALERT_ID_PATTERN.test(identifier)) return 'content'; + return undefined; +} + export function findAndroidAlertCandidate(nodes: RawSnapshotNode[]): AndroidAlertCandidate | null { const candidate = findAndroidAlertNodes(nodes); const candidateNodes = candidate.nodes; diff --git a/src/platforms/android/snapshot-content-recovery.ts b/src/platforms/android/snapshot-content-recovery.ts index 747e4a861..e2e1f3879 100644 --- a/src/platforms/android/snapshot-content-recovery.ts +++ b/src/platforms/android/snapshot-content-recovery.ts @@ -1,20 +1,23 @@ import type { AndroidSnapshotBackendMetadata } from './snapshot-types.ts'; import { isAndroidInputMethodOwnedNode } from '../../contracts/android-input-ownership.ts'; +import { classifyAndroidAlertIdentifier } from './alert-detection.ts'; import { androidUiNodes, type AndroidUiNodeMetadata } from './ui-hierarchy.ts'; const ANDROID_WINDOW_TYPE_APPLICATION = 1; const MAX_REPORTED_WINDOW_TYPES = 8; const MIN_FOREGROUND_APP_MEANINGFUL_NODES = 2; const MIN_INPUT_METHOD_MEANINGFUL_NODES = 2; +const ANDROID_SYSTEM_UI_PACKAGE = 'com.android.systemui'; const ANDROID_SYSTEM_PACKAGES = new Set(['android', 'com.android.systemui']); const INSUFFICIENT_APP_CONTENT_REASON = 'Android snapshot helper returned insufficient application window content'; export type AndroidHelperContentRecoveryDecision = { reason: 'empty-helper-output' | 'system-window-only' | 'content-poor-app-window'; - fallbackReason: string; + failureReason: string; diagnostics: { helperNodeCount: number; + helperSystemUiNodeCount: number; helperWindowRootCount: number; helperApplicationWindowRootCount: number; helperMeaningfulNodeCount: number; @@ -46,6 +49,7 @@ export function classifyAndroidHelperContentRecovery( ); } + if (hasRecognizedAndroidAlertSurface(summary)) return undefined; if (isForegroundAppContentHiddenByInputMethod(summary)) return undefined; if (isForegroundAppContentPoor(summary)) { return buildRecoveryDecision( @@ -103,7 +107,9 @@ function isForegroundAppContentHiddenByInputMethod(summary: AndroidHelperXmlSumm function isForegroundAppContentPoor(summary: AndroidHelperXmlSummary): boolean { const foregroundCount = summary.foregroundAppMeaningfulNodeCount; if (foregroundCount === undefined) return false; - if (foregroundCount === 0) return true; + if (foregroundCount === 0) { + return summary.applicationMeaningfulNodeCount < MIN_FOREGROUND_APP_MEANINGFUL_NODES; + } return ( foregroundCount < MIN_FOREGROUND_APP_MEANINGFUL_NODES && summary.meaningfulNodeCount > foregroundCount @@ -131,30 +137,42 @@ function isWindowlessMultiWindowContentPoor( } function isSystemWindowOnly(summary: AndroidHelperXmlSummary): boolean { - return summary.windowRootCount > 0 && summary.applicationWindowRootCount === 0; + return ( + (summary.windowRootCount > 0 && summary.applicationWindowRootCount === 0) || + (summary.windowRootCount === 0 && + summary.nodeCount > 0 && + summary.systemUiNodeCount === summary.nodeCount) + ); +} + +function hasRecognizedAndroidAlertSurface(summary: AndroidHelperXmlSummary): boolean { + return summary.hasAlertButtonIdentifier && summary.hasAlertContentIdentifier; } function buildRecoveryDecision( summary: AndroidHelperXmlSummary, metadata: AndroidSnapshotBackendMetadata, reason: AndroidHelperContentRecoveryDecision['reason'], - fallbackReason: string, + failureReason: string, ): AndroidHelperContentRecoveryDecision { return { reason, - fallbackReason, + failureReason, diagnostics: buildRecoveryDiagnostics(summary, metadata), }; } type AndroidHelperXmlSummary = { nodeCount: number; + systemUiNodeCount: number; windowRootCount: number; applicationWindowRootCount: number; meaningfulNodeCount: number; applicationMeaningfulNodeCount: number; nonSystemMeaningfulNodeCount: number; inputMethodMeaningfulNodeCount: number; + hasAlertButtonIdentifier: boolean; + hasAlertContentIdentifier: boolean; foregroundAppPackage?: string; foregroundAppMeaningfulNodeCount?: number; windowTypes: number[]; @@ -183,12 +201,15 @@ function createAndroidHelperXmlSummaryState( ): AndroidHelperXmlSummaryState { return { nodeCount: 0, + systemUiNodeCount: 0, windowRootCount: 0, applicationWindowRootCount: 0, meaningfulNodeCount: 0, applicationMeaningfulNodeCount: 0, nonSystemMeaningfulNodeCount: 0, inputMethodMeaningfulNodeCount: 0, + hasAlertButtonIdentifier: false, + hasAlertContentIdentifier: false, ...(foregroundAppPackage !== undefined ? { foregroundAppPackage, foregroundAppMeaningfulNodeCount: 0 } : {}), @@ -201,10 +222,21 @@ function recordAndroidHelperSummaryNode( node: AndroidUiNodeMetadata, ): void { summary.nodeCount += 1; + if (node.packageName === ANDROID_SYSTEM_UI_PACKAGE) summary.systemUiNodeCount += 1; + if (node.visibleToUser !== false) recordAndroidAlertIdentifier(summary, node.resourceId); recordAndroidHelperWindowNode(summary, node); recordAndroidHelperMeaningfulNode(summary, node); } +function recordAndroidAlertIdentifier( + summary: AndroidHelperXmlSummaryState, + resourceId: string | null, +): void { + const kind = classifyAndroidAlertIdentifier(resourceId); + if (kind === 'button') summary.hasAlertButtonIdentifier = true; + if (kind === 'content') summary.hasAlertContentIdentifier = true; +} + function recordAndroidHelperWindowNode( summary: AndroidHelperXmlSummaryState, node: AndroidUiNodeMetadata, @@ -226,7 +258,10 @@ function recordAndroidHelperMeaningfulNode( if (!isMeaningfulContentNode(node)) return; summary.meaningfulNodeCount += 1; - if (summary.currentWindowType === ANDROID_WINDOW_TYPE_APPLICATION) { + if ( + summary.currentWindowType === ANDROID_WINDOW_TYPE_APPLICATION && + !isAndroidSystemPackage(node.packageName) + ) { summary.applicationMeaningfulNodeCount += 1; } if (!isAndroidSystemPackage(node.packageName)) { @@ -253,12 +288,15 @@ function finalizeAndroidHelperXmlSummary( ): AndroidHelperXmlSummary { return { nodeCount: summary.nodeCount, + systemUiNodeCount: summary.systemUiNodeCount, windowRootCount: summary.windowRootCount, applicationWindowRootCount: summary.applicationWindowRootCount, meaningfulNodeCount: summary.meaningfulNodeCount, applicationMeaningfulNodeCount: summary.applicationMeaningfulNodeCount, nonSystemMeaningfulNodeCount: summary.nonSystemMeaningfulNodeCount, inputMethodMeaningfulNodeCount: summary.inputMethodMeaningfulNodeCount, + hasAlertButtonIdentifier: summary.hasAlertButtonIdentifier, + hasAlertContentIdentifier: summary.hasAlertContentIdentifier, ...(summary.foregroundAppPackage !== undefined ? { foregroundAppPackage: summary.foregroundAppPackage, @@ -288,6 +326,7 @@ function buildRecoveryDiagnostics( ): AndroidHelperContentRecoveryDecision['diagnostics'] { return { helperNodeCount: summary.nodeCount, + helperSystemUiNodeCount: summary.systemUiNodeCount, helperWindowRootCount: summary.windowRootCount, helperApplicationWindowRootCount: summary.applicationWindowRootCount, helperMeaningfulNodeCount: summary.meaningfulNodeCount, diff --git a/src/platforms/android/snapshot-types.ts b/src/platforms/android/snapshot-types.ts index b25870e28..85e537b17 100644 --- a/src/platforms/android/snapshot-types.ts +++ b/src/platforms/android/snapshot-types.ts @@ -5,12 +5,11 @@ import type { } from './snapshot-helper-types.ts'; export type AndroidSnapshotBackendMetadata = { - backend: 'android-helper' | 'uiautomator-dump'; + backend: 'android-helper'; helperVersion?: string; helperApiVersion?: string; helperTransport?: AndroidSnapshotHelperTransport; helperSessionReused?: boolean; - fallbackReason?: string; installReason?: AndroidSnapshotHelperInstallReason; waitForIdleTimeoutMs?: number; waitForIdleQuietMs?: number; diff --git a/src/platforms/android/snapshot.ts b/src/platforms/android/snapshot.ts index b7d33bf77..c0d06872d 100644 --- a/src/platforms/android/snapshot.ts +++ b/src/platforms/android/snapshot.ts @@ -1,6 +1,5 @@ import fs from 'node:fs/promises'; import path from 'node:path'; -import { withRetry } from '../../utils/retry.ts'; import { AppError, normalizeError, toAppErrorCode } from '../../kernel/errors.ts'; import { emitDiagnostic, withDiagnosticTimer } from '../../utils/diagnostics.ts'; import type { DeviceInfo } from '../../kernel/device.ts'; @@ -22,8 +21,6 @@ import { type AndroidUiHierarchy, } from './ui-hierarchy.ts'; import { - androidAdbResultError, - classifyAdbFailure, resolveAndroidAdbExecutor, resolveAndroidAdbProvider, type AndroidAdbProvider, @@ -51,20 +48,11 @@ import { type AndroidHelperContentRecoveryDecision, } from './snapshot-content-recovery.ts'; -const UI_HIERARCHY_DUMP_TIMEOUT_MS = 8_000; const HELPER_INSTALL_TIMEOUT_MS = 30_000; const HELPER_CAPTURE_TIMEOUT_MS = 5_000; const HELPER_COMMAND_TIMEOUT_MS = 30_000; const HELPER_RUNTIME_RESET_DELAY_MS = 150; const HELPER_RUNTIME_RESET_TIMEOUT_MS = 2_000; -// Transient adb transport families (device offline/not found, transport error, -// connection reset, broken pipe) come from the shared classifier; these extras -// are snapshot-specific races (dump timeouts, dump-file reads) worth retrying. -const SNAPSHOT_ONLY_RETRYABLE_ADB_STDERR_PATTERNS = [ - 'timed out', - 'no such file or directory', -] as const; - export type AndroidSnapshotOptions = SnapshotOptions & { appBundleId?: string; helperArtifact?: AndroidSnapshotHelperArtifact; @@ -192,20 +180,24 @@ async function captureAndroidUiHierarchy( options: AndroidSnapshotOptions, adb: AndroidAdbExecutor, ): Promise<{ xml: string; metadata: AndroidSnapshotBackendMetadata }> { + const adbProvider = resolveAndroidAdbProvider(device, options.helperAdb); const helper = await withDiagnosticTimer( 'android_snapshot_helper_artifact_resolution', - async () => await resolveAndroidSnapshotHelperArtifact(options.helperArtifact), + async () => + await resolveAndroidSnapshotHelperArtifact( + options.helperArtifact ?? adbProvider.snapshotHelperArtifact, + ), ); if (helper.artifact) { return await captureAndroidUiHierarchyWithHelper(device, options, adb, helper.artifact); } emitDiagnostic({ - level: helper.fallbackReason ? 'warn' : 'info', + level: 'error', phase: 'android_snapshot_helper_unavailable', - data: { reason: helper.fallbackReason ?? 'artifact_not_found' }, + data: { reason: helper.errorReason ?? 'artifact_not_found' }, }); - return await captureStockUiHierarchy(device, helper.fallbackReason, adb); + throw androidSnapshotHelperUnavailableError(helper.errorReason); } async function captureAndroidUiHierarchyWithHelper( @@ -218,43 +210,45 @@ async function captureAndroidUiHierarchyWithHelper( const adbProvider = resolveAndroidAdbProvider(device, options.helperAdb); const commandScopedHelperSession = options.helperSessionScope !== 'daemon-session'; try { - const install = await installAndroidSnapshotHelper( - options, - adb, - adbProvider, - artifact, - helperDeviceKey, - ); - if (install.installed) { - await stopAndroidSnapshotHelperSession(helperDeviceKey); + let helperCapture: { xml: string; metadata: AndroidSnapshotBackendMetadata }; + try { + const install = await installAndroidSnapshotHelper( + options, + adb, + adbProvider, + artifact, + helperDeviceKey, + ); + if (install.installed) { + await stopAndroidSnapshotHelperSession(helperDeviceKey); + } + const capture = await captureAndroidUiHierarchyFromHelper({ + options, + adb, + adbProvider, + artifact, + helperDeviceKey, + }); + helperCapture = formatAndroidHelperCaptureResult(capture, artifact, install.reason); + } catch (error) { + return await rejectAndroidHelperCaptureFailure({ + error, + helperDeviceKey, + artifact, + adb, + }); } - const capture = await captureAndroidUiHierarchyFromHelper({ - options, - adb, - adbProvider, - artifact, - helperDeviceKey, - }); - const helperCapture = formatAndroidHelperCaptureResult(capture, artifact, install.reason); + const contentRecovery = classifyAndroidHelperContentRecovery( helperCapture.xml, helperCapture.metadata, { foregroundAppPackage: options.appBundleId }, ); if (!contentRecovery) return helperCapture; - return await recoverAndroidHelperContentUnavailable({ + return await rejectAndroidHelperContentUnavailable({ contentRecovery, helperDeviceKey, artifact, - device, - adb, - }); - } catch (error) { - return await recoverAndroidHelperCaptureFailure({ - error, - helperDeviceKey, - artifact, - device, adb, }); } finally { @@ -384,44 +378,41 @@ function formatAndroidHelperCaptureResult( }; } -async function recoverAndroidHelperContentUnavailable(params: { +async function rejectAndroidHelperContentUnavailable(params: { contentRecovery: AndroidHelperContentRecoveryDecision; helperDeviceKey: string; artifact: AndroidSnapshotHelperArtifact; - device: DeviceInfo; adb: AndroidAdbExecutor; }): Promise<{ xml: string; metadata: AndroidSnapshotBackendMetadata }> { emitDiagnostic({ - level: 'warn', - phase: 'android_snapshot_helper_content_fallback', + level: 'error', + phase: 'android_snapshot_helper_content_invalid', data: { reason: params.contentRecovery.reason, - fallbackReason: params.contentRecovery.fallbackReason, + failureReason: params.contentRecovery.failureReason, ...params.contentRecovery.diagnostics, }, }); await resetAndroidSnapshotHelperRuntime(params.adb, params.artifact.manifest.packageName); - return await captureStockUiHierarchy( - params.device, - params.contentRecovery.fallbackReason, - params.adb, - ); + throw new AppError('COMMAND_FAILED', params.contentRecovery.failureReason, { + ...params.contentRecovery.diagnostics, + androidSnapshotHelperFailureReason: params.contentRecovery.reason, + retriable: true, + hint: 'Retry after the app UI stabilizes. If this persists, capture a screenshot and report the helper diagnostics; agent-device does not substitute a second snapshot engine.', + }); } -async function recoverAndroidHelperCaptureFailure(params: { +async function rejectAndroidHelperCaptureFailure(params: { error: unknown; helperDeviceKey: string; artifact: AndroidSnapshotHelperArtifact; - device: DeviceInfo; adb: AndroidAdbExecutor; }): Promise<{ xml: string; metadata: AndroidSnapshotBackendMetadata }> { - const busyError = formatAndroidSnapshotHelperBusyError(params.error); - if (busyError) throw busyError; - const fallbackReason = formatAndroidSnapshotHelperFallbackReason(params.error); + const failureReason = formatAndroidSnapshotHelperFailureReason(params.error); emitDiagnostic({ - level: 'warn', - phase: 'android_snapshot_helper_fallback', - data: { reason: fallbackReason }, + level: 'error', + phase: 'android_snapshot_helper_failed', + data: { reason: failureReason }, }); await stopAndroidSnapshotHelperSession(params.helperDeviceKey); await resetAndroidSnapshotHelperRuntime(params.adb, params.artifact.manifest.packageName); @@ -430,7 +421,7 @@ async function recoverAndroidHelperCaptureFailure(params: { packageName: params.artifact.manifest.packageName, versionCode: params.artifact.manifest.versionCode, }); - return await captureStockUiHierarchy(params.device, fallbackReason, params.adb); + throw androidSnapshotHelperCaptureError(params.error, failureReason); } async function resetAndroidSnapshotHelperRuntime( @@ -457,7 +448,7 @@ async function resetAndroidSnapshotHelperRuntime( } } -function formatAndroidSnapshotHelperFallbackReason(error: unknown): string { +function formatAndroidSnapshotHelperFailureReason(error: unknown): string { const normalized = normalizeError(error); const helperMessage = readHelperMessage(normalized.details?.helper); if (helperMessage && helperMessage !== normalized.message) { @@ -470,22 +461,21 @@ function formatAndroidSnapshotHelperFallbackReason(error: unknown): string { return firstLine ? `${normalized.message}: ${firstLine}` : normalized.message; } -function formatAndroidSnapshotHelperBusyError(error: unknown): AppError | undefined { +function androidSnapshotHelperCaptureError(error: unknown, reason: string): AppError { const normalized = normalizeError(error); - if ( - !isStructuredHelperTimeout(normalized.details?.helper, normalized.message) && - !isKilledHelperInstrumentationFailure(normalized) - ) { - return undefined; - } - const reason = formatAndroidSnapshotHelperFallbackReason(error); - const hint = - 'Android accessibility snapshots can be blocked by busy or continuously changing app UI. Use screenshot as visual truth after this timeout and report the busy UI if it persists.'; + const busy = + isStructuredHelperTimeout(normalized.details?.helper, normalized.message) || + isKilledHelperInstrumentationFailure(normalized); + const hint = busy + ? 'Android accessibility snapshots can be blocked by busy or continuously changing app UI. Use screenshot as visual truth after this timeout and report the busy UI if it persists.' + : (normalized.hint ?? + 'Retry once. If the helper still fails, run agent-device doctor and report the diagnostic log; agent-device does not substitute a second snapshot engine.'); return new AppError( toAppErrorCode(normalized.code), - `${reason}. Stock UIAutomator fallback was skipped because this usually means the Android accessibility tree is busy or stalled.`, + `Android snapshot helper failed: ${reason}`, { ...normalized.details, + androidSnapshotHelperFailureReason: reason, hint, }, error, @@ -517,7 +507,7 @@ function isStructuredHelperTimeout(helper: unknown, fallbackMessage: string): bo async function resolveAndroidSnapshotHelperArtifact( explicitArtifact?: AndroidSnapshotHelperArtifact, -): Promise<{ artifact?: AndroidSnapshotHelperArtifact; fallbackReason?: string }> { +): Promise<{ artifact?: AndroidSnapshotHelperArtifact; errorReason?: string }> { if (explicitArtifact) { return { artifact: explicitArtifact }; } @@ -546,55 +536,16 @@ async function resolveAndroidSnapshotHelperArtifact( await fs.access(apkPath); return { artifact: { apkPath, manifest } }; } catch (error) { - return { fallbackReason: normalizeError(error).message }; - } -} - -async function captureStockUiHierarchy( - device: DeviceInfo, - fallbackReason?: string, - adb?: AndroidAdbExecutor, -): Promise<{ xml: string; metadata: AndroidSnapshotBackendMetadata }> { - let xml: string; - try { - xml = await withDiagnosticTimer( - 'android_snapshot_stock_capture', - async () => await dumpUiHierarchy(device, adb), - { - fallbackReason, - timeoutMs: UI_HIERARCHY_DUMP_TIMEOUT_MS, - }, - ); - } catch (error) { - if (fallbackReason) { - throw enrichStockSnapshotFailureWithHelperReason(error, fallbackReason); - } - throw error; + return { errorReason: normalizeError(error).message }; } - return { - xml, - metadata: { - backend: 'uiautomator-dump', - ...(fallbackReason ? { fallbackReason } : {}), - }, - }; } -function enrichStockSnapshotFailureWithHelperReason( - error: unknown, - fallbackReason: string, -): AppError { - const normalized = normalizeError(error); - return new AppError( - toAppErrorCode(normalized.code), - `${normalized.message} Android snapshot helper failed before stock fallback: ${fallbackReason}`, - { - ...normalized.details, - androidSnapshotHelperFallbackReason: fallbackReason, - ...(normalized.hint ? { hint: normalized.hint } : {}), - }, - error, - ); +function androidSnapshotHelperUnavailableError(errorReason: string | undefined): AppError { + const reason = errorReason ?? 'the bundled helper artifact was not found'; + return new AppError('COMMAND_FAILED', `Android snapshot helper is unavailable: ${reason}`, { + androidSnapshotHelperFailureReason: reason, + hint: 'For a source checkout, run pnpm build:android. For a packaged install, reinstall agent-device; the Android snapshot helper must ship with the package.', + }); } async function deriveScrollableContentHintsIfNeeded( @@ -640,110 +591,6 @@ function collectExistingHiddenContentHints( return hintsByIndex; } -export async function dumpUiHierarchy( - device: DeviceInfo, - adb = resolveAndroidAdbExecutor(device), -): Promise { - try { - return await withRetry(() => dumpUiHierarchyOnce(adb), { - shouldRetry: isRetryableAdbError, - }); - } catch (error) { - if (isUiHierarchyDumpTimeout(error)) { - const hint = - 'Android accessibility snapshots can be blocked by busy or continuously changing app UI. Use screenshot as visual truth after this timeout. Stock Android UIAutomator may still time out on app-owned infinite animations.'; - throw new AppError( - 'COMMAND_FAILED', - `Android UI hierarchy dump timed out while waiting for the UI to become idle. ${hint}`, - { - ...(error.details ?? {}), - hint, - }, - error, - ); - } - throw error; - } -} - -async function dumpUiHierarchyOnce(adb: AndroidAdbExecutor): Promise { - // Preferred: stream XML directly to stdout, avoiding file I/O race conditions. - const streamed = await adb(['exec-out', 'uiautomator', 'dump', '/dev/tty'], { - allowFailure: true, - timeoutMs: UI_HIERARCHY_DUMP_TIMEOUT_MS, - }); - const fromStream = extractUiDumpXml(streamed.stdout, streamed.stderr); - if (fromStream) return fromStream; - - // Fallback: dump to file and read back. - // If `cat` fails with "no such file", the outer withRetry (via isRetryableAdbError) handles it. - const dumpPath = '/sdcard/window_dump.xml'; - const dumpResult = await adb(['shell', 'uiautomator', 'dump', dumpPath], { - allowFailure: true, - timeoutMs: UI_HIERARCHY_DUMP_TIMEOUT_MS, - }); - const reportedPath = readDumpPath(dumpResult.stdout, dumpResult.stderr); - if (dumpResult.exitCode !== 0 && !reportedPath) { - throw androidAdbResultError('uiautomator dump did not return XML', dumpResult, { - reason: 'missing_fresh_dump', - }); - } - const actualPath = reportedPath ?? dumpPath; - - const result = await adb(['shell', 'cat', actualPath]); - const xml = extractUiDumpXml(result.stdout, result.stderr); - if (!xml) { - throw new AppError('COMMAND_FAILED', 'uiautomator dump did not return XML', { - stdout: result.stdout, - stderr: result.stderr, - }); - } - return xml; -} - -function readDumpPath(stdout: string, stderr: string): string | undefined { - const text = `${stdout}\n${stderr}`; - const match = /dumped to:\s*(\S+)/i.exec(text); - return match?.[1]; -} - -function extractUiDumpXml(stdout: string, stderr: string): string | null { - const text = `${stdout}\n${stderr}`; - const start = text.indexOf('= 0 ? start : text.indexOf(''); - if (end < 0 || end < hierarchyStart) return null; - const xml = text.slice(hierarchyStart, end + ''.length).trim(); - return xml.length > 0 ? xml : null; -} - -function isRetryableAdbError(err: unknown): boolean { - if (!(err instanceof AppError)) return false; - if (err.code !== 'COMMAND_FAILED') return false; - const rawStderr = err.details?.stderr; - const stderr = (typeof rawStderr === 'string' ? rawStderr : '').toLowerCase(); - if (classifyAdbFailure(stderr)?.retriable === true) return true; - return SNAPSHOT_ONLY_RETRYABLE_ADB_STDERR_PATTERNS.some((pattern) => stderr.includes(pattern)); -} - -function isUiHierarchyDumpTimeout(err: unknown): err is AppError { - if (!(err instanceof AppError)) return false; - if (err.code !== 'COMMAND_FAILED') return false; - const timeoutMs = err.details?.timeoutMs; - if (typeof timeoutMs !== 'number') return false; - return err.details?.cmd === 'adb' && isUiAutomatorDumpArgs(err.details?.args); -} - -function isUiAutomatorDumpArgs(rawArgs: unknown): boolean { - const args = Array.isArray(rawArgs) - ? rawArgs.map(String) - : typeof rawArgs === 'string' - ? rawArgs.split(/\s+/) - : []; - return args.includes('uiautomator') && args.includes('dump'); -} - async function dumpActivityTop( device: DeviceInfo, adb = resolveAndroidAdbExecutor(device), diff --git a/src/platforms/apple/core/__tests__/index.test.ts b/src/platforms/apple/core/__tests__/index.test.ts index fbfc315a2..3744e0706 100644 --- a/src/platforms/apple/core/__tests__/index.test.ts +++ b/src/platforms/apple/core/__tests__/index.test.ts @@ -3421,6 +3421,7 @@ function singlePanPlan(): Extract { return { topology: 'single', intent: 'pan', + executionProfile: 'timed-pan', durationMs: 500, viewport: { x: 0, y: 0, width: 400, height: 800 }, pointers: [ diff --git a/src/replay/__tests__/plan-digest.test.ts b/src/replay/__tests__/plan-digest.test.ts index 2a9fb611d..160cae0e0 100644 --- a/src/replay/__tests__/plan-digest.test.ts +++ b/src/replay/__tests__/plan-digest.test.ts @@ -72,42 +72,6 @@ test('computeReplayPlanDigest changes when platform-conditioned metadata changes assert.notEqual(ios, android); }); -test('computeReplayPlanDigest folds runtime control-flow shape (retry) into the digest', () => { - const retryOne: SessionAction = action({ - command: 'back', - positionals: [], - replayControl: { kind: 'retry', maxRetries: 1, actions: [action({ command: 'back' })] }, - }); - const retryTwo: SessionAction = { - ...retryOne, - replayControl: { kind: 'retry', maxRetries: 2, actions: [action({ command: 'back' })] }, - }; - assert.notEqual(digestFor([retryOne]), digestFor([retryTwo])); -}); - -test('computeReplayPlanDigest folds runtime control-flow shape (maestroRunFlowWhen) into the digest', () => { - const base: SessionAction = action({ - command: 'back', - positionals: [], - replayControl: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: 'label="Continue"', - actions: [action({ command: 'back' })], - }, - }); - const changedSelector: SessionAction = { - ...base, - replayControl: { - kind: 'maestroRunFlowWhen', - mode: 'visible', - selector: 'label="Skip"', - actions: [action({ command: 'back' })], - }, - }; - assert.notEqual(digestFor([base]), digestFor([changedSelector])); -}); - test('computeReplayPlanDigest changes when an execution runtime hint changes', () => { const runtime = { platform: 'ios' as const, diff --git a/src/replay/control-flow-runtime.ts b/src/replay/control-flow-runtime.ts deleted file mode 100644 index af64b25fb..000000000 --- a/src/replay/control-flow-runtime.ts +++ /dev/null @@ -1,59 +0,0 @@ -import type { DaemonResponse, ReplayControlActionSource, SessionAction } from '../daemon/types.ts'; - -export type ReplayActionBlockInvoker = (params: { - action: SessionAction; - line: number; - step: number; - /** From `replayControl.actionSources`; `undefined` = the wrapper's own file. */ - sourcePath?: string; -}) => Promise; - -export async function invokeReplayActionBlock(params: { - actions: SessionAction[]; - actionSources?: (ReplayControlActionSource | undefined)[]; - line: number; - step: number; - invokeReplayAction: ReplayActionBlockInvoker; -}): Promise { - for (const [index, action] of params.actions.entries()) { - const source = params.actionSources?.[index]; - const response = await params.invokeReplayAction({ - action, - line: source?.line ?? params.line, - step: params.step + index / 1000, - ...(source?.path ? { sourcePath: source.path } : {}), - }); - if (!response.ok) return response; - } - return { ok: true, data: { ran: params.actions.length } }; -} - -export async function invokeReplayRetryBlock(params: { - actions: SessionAction[]; - actionSources?: (ReplayControlActionSource | undefined)[]; - maxRetries: number; - line: number; - step: number; - invokeReplayAction: ReplayActionBlockInvoker; -}): Promise { - let lastResponse: DaemonResponse | undefined; - for (let attempt = 0; attempt <= params.maxRetries; attempt += 1) { - const response = await invokeReplayActionBlock({ - actions: params.actions, - actionSources: params.actionSources, - line: params.line, - step: params.step + attempt, - invokeReplayAction: params.invokeReplayAction, - }); - if (response.ok) { - return { ok: true, data: { attempts: attempt + 1, retried: attempt > 0 } }; - } - lastResponse = response; - } - return ( - lastResponse ?? { - ok: false, - error: { code: 'COMMAND_FAILED', message: 'retry commands failed.' }, - } - ); -} diff --git a/src/replay/plan-digest.ts b/src/replay/plan-digest.ts index 6d08ccb9a..7d13e46aa 100644 --- a/src/replay/plan-digest.ts +++ b/src/replay/plan-digest.ts @@ -3,15 +3,10 @@ import type { SessionAction } from '../daemon/types.ts'; /** * ADR 0012 decision 4 / migration step 5: `planDigest` is SHA-256 over the - * canonical fully expanded plan — the SAME `actions` array the replay - * runtime iterates, already flattened at parse time (static includes, - * platform conditions, and fixed-count repeats expand before this point; - * see `parseReplayInput`/`parseMaestroReplayFlow`). Runtime-only control flow - * (`retry`, `maestroRunFlowWhen`) stays a single plan entry — its shape - * (kind/mode/selector/maxRetries and nested action shapes) is folded into the - * digest so an edit inside a control-flow block still changes the digest, - * even though its nested actions are never individually addressable by - * `--from`. + * canonical generic `.ad` plan — the same `actions` array the replay runtime + * iterates, with one entry per executable script action. Maestro YAML uses its + * typed replay-plan and digest implementation instead of this generic action + * representation. * * Deliberately excluded: values resolved at invocation time. Variable * substitution happens in `resolveReplayAction`, after this digest is @@ -62,24 +57,11 @@ function canonicalizeAction( positionals: action.positionals ?? [], flags: action.flags ?? {}, runtime: action.runtime ?? null, - control: canonicalizeControl(action.replayControl), targetEvidence: action.targetEvidence ?? null, source: { path: sourcePath, line }, }; } -function canonicalizeControl(control: SessionAction['replayControl']): unknown { - if (!control) return null; - const nested = control.actions.map((nestedAction, index) => { - const source = control.actionSources?.[index]; - return canonicalizeAction(nestedAction, source?.line ?? null, source?.path ?? null); - }); - if (control.kind === 'retry') { - return { kind: control.kind, maxRetries: control.maxRetries, actions: nested }; - } - return { kind: control.kind, mode: control.mode, selector: control.selector, actions: nested }; -} - /** * Deterministic JSON serialization: object keys are sorted so the digest * never depends on incidental property insertion order (e.g. how an action's diff --git a/src/replay/test/trace.ts b/src/replay/test/trace.ts index 15d0c6440..995d356f5 100644 --- a/src/replay/test/trace.ts +++ b/src/replay/test/trace.ts @@ -186,10 +186,7 @@ function formatReplayStepCommand( } function formatReplayStepCommandName(command: string | undefined): string { - if (!command) return 'unknown'; - if (!command.startsWith('__maestro')) return command; - const name = command.slice('__maestro'.length); - return name.length > 0 ? name[0]!.toLowerCase() + name.slice(1) : command; + return command ?? 'unknown'; } function formatReplayStepArg(value: unknown): string { diff --git a/src/replay/vars.ts b/src/replay/vars.ts index c1d6474a6..0ced7d685 100644 --- a/src/replay/vars.ts +++ b/src/replay/vars.ts @@ -55,13 +55,6 @@ export function buildReplayVarScope(sources: ReplayVarSources): ReplayVarScope { return { values: merged, expandedBuiltinNames: new Set() }; } -export function mergeReplayVarScopeValues( - scope: ReplayVarScope, - values: Record, -): void { - Object.assign(scope.values as Record, values); -} - export function collectReplayShellEnv(processEnv: NodeJS.ProcessEnv): Record { const result: Record = {}; for (const [rawKey, value] of Object.entries(processEnv)) { @@ -159,29 +152,9 @@ export function resolveReplayAction( positionals: (action.positionals ?? []).map((token) => resolveReplayString(token, scope, loc)), flags: resolveStringProps(action.flags, scope, loc) ?? {}, runtime: resolveStringProps(action.runtime, scope, loc), - replayControl: resolveReplayControl(action.replayControl, scope, loc), }; } -function resolveReplayControl( - control: SessionAction['replayControl'] | undefined, - scope: ReplayVarScope, - loc: { file: string; line: number }, -): SessionAction['replayControl'] | undefined { - if (!control) return control; - switch (control.kind) { - case 'maestroRunFlowWhen': - return { - ...control, - selector: resolveReplayString(control.selector, scope, loc), - }; - case 'retry': - return control; - } - const _exhaustive: never = control; - return _exhaustive; -} - function resolveStringProps( obj: T | undefined, scope: ReplayVarScope, diff --git a/src/utils/__tests__/errors.test.ts b/src/utils/__tests__/errors.test.ts index ddfa49ec5..6c1499aae 100644 --- a/src/utils/__tests__/errors.test.ts +++ b/src/utils/__tests__/errors.test.ts @@ -32,13 +32,16 @@ test('normalizeError enriches generic command-failed message with stderr excerpt }); test('normalizeError appends stderr excerpt to specific command-failed messages', () => { - const err = new AppError('COMMAND_FAILED', 'uiautomator dump did not return XML', { + const err = new AppError('COMMAND_FAILED', 'Android snapshot helper did not return XML', { exitCode: 1, processExitError: true, - stderr: 'uiautomator unavailable\n', + stderr: 'instrumentation unavailable\n', }); const normalized = normalizeError(err); - assert.equal(normalized.message, 'uiautomator dump did not return XML: uiautomator unavailable'); + assert.equal( + normalized.message, + 'Android snapshot helper did not return XML: instrumentation unavailable', + ); }); test('normalizeError does not duplicate an excerpt already present in the message', () => { diff --git a/test/integration/interaction-contract/maestro-fallback.contract.test.ts b/test/integration/interaction-contract/maestro-fallback.contract.test.ts index c1919fd4d..e4823d643 100644 --- a/test/integration/interaction-contract/maestro-fallback.contract.test.ts +++ b/test/integration/interaction-contract/maestro-fallback.contract.test.ts @@ -5,9 +5,7 @@ import { AppError } from '../../../src/kernel/errors.ts'; import { assertRpcError, assertRpcOk } from '../provider-scenarios/assertions.ts'; import { scenarioName } from './coverage-manifest.ts'; import { MAESTRO_FALLBACK_COVERAGE } from './maestro-fallback.coverage.ts'; -import { RUNNER_CLOSED_DRAWER_NODES } from './fixtures.ts'; import { - runnerSnapshotEntry, runnerTapEntry, runnerTapErrorEntry, runnerTypeEntry, @@ -130,21 +128,13 @@ test('maestro-non-hittable-fallback fill resolutionDisclosure: allowed-but-not-t test(scenario('offscreen'), async () => { await withIosContractDaemon( [ - // hasTappableFrame refuses empty/out-of-app frames runner-side; the - // daemon then falls back to the runtime path, which refuses with the - // actionable offscreen shape instead of tapping blind coordinates. + // The runner refuses empty/out-of-app frames. Maestro replay preserves + // this typed result so the compat runtime can own fresh-geometry fallback. runnerTapErrorEntry(new AppError('ELEMENT_OFFSCREEN', 'Element has no tappable frame')), - runnerSnapshotEntry(RUNNER_CLOSED_DRAWER_NODES), ], async (daemon) => { const click = await daemon.callCommand('click', ['label=Explore'], MAESTRO_FLAGS); - const error = assertRpcError( - click, - 'COMMAND_FAILED', - /off-screen element and is not safe to click/, - ); - const details = error.details as Record | undefined; - assert.equal(details?.reason, 'offscreen_selector'); + assertRpcError(click, 'ELEMENT_OFFSCREEN', /no tappable frame/); }, ); }); diff --git a/test/integration/interaction-contract/maestro-fallback.coverage.ts b/test/integration/interaction-contract/maestro-fallback.coverage.ts index 50fe03c71..4774184f6 100644 --- a/test/integration/interaction-contract/maestro-fallback.coverage.ts +++ b/test/integration/interaction-contract/maestro-fallback.coverage.ts @@ -2,7 +2,7 @@ import { definePathCoverage } from './coverage-manifest.ts'; export const MAESTRO_FALLBACK_COVERAGE = definePathCoverage('maestro-non-hittable-fallback', { offscreen: - 'maestro-non-hittable-fallback offscreen: runner ELEMENT_OFFSCREEN falls back to the runtime refusal', + 'maestro-non-hittable-fallback offscreen: runner rejects an element without a tappable frame', responseConstruction: 'maestro-non-hittable-fallback responseConstruction: fallback tap response carries the canonical field set and fallback markers', }); diff --git a/test/integration/provider-scenarios/android-find.test.ts b/test/integration/provider-scenarios/android-find.test.ts index 67fb1160e..65c4a4f07 100644 --- a/test/integration/provider-scenarios/android-find.test.ts +++ b/test/integration/provider-scenarios/android-find.test.ts @@ -189,13 +189,6 @@ function androidFindAdbResult( exitCode: 0, }; } - if (args.join(' ') === 'exec-out uiautomator dump /dev/tty') { - return { - stdout: androidSettingsXml(searchText, { duplicateAppsRow: includeDuplicateAppsRow }), - stderr: '', - exitCode: 0, - }; - } if (args.join(' ').startsWith('shell am instrument ')) { return { stdout: androidSnapshotHelperOutput( diff --git a/test/integration/provider-scenarios/android-lifecycle.test.ts b/test/integration/provider-scenarios/android-lifecycle.test.ts index 4b4a95e96..769a901b1 100644 --- a/test/integration/provider-scenarios/android-lifecycle.test.ts +++ b/test/integration/provider-scenarios/android-lifecycle.test.ts @@ -1457,11 +1457,10 @@ function assertAndroidInteractionContract(world: AndroidSettingsWorld): void { assert.ok( adbCalls.some( (call) => - arrayEqual(call, ['exec-out', 'uiautomator', 'dump', '/dev/tty']) || - (call[0] === 'shell' && - call[1] === 'am' && - call[2] === 'instrument' && - call.includes('com.callstack.agentdevice.snapshothelper/.SnapshotInstrumentation')), + call[0] === 'shell' && + call[1] === 'am' && + call[2] === 'instrument' && + call.includes('com.callstack.agentdevice.snapshothelper/.SnapshotInstrumentation'), ), JSON.stringify(adbCalls), ); diff --git a/test/integration/provider-scenarios/android-test-suite.test.ts b/test/integration/provider-scenarios/android-test-suite.test.ts index f12fa9cfa..6ca764b84 100644 --- a/test/integration/provider-scenarios/android-test-suite.test.ts +++ b/test/integration/provider-scenarios/android-test-suite.test.ts @@ -69,7 +69,7 @@ test('Provider-backed integration Android replay test suite covers retries and f }); }); -test('Provider-backed integration Android Maestro replay uses fresh snapshots and authored swipe points', async () => { +test('Provider-backed integration Android Maestro reuses the exact observation snapshot and preserves authored swipe points', async () => { let snapshots = 0; await withProviderScenarioResource( async () => @@ -116,7 +116,7 @@ test('Provider-backed integration Android Maestro replay uses fresh snapshots an assert.equal(suite.failed, 0, JSON.stringify(suite)); assert.deepEqual( world.adbCalls.find((call) => call.slice(0, 3).join(' ') === 'shell input tap'), - ['shell', 'input', 'tap', '180', '330'], + ['shell', 'input', 'tap', '195', '52'], ); assert.equal(world.touchInjectionCalls.length, 1); const swipePlan = world.touchInjectionCalls[0]!; @@ -124,7 +124,8 @@ test('Provider-backed integration Android Maestro replay uses fresh snapshots an assert.equal(swipePlan.durationMs, 300); assert.deepEqual(swipePlan.pointers[0]?.samples[0]?.point, { x: 351, y: 300 }); assert.deepEqual(swipePlan.pointers[0]?.samples.at(-1)?.point, { x: 39, y: 300 }); - // Percentage resolution snapshots remain fresh; gesture planning uses the provider viewport. + // The snapshot frame used to resolve percentages is paired with the gesture plan. + assert.equal(world.gestureViewportCalls, 0); assertSnapshotCountInRange(snapshots, 3, 5); }, ); @@ -240,7 +241,7 @@ test('Provider-backed integration Android Maestro types after tapOn inputText wi ); }); -test('Provider-backed integration Android Maestro preserves pressKey Enter after native fill', async () => { +test('Provider-backed integration Android Maestro preserves authored inputText then pressKey Enter', async () => { await withProviderScenarioResource( async () => await createAndroidSettingsWorld({ nativeTextInjection: true }), async (world) => { @@ -274,8 +275,7 @@ test('Provider-backed integration Android Maestro preserves pressKey Enter after assert.equal(suite.failed, 0, JSON.stringify(suite)); assert.deepEqual(world.textInjectionCalls, [ { - action: 'fill', - target: { x: 195, y: 52 }, + action: 'type', text: 'Łódź café', delayMs: 0, }, @@ -345,7 +345,7 @@ test('Provider-backed integration Android Maestro executes runFlow conditions an world.adbCalls.filter((call) => call.slice(0, 3).join(' ') === 'shell input tap'), [['shell', 'input', 'tap', '180', '330']], ); - assertSnapshotCountInRange(snapshots, 4, 5); + assertSnapshotCountInRange(snapshots, 3, 4); }, ); }); diff --git a/test/integration/provider-scenarios/android-world.ts b/test/integration/provider-scenarios/android-world.ts index d605da3c0..b556d5cc3 100644 --- a/test/integration/provider-scenarios/android-world.ts +++ b/test/integration/provider-scenarios/android-world.ts @@ -9,6 +9,7 @@ import type { AndroidAdbProvider, } from '../../../src/platforms/android/adb-executor.ts'; import type { DeviceInventoryRequest } from '../../../src/core/dispatch-resolve.ts'; +import { ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT } from '../../../src/__tests__/test-utils/index.ts'; import { runCmd } from '../../../src/utils/exec.ts'; import { validPng } from './assertions.ts'; import { PROVIDER_SCENARIO_ANDROID } from './fixtures.ts'; @@ -78,6 +79,7 @@ export async function createAndroidSettingsWorld(options?: { packageName: 'io.example.demo_manifest', }); const adbProvider: AndroidAdbProvider = { + snapshotHelperArtifact: ANDROID_SNAPSHOT_HELPER_FIXTURE_ARTIFACT, gestureViewport: async () => { gestureViewportCalls += 1; return { x: 0, y: 0, width: 390, height: 600 }; @@ -445,13 +447,6 @@ function androidCaptureAdbResult( exitCode: 0, }; } - if (key === 'exec-out uiautomator dump /dev/tty') { - return { - stdout: snapshotXml?.() ?? androidSettingsXml(searchText), - stderr: '', - exitCode: 0, - }; - } if (key === 'exec-out screencap -p') { return { stdout: '', stderr: '', exitCode: 0, stdoutBuffer: validPng() }; } diff --git a/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts b/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts index a9eca1ce9..d752213b6 100644 --- a/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts +++ b/test/integration/provider-scenarios/interaction-direct-selector-fallback.test.ts @@ -271,3 +271,28 @@ test('Provider-backed integration maestro replay dispatch keeps runner AMBIGUOUS assertRpcError(click, 'AMBIGUOUS_MATCH', /matched multiple/); }); }); + +test('Provider-backed integration maestro replay dispatch keeps runner ELEMENT_OFFSCREEN without fallback', async () => { + const transcript = createProviderTranscript([ + { + command: 'ios.runner.tap', + deviceId: DEVICE_ID, + platform: 'apple', + request: { + command: 'tap', + selectorKey: 'label', + selectorValue: 'Continue', + allowNonHittableCoordinateFallback: true, + appBundleId: APP, + }, + error: new AppError('ELEMENT_OFFSCREEN', 'element resolved off-screen at (-161, 265)'), + }, + ]); + + await withDirectSelectorScenario(transcript, async (daemon) => { + const click = await daemon.callCommand('click', ['label="Continue"'], { + maestro: { allowNonHittableCoordinateFallback: true }, + }); + assertRpcError(click, 'ELEMENT_OFFSCREEN', /resolved off-screen/); + }); +}); diff --git a/test/integration/provider-scenarios/provider-failures.test.ts b/test/integration/provider-scenarios/provider-failures.test.ts index 27c2891d2..6c835183a 100644 --- a/test/integration/provider-scenarios/provider-failures.test.ts +++ b/test/integration/provider-scenarios/provider-failures.test.ts @@ -32,13 +32,17 @@ test('Provider-backed integration normalizes provider failures through the reque assert.equal(response.json?.error?.data?.code, 'COMMAND_FAILED'); assert.match( response.json?.error?.message ?? '', - /uiautomator dump did not return XML: uiautomator unavailable/i, + /Android snapshot helper failed.*uiautomator unavailable/i, ); assert.equal(typeof response.json?.error?.data?.diagnosticId, 'string'); assert.ok( - adbCalls.some((call) => call.join(' ') === 'exec-out uiautomator dump /dev/tty'), + adbCalls.some((call) => call.includes('--show-versioncode')), JSON.stringify(adbCalls), ); + assert.equal( + adbCalls.some((call) => call.join(' ') === 'exec-out uiautomator dump /dev/tty'), + false, + ); }, ); }); diff --git a/website/docs/docs/commands.md b/website/docs/docs/commands.md index af8dc5bcd..625b60582 100644 --- a/website/docs/docs/commands.md +++ b/website/docs/docs/commands.md @@ -283,13 +283,13 @@ agent-device get attrs @e1 ``` - iOS snapshots use XCTest on simulators and physical devices. -- Android snapshots use the bundled Android snapshot helper when the npm package includes it. The - first helper-backed snapshot verifies and installs the helper APK if it is missing or outdated; - helper failures fall back to one-shot helper capture, then stock UIAutomator, and include - `androidSnapshot.fallbackReason` in typed results. Local ADB-backed sessions keep the helper +- Android snapshots require the bundled Android snapshot helper. The first snapshot verifies and + installs the helper APK if it is missing or outdated. Local ADB-backed sessions keep the helper process warm over an `adb forward` socket and report `androidSnapshot.helperTransport` as - `persistent-session`; set `AGENT_DEVICE_ANDROID_SNAPSHOT_HELPER_SESSION=0` to disable that fast - path. Source checkouts without a bundled helper use stock UIAutomator. The helper serializes + `persistent-session`; if that transport is unavailable, capture retries through one-shot + instrumentation in the same helper. Set `AGENT_DEVICE_ANDROID_SNAPSHOT_HELPER_SESSION=0` to + disable the persistent fast path. Missing or failed helper artifacts are reported directly; a + source checkout must run `pnpm build:android` before Android verification. The helper serializes Android interactive window roots when available, so keyboard and system-overlay nodes can appear alongside the app root; `androidSnapshot.captureMode` and `androidSnapshot.windowCount` describe the capture.