Fix scoring, CU agent error handling, and PolicyAgent prompt alignment

[abrichr]

Summary

Comprehensive reliability fixes for evaluation infrastructure, addressing false-positive scoring, agent error handling, PolicyAgent prompt alignment with training data, and Docker networking workarounds.

Scoring & Infrastructure Reliability

• Guard empty metric_results in evaluate_endpoint.py — min([]) crashed, all([]) returned True (false positive)
• Remove reasonable_completion false-positive path in mock adapter — done after 2+ actions no longer counts as success
• Add error_type field to BenchmarkResult ("infrastructure", "agent", "evaluation", or None)
• Use config.timeout for screenshot/a11y requests in live adapter (was hardcoded 30.0)
• Health check probes evaluate server (port 5050) in addition to WAA server (port 5001)

CU Agent Error Handling

• Return type="error" instead of type="done" on all failure paths (no screenshot, API failure, retry exhaustion) — runner can now distinguish infrastructure failures from genuine task completion
• Early exit on None screenshot — prevents sending "Screenshot unavailable" text to Claude and wasting retries
• Runner handles type="error" as terminal action, propagates error_type to BenchmarkResult
• Initialize action = None before runner loop to prevent NameError on 0-iteration edge case

PolicyAgent Prompt Alignment

• Use parse_qwen_action() instead of parse_action_response() — the base parser only recognizes UPPERCASE format (CLICK(0.5, 0.3)) but the trained model outputs lowercase keyword format (click(x=500, y=300)). Without this fix, every model output silently fell through to type="done".
• Rewrite _build_prompt() to match training format from convert_demos.py: <image>, Instruction: label, indented step history, <think> instruction
• Add _format_action_qwen() — formats actions in [0,1000] coordinate range matching training data
• Load model via QwenVLAdapter.from_pretrained() with proper LoRA config
• Remove dead SYSTEM_PROMPT — QwenVLAdapter.generate() drops system role messages; training and inference are now consistently aligned without it
• Fix temp file leak — track screenshot temp files, clean up in reset()

Qwen3VL Agent Fixes

• Map wait() to type="wait" instead of type="done" — prevents premature episode termination when model outputs wait()
• Fix Modal adapter path from /training/results/final to /training/adapter

Adapter Defensive Handling

• Handle type="error" in adapters — live adapter _translate_action() returns None, mock adapter treats as terminal
• Update BenchmarkAction type docstring to include "error"

Infrastructure: Docker Networking Workaround

• Re-establish socat proxy after container restart — Docker port forwarding for 5050 is broken due to QEMU's --cap-add NET_ADMIN tap networking. The socat proxy (VM:5051 -> docker exec -> container:5050) was only set up on initial creation. Now re-established in:
  • _restart_container() in run_dc_eval.py
  • ensure_waa_ready() tunnel reconnect step
  • WAA_START_SCRIPT ALREADY_RUNNING path in pool.py

Test plan

• 316 tests pass
• Mock adapter no longer gives false positives on done after 2+ actions
• CU agent returns type=error with error_type=infrastructure on failures
• PolicyAgent uses parse_qwen_action() for lowercase keyword format
• wait() action continues episode instead of terminating it
• Socat proxy re-established after container restart in all code paths