[aw-failures] [aw] Failure Investigator Report — 2026-08-07 (6h)

[github-actions]

Fix the Daily BYOK Ollama Test warm-up loop first — it has failed every scheduled run for 8 straight days, and nobody is tracking it.

15 failed runs analyzed from this window (2026-08-06 19:42–2026-08-07 01:42 UTC). 5 distinct failure clusters + 8 isolated single-run failures across unrelated workflows. 1 issue reopened (wrongly closed, recurring again within hours), 1 new P0 sub-issue filed, 1 tracked cluster reconfirmed and left untouched, 1 P1 gap folded into this report (create_issue budget: 2/run).

Cluster summary

Sev	Cluster	Runs	Status
P0	Daily BYOK Ollama Test — qwen2.5:0.5b never completes warm-up, 100% failure rate (10/10 runs, 2026-07-27 → 2026-08-07)	§31134143397 (+9 prior)	Untracked — sub-issue filed below
P0	Copilot "No model available" — PR Code Quality Reviewer general-purpose sub-agent exhausts all 4 launch attempts	§31134098453, §31131741357, §31131713448	Reopened #49022 — closed not_planned 2026-08-06 09:31 UTC, refired 3× within 14 hours
P1	Execute GitHub Copilot CLI hits the hard-coded 15-minute step timeout mid a legitimate, long-running security-fix task	§31132973257, §31132652013	Untracked, no separate issue this cycle (budget) — see below
—	Daily Reliability Review — opencode postinstall script never ran	§31135630380	Tracked by #50735, left untouched
P2	8 isolated single-run failures, 8 unrelated workflows, no shared signature	see evidence	No action — monitor only

Evidence

Daily BYOK Ollama Test — 100% failure rate (P0, new sub-issue)

• Start Ollama service and Pull small model (397 MB) both succeed in §31134143397.
• Warm up model then loops curl -sf .../api/generate 10× at 3s intervals — every single attempt fails, not just the first few — before the step hard-fails with Model 'qwen2.5:0.5b' failed to warm up after 10 attempts.
• -sf on curl suppresses the actual HTTP status/body, so the real cause (crash, OOM, request-shape mismatch against the pinned Ollama v0.31.1) is currently invisible in the logs.
• Confirmed via agenticworkflows logs --workflow_name "Daily BYOK Ollama Test": 10/10 of the last 10 scheduled runs (back to 2026-07-27) failed identically — this is not a one-off, it is a fully broken daily job.
• Full root-cause detail and remediation steps are in the new sub-issue.

Copilot "No model available" — reopening #49022 (P0)

agent-stdio.log for §31134098453:

[copilot-sdk-driver] [sdk-driver] error: Execution failed: Error: No model available. Check policy enablement under GitHub Settings > Copilot
[copilot-harness] attempt 4 failed: exitCode=1 failureClass=partial_execution ... retriesRemaining=0
[copilot-harness] all 3 retries exhausted — giving up (exitCode=1)

Same signature confirmed independently in §31131741357 and §31131713448 (8 matching log lines = 4 attempts × 2 emit points each).

This is the exact signature #49022 was filed for, closed not_planned on 2026-08-06 09:31 UTC, and already reopened once before (2026-08-06 07:41 UTC pass) after recurring within hours of an earlier close. It has now recurred a 4th consecutive 6h window. Reopened again this pass with the evidence above — recommend not closing without a code change landing.

Copilot CLI 15-minute step timeout during a legitimate long-running task (P1, untracked, no separate issue this cycle)

agent-stdio.log for §31132973257 shows the agent actively iterating — one bash tool call every 10-20s, methodically reproducing a real sanitizer bypass (protocol-relative-URL userinfo bug) with a dozen node -e regex experiments — right up until:

##[error]The action 'Execute GitHub Copilot CLI' has timed out after 15 minutes.

§31132652013 fails identically. Neither run shows a stuck/hung process — both were doing real, useful work when the clock ran out. This is a task-shape problem, not a bug in the agent: deep security-fix reviews on this workflow can legitimately exceed 15 minutes.

Not filed as a separate sub-issue this cycle (create_issue budget: 1 parent + 1 P0 sub-issue). Recommend either raising PR Code Quality Reviewer's step timeout for its Copilot CLI execution, or splitting long investigative tasks into checkpointed sub-agent calls so partial progress survives a timeout.

Daily Reliability Review — opencode postinstall (tracked, no action)

§31135630380 fails identically to the cluster #50735 already tracks (opencode-ai's postinstall script was not run). No new evidence beyond what #50735 already documents — left untouched.

8 isolated single-run failures — P2, monitor only

Each of the following failed exactly once this window, in 8 different workflows with no shared signature — insufficient signal to justify individual investigation against this cycle's budget. Re-check next window if any recur:

• Design Decision Gate 🏗️ — §31129777534
• Daily Safe Output Tool Optimizer — §31129069192
• Daily Ambient Context Optimizer — §31128139375
• Issue Monster — §31128129446
• Daily Graft Intelligence — §31127878842
• Smoke Copilot Sub Agents — §31127680307
• PR Sous Chef — §31127163345
• Code Scanning Fixer — §31127136183

Existing issue correlation

Reopened #49022 — see above. Don't touch the other 6 curated open issues — none show fresh evidence of a fix or staleness this window: #49583 (dispatch_workflow), #49446 (unguarded arithmetic), #50735 (opencode migration — reconfirmed recurring, left open), #50734 (prior parent report, 2026-08-06 — still has open sub-issue work, left open), #50687 (Ambient Context report), #48838 (No-Op Runs), #50819 (Evals Feature report). None closed.

Fix roadmap

P0 — do this first:

1. Fix Warm up model in daily-byok-ollama-test.md — 100% failure rate, 10/10 runs over 8 days. See sub-issue below.
2. Ship the [aw-failures] Fleet-wide: Copilot CLI subagent model allocation fails with 'No model available' (hit PR Code Quality Reviewer, L [Content truncated due to length] #49022 hardening — treat Copilot sub-agent model-allocation failure as non-fatal/best-effort instead of 4× full retries on an identical, non-recoverable-within-job error. 4th consecutive 6h window with a recurrence.

P1 — next:
3. Raise PR Code Quality Reviewer's Copilot CLI step timeout (currently 15 min) or checkpoint long investigative sub-tasks so legitimate deep-dive fixes don't get killed mid-work.

P2 — monitor only:

• The 8 isolated single-run failures above — no action unless any recur next window.

Sub-issues created

• Fix the Warm up model retry loop in daily-byok-ollama-test.md (P0)

References:

• §31134143397 — Daily BYOK Ollama Test
• §31134098453 — PR Code Quality Reviewer
• §31132973257 — PR Code Quality Reviewer

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 214.9 AIC · ⊞ 5.5K · ◷

• expires on Aug 13, 2026, 5:54 PM UTC-08:00

[github-actions]

🎉 Automatically closed by Sub-Issue Closer

All sub-issues have been completed. This parent issue is now closed automatically.

Sub-issues status: 1/1 closed (100%)

Generated by 🔧 Sub-Issue Closer · auto · 34.4 AIC · ⌖ 2.65 AIC · ⊞ 7.3K · ◷