Cap chat iterations at 12 with graceful fallback (closes #84)

[vahid-ahmadi]

Summary

• Lowered the per-stream tool-use cap from 60 to 12 iterations in chat_message's generate_stream loop. Twelve gives run_python enough rope to converge on real multi-step questions (capabilities check → simulation → reform → comparison → write-up) without leaving room for a stuck agent to silently exhaust the Vercel proxy timeout.
• When the cap is hit, the loop now emits a user-facing chunk event followed by a done event with stop_reason: "iteration_cap". Previously the cap branch emitted only *[Reached maximum iterations]* and a done event with no stop_reason, which often raced the Vercel proxy timeout and surfaced as "Failed to fetch".
• Added [CHAT] Session …: converged at N iterations and [CHAT] Session …: iteration cap hit at N iterations — tool_counts={…} log lines so we can track convergence rates and cap-hit frequency in metrics.

Exact fallback message

The chunk event payload (with leading newlines so it appends cleanly to any prior text):

I'm spending more iterations than expected on this without converging. Here's what I tried: ran <tool> N×, …, last attempt errored with "". Could you (a) rephrase the question, (b) enable Plan mode so I can ask clarifying questions first, or (c) try a more specific scenario?

The error clause is dropped if no tool error was captured; the tool list reads "didn't complete any tool calls" if the cap was hit before any tool ran (rare — mostly indicates a runaway text generation loop).

Continue-turn budget decision

The "continue" affordance in the frontend (ChatPage.tsx#continueMessage) re-sends the full conversation (including the partial assistant message and prior tool transcripts) to /chat/message. It therefore re-enters generate_stream with a fresh 12-iteration budget — we don't track or thread per-turn budget across requests. Rationale:

• The continue request includes the entire prior tool transcript in the conversation, so the model resumes mid-thought rather than restarting from scratch — it usually only needs a couple more iterations to wrap up.
• A "stuck" continue will hit the new 12-cap on its own and surface the same graceful fallback, so the safety property (no silent timeouts) is preserved.
• Threading per-turn budget across stateless SSE requests would require storing it in conversation state, which is out of scope for this fix.

This is documented in the comment above the MAX_ITERATIONS constant.

Plan mode

Plan mode omits tools from the Anthropic request, so it always exits the loop on iteration 1 (not tool_uses triggers the natural-completion branch). No separate plan-mode cap is needed.

Test plan

• python -c "import ast; ast.parse(open('backend/routes/chatbot.py').read())" — passes locally.
• Manually trigger a stuck loop (e.g. ask the agent to build a complex StructuralReform with deliberately ambiguous parameters) and confirm the fallback message renders and the stream closes cleanly with stop_reason: "iteration_cap" in the done event.
• Confirm a normal multi-tool question (e.g. "show the poverty impact of raising the personal allowance to £15k") still converges below the new cap.
• Confirm Plan mode still emits only clarifying questions and exits on iteration 1.
• Check Render/Vercel logs for the new converged at N / iteration cap hit at N lines.

Closes #84

Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policyengine-uk-chat	Ready	Preview, Comment	May 29, 2026 9:09am

[github-actions]

Beta preview is ready.

• Frontend: open preview
• Backend: open backend