Skip to content

feat(core): continue responses after output token limits - #53876

Open
rekram1-node wants to merge 6 commits into
v2from
output-continuation
Open

rekram1-node wants to merge 6 commits into
v2from
output-continuation

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Continue responses that finish with length when no local tool results already drive another request. Preserve partial output and add this synthetic user-role instruction:

Your last response hit the output token limit (stop reason: length). Do not apologize, recap, or repeat yourself. Break the remaining work into smaller pieces.

Allow at most two consecutive automatic continuations. Read only the three most recent user/assistant messages, including the current response. If all three ended with length, preserve the response and report an output-limit error. This applies whether the responses contain text, reasoning, or tools. New user input or a different finish reason breaks the streak; intervening synthetic messages and tool results do not. No separate nudge counter or metadata is needed, and the bounded history check survives compaction and restart.

Failed tool input gets one error tool result containing recovery guidance and an input excerpt capped at 2,048 characters, with the original length and a truncation marker when needed:

  • Malformed JSON: explain that parsing failed and the call was not executed; ask for valid JSON arguments.
  • Arguments interrupted by length: explain that the call was not executed; ask for smaller tool calls with complete arguments.

Wait for the finish reason before choosing the malformed-input error wording. Completed tool calls still execute eagerly, and failed arguments remain {} rather than being repaired or executed. Local tool results drive the next request without a separate synthetic nudge, but their assistant responses still count toward the consecutive output-limit cap. Provider-executed tools do not suppress text completion.

Reuse the existing within-step request loop. Transport retries, compaction policy, and public APIs remain unchanged.

Verification

  • Focused coverage for text/reasoning continuation, durable cap and new-input reset, provider-executed tool output, and bounded malformed/truncated input excerpts.
  • bun test test/session-runner.test.ts test/session-step.test.ts test/session-runner-tool-events.test.ts: 257 passed.
  • bun test test/session-runner-message.test.ts: 29 passed.
  • bun typecheck in packages/core: passed.
  • bun run check: passed (repository lint warnings remain).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant