Skip to content

fix(core): estimate semantic request context#35616

Open
rekram1-node wants to merge 7 commits into
v2from
media-attachment
Open

fix(core): estimate semantic request context#35616
rekram1-node wants to merge 7 commits into
v2from
media-attachment

Conversation

@rekram1-node

@rekram1-node rekram1-node commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • build one token-estimation string from system text, JSON-encoded tool definitions, and semantic message history
  • count tool names and inputs plus textual, JSON, and error tool results using provider-aligned serialization
  • exclude direct media, tool-result files, and referenced OpenAI image-generation payloads
  • avoid counting internal message structure, metadata, identifiers, and media transport bytes
  • leave the canonical provider request and overflow-recovery behavior unchanged
  • cover image exclusion and tool-input pressure with focused regressions

Testing

  • bun test test/session-compaction.test.ts from packages/core
  • bun typecheck from packages/core
  • focused OpenAI Chat and Anthropic Messages image-lowering tests from packages/llm

@rekram1-node rekram1-node changed the title fix(core): exclude media bytes from compaction estimate fix(core): ignore attachments in compaction estimates Jul 6, 2026
@rekram1-node

rekram1-node commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Final estimator shape after independent review:

  • System-part text is appended directly.
  • Tool definitions are appended once as JSON.
  • Message history is flattened to model-visible content.
  • Tool calls contribute the tool name and safely encoded input.
  • Text, JSON, and error tool results follow their provider-visible serialization semantics.
  • Direct media and file items nested in multimodal tool results contribute no base64 transport bytes.
  • Referenced OpenAI Responses image-generation results omit their opaque image payload because replay sends an item reference.
  • Message IDs, metadata, native state, canonical wrapper keys, and other internal structure are not estimated.

The actual provider request and one-shot context-overflow recovery remain unchanged. Focused regressions verify real user image lowering, hosted image-generation payload exclusion, and tool-call input pressure. The independent review found no remaining substantive issue after these corrections; model-specific media token cost remains an accepted estimation limitation.

@rekram1-node rekram1-node changed the title fix(core): ignore attachments in compaction estimates fix(core): ignore media in compaction estimates Jul 6, 2026
@rekram1-node rekram1-node changed the title fix(core): ignore media in compaction estimates fix(core): estimate semantic request context Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant