fix(ai-gateway): append placeholder user message for trailing assistant message on Anthropic chat_completions#2530
Conversation
…ompletions when last message is from assistant
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
| // anthropic doesn't allow '.' in tool call ids | ||
| // we can fix this later for the responses api if it's still a problem | ||
| normalizeToolCallIds(requestToMutate.body, toolCallId => toolCallId.includes('.'), undefined); | ||
| appendPlaceholderUserMessageIfLastIsAssistant(requestToMutate.body); |
There was a problem hiding this comment.
WARNING: This rewrites valid assistant-prefill requests on providers that already accept them
applyAnthropicModelSettings runs for every Anthropic model before we know which upstream provider will handle the request. We already prefer Bedrock/Anthropic specifically because Vertex is the provider that rejects assistant prefills, so unconditionally turning a trailing assistant message into a new user turn here changes the semantics of requests that would otherwise be valid. Prefix-steering cases like JSON/code prefills will now be treated as a follow-up user prompt instead of an assistant prefill.
There was a problem hiding this comment.
Intentional. The OpenAI chat_completions wire format has no native concept of an assistant prefill — the last message being assistant is just a normal conversation turn that expects a follow-up user message. This workaround is specifically for chat_completions clients that accidentally leave a trailing assistant message (which Anthropic's Messages API would reject when the request is translated). Clients that want true Anthropic assistant prefill semantics should use the native Messages API path, which this code leaves untouched.
Code Review SummaryStatus: 1 Issues Found | Recommendation: Address before merge Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
Other Observations (not in diff)No additional issues found outside the diff. Files Reviewed (2 files)
Reviewed by gpt-5.4-20260305 · 458,825 tokens |
chrarnoldus
deleted the
fix/anthropic-chat-completions-trailing-assistant-message
branch
1 participant
Summary
assistantmessage, append a minimal placeholderusermessage (Continue.) so the upstream provider accepts the request.chat_completionsonly — the native Messages API (which supports assistant prefills) and the Responses API are untouched.applyAnthropicModelSettingsnext to the existing chat_completions-only tool-call-id normalization.Verification
pnpm --filter web typecheckpnpm formatpnpm --filter web lintapps/web/src/lib/ai-gateway/providers/anthropic.test.tscovering the new function and the three request kinds; tests could not be executed locally because Docker/Postgres are unavailable in this environment, so CI is the primary verification.Visual Changes
N/A
Reviewer Notes
"Continue."— adjust if a different wording is preferred.applyAnthropicModelSettings, which is gated byisAnthropicModel(requestedModel).