feat: honor tool_not_found_behavior for missing custom tool calls#3707
feat: honor tool_not_found_behavior for missing custom tool calls#3707gaurav0107 wants to merge 2 commits into
Conversation
When the model emitted a custom tool call for a tool the agent does not have, process_model_response always raised ModelBehaviorError and aborted the run, ignoring RunConfig.tool_not_found_behavior. Only function tool calls honored the "return_error_to_model" opt-in. Surface a missing custom tool back to the model as a custom_tool_call_output error when return_error_to_model is set, mirroring the existing function-tool path, so the run can recover instead of crashing. The default raise_error behavior is unchanged. Related to openai#325
Add Runner-level tests that drive a missing custom tool call through return_error_to_model and assert the run recovers and the next turn input contains a custom_tool_call_output with the missing call id and the not-found message, including a tool_error_formatter variant that confirms tool_type is "custom".
|
Thanks for the contribution. The implementation demonstrates that missing custom tool calls could follow the existing function-tool recovery path, but we do not currently have evidence of a concrete user-facing need for this extension. The reports in #325 and #3459 concern missing function tools, which are already covered by #3461. This PR does not include a real custom-tool reproduction, provider scenario, or user report showing that an unavailable custom tool call occurs in practice. Given the narrow reach and the additional runtime, tracking, testing, and documentation surface, I do not think the maintenance cost is justified yet. I am going to close this PR for now. We can revisit the design if we receive a concrete custom-tool reproduction, including the provider and how the unavailable tool call was produced. |
2 participants
Summary
When the model emits a custom tool call (
custom_tool_call) for a tool theagent does not have,
process_model_responsealways raisedModelBehaviorErrorand aborted the whole run, ignoringRunConfig.tool_not_found_behavior. The"return_error_to_model"opt-in wasonly honored for function tool calls.
This change makes a missing custom tool follow the same recovery path as a
missing function tool: when
tool_not_found_behavior="return_error_to_model"is set, the unresolved custom tool call is surfaced back to the model as a
custom_tool_call_outputerror (the correct output type for a custom toolcall) and the run continues, instead of crashing. The default
"raise_error"behavior is completely unchanged.Specifically:
ToolRunCustomNotFoundand acustom_tools_not_foundfield onProcessedResponse(appended at the end; internal run-loop types).return_error_to_modelin theResponseCustomToolCallnot-foundbranch, mirroring the existing function-tool branch.
custom_tool_call_outputerror item for missing custom tools,reusing the same shape already produced by the successful and rejected
custom-tool paths.
tool_type="custom"through toToolErrorFormatterArgsso a customtool_error_formattersees the correct tool type.tool_not_found_behaviordocstring to cover custom tools.Test plan
make format,make lint,make typecheck(mypy + pyright) all pass.make testspasses (full suite).tests/test_process_model_response.py:ModelBehaviorErrorby default, andtool_not_found_behavior="return_error_to_model"it records onecustom_tools_not_foundentry and does not raise.Issue number
Related to #325 (focused slice: recover from the model calling a
non-existent custom tool, rather than a general retry mechanism).
Checks
make format,make lint,make typecheck, andmake tests