feat(tracing): migrate to OpenLLMetry, drop LangSmith auto-bridge#14
Merged
dominik-bura merged 2 commits intomainfrom Apr 22, 2026
Merged
feat(tracing): migrate to OpenLLMetry, drop LangSmith auto-bridge#14dominik-bura merged 2 commits intomainfrom
dominik-bura merged 2 commits intomainfrom
Conversation
The LangSmith OTel bridge we auto-enabled for LangGraph / deepagents coverage has a critical gap: its OTel exporter never emits tool definitions (verified against langsmith 0.7.33 source and an empirical test harness). That broke the unused_tool insight in Promptic's agent evaluation — tools defined but never called were invisible. OpenLLMetry's LangchainInstrumentor (v0.60.0) now correctly covers LangGraph's create_agent and deepagents with subagents, and emits the official OpenTelemetry GenAI semantic conventions including gen_ai.tool.definitions, gen_ai.operation.name, and gen_ai.usage.*. - Remove _configure_langsmith_otel() auto-call in init(). - Remove the "LangSmith active -> skip all OpenLLMetry instrumentors" short-circuit in _auto_instrument(). OpenLLMetry instrumentors now run by default. - Bump the [langchain] extra to >=0.60.0 and pin opentelemetry-instrumentation<0.62b0 as a workaround for openllmetry#4009 (upstream PR #4025 pending). - Add direct-SDK instrumentors for AWS Bedrock, Google Vertex AI, and Mistral to _INSTRUMENTORS plus matching extras. - Add framework instrumentors for the OpenAI Agents SDK (opentelemetry-instrumentation-openai-agents-v2) and Claude Agent SDK (opentelemetry-claude-agent-sdk) plus matching extras. - _langsmith_tracing_context() and its ai_component() injection are kept: users who still prefer the LangSmith bridge can opt in by setting LANGSMITH_TRACING=true and LANGSMITH_OTEL_ENABLED=true before init(), and their Promptic tags will propagate through LangSmith run metadata. - README: update the auto-detected instrumentor list from 5 names to 10 and expand the install-extras block to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Merged
2 tasks
Contributor
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Swap the SDK's default tracing path from the LangSmith OTel bridge to OpenLLMetry, and expand the auto-instrumentor list to cover AWS Bedrock, Google Vertex AI, Mistral, the OpenAI Agents SDK, and the Claude Agent SDK.
Why
The LangSmith OTel bridge (auto-enabled since
8523ab7) was chosen because OpenLLMetry'sLangchainInstrumentorhistorically didn't capture LangGraph's internal run tree for nested graphs (deepagents subagents). But the LangSmith bridge has an unfixable gap: its OTel exporter never emits tool definitions. Verified empirically againstlangsmith==0.7.33source and a test harness — the words "function" and "bind" don't appear in the exporter code, and nothing on the langsmith-sdk issue tracker requests it.OpenLLMetry's
LangchainInstrumentorv0.60.0 now correctly covers LangGraph (create_agent) and deepagents with subagents, and emits the official OTel GenAI semantic conventions (gen_ai.tool.definitions,gen_ai.operation.name,gen_ai.usage.*). A companion PR inpromptic(#457) teaches the backend parser to read these keys.Changes
init()no longer auto-enables the LangSmith OTel bridge. The_configure_langsmith_otel()auto-call is removed;_langsmith_tracing_context()itself is kept so users who manually setLANGSMITH_OTEL_ENABLED=truestill get their Promptic tags propagated through LangSmith run metadata._auto_instrument()no longer short-circuits "skip all OpenLLMetry instrumentors when LangSmith is active."_INSTRUMENTORSexpanded from 5 → 10: +bedrock, +vertexai, +mistralai, +openai_agents, +claude_agent_sdk. Each has a matching install extra.[langchain]extra bumped to>=0.60.0and pinned againstopentelemetry-instrumentation<0.62b0as a workaround for upstream openllmetry#4009 until PR #4025 ships.[claude-agent]extra uses the community PyPI packageopentelemetry-claude-agent-sdk; the official opentelemetry-python-contrib package of the same name is not yet released.README.mdupdated to reflect the new auto-detected list and install extras.What users see
LANGSMITH_TRACING=true LANGSMITH_OTEL_ENABLED=true.Test plan
uv buildsucceeds with the new extras and pins.🤖 Generated with Claude Code