Skip to content

feat: Point coding agents to a setup checklist - #8269

Open
wedamija wants to merge 14 commits into
masterfrom
danf/warn-zero-traces-sample-rate
Open

wedamija wants to merge 14 commits into
masterfrom
danf/warn-zero-traces-sample-rate

Conversation

@wedamija

@wedamija wedamija commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

What

If a coding agent sets up Sentry and leaves tracing or logs off, the SDK prints a one-line note for the agent. The note says what Sentry won't be getting and points to a short setup checklist that ships inside the package.

Why

Coding agents usually set up Sentry from memory, and tend to enable errors but not tracing or logs. They won't open a URL to our docs if they see in command output, but they will read a file that's included in the SDK. When an agent reads the checklist, it is much more likely to turn tracing and logs on.

Not annoying people

We only want this to show up while an agent is actually setting up Sentry. If a project decided on errors only, it shouldn't see the note in every agent session after that. So the note only prints if:

  1. A coding agent is running the app, and it's not CI. People running the app themselves never see it.
  2. A DSN is set, and tracing or logs is off.
  3. The file that calls sentry_sdk.init() changed in the last hour.
  4. That change is yours: either uncommitted, or committed on this machine. A fresh checkout or a teammate's commit doesn't count. If there's no git, the file has to have changed in the last 10 minutes.

It prints once, when the process exits. You can turn it off with SENTRY_AGENT_HINTS=0, or with the new agent_setup_complete=True option, which the checklist asks the agent to set once it's been through it.

A lot of new setups, many of them written by AI coding agents, configure
tracing but set traces_sample_rate to 0, often through a setting or
environment variable that defaults to 0 and is never set. The SDK then
instruments every request, starts no traces and sends no performance data,
and nothing tells anyone. Most of these setups stay that way.

Warn at init when traces_sample_rate is 0 and no traces_sampler is set. It
uses warnings.warn rather than the SDK logger, because the logger only
writes when debug is on. The message says how to send traces, and that the
warning can be hidden with warnings.filterwarnings when 0 is intentional
(for example, to only continue traces sampled upstream).

There's no warning when traces_sample_rate isn't set, is None, or is above
0, or when a traces_sampler is set.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wedamija wedamija closed this Nov 14, 2026
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ 139813 passed | ⏭️ 7857 skipped | Total: 147670 | Pass Rate: 94.68% | Execution Time: 882m 33s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +509
Passed Tests 📈 +510
Failed Tests 📉 -1
Skipped Tests —

All tests are passing successfully.

✅ Patch coverage is 90.36% (75 of 83 changed executable lines covered; target 80%).
Project statement coverage is 90.21% (down 0.01 percentage points from base (e9c62e5) to head (1ae4e2c)).

Changed files with executable lines (2)
File Patch coverage Changed executable lines
sentry_sdk/_agent_hint.py 90.00% 72/80 covered; missed: 79, 87, 88, 99, 100, 110, 111, 180; partial branches: 73, 85, 179
sentry_sdk/client.py 100.00% 3/3 covered
Coverage diff
@@            Coverage Diff             @@
##        master     #8269       +/-##
==========================================
- Coverage    90.22%    90.21%    -0.01%
==========================================
  Files          203       204        +1
  Tracked lines     27333     27416       +83
  Branches     10242     10268       +26
==========================================
+ Hits         24660     24733       +73
- Misses        2673      2683       +10
- Partials      1556      1561        +5

Generated by Coverage Action

Coding agents that set up Sentry usually write the config from memory and leave tracing and logs off. This adds a one-line hint for them, and a short setup checklist that ships with the package.

When the SDK runs under a coding agent (`AI_AGENT`, `CLAUDECODE`, `CODEX_THREAD_ID`, `CODEX_SANDBOX`, `CURSOR_AGENT` or `GEMINI_CLI`, and not in `CI`), a DSN is set, and tracing or logs is off, it prints one line to stderr at exit. The line points to `sentry_sdk/SETUP_CHECKLIST.md`. It doesn't print for people who run the app themselves.

The checklist ships in the package instead of linking to the docs because agents rarely open a URL they see in command output, but they do read local files. A file in the package also always matches the installed SDK version.

The hint can be turned off with `SENTRY_AGENT_HINTS=0`, or with the new `agent_setup_complete=True` option, which the checklist asks agents to set once they've gone through it.
@wedamija wedamija changed the title [DO NOT MERGE] feat(tracing): Warn when traces_sample_rate is 0 [DO NOT MERGE] feat: Point coding agents to a setup checklist Nov 16, 2026
@wedamija wedamija reopened this Nov 16, 2026
Start the hint with what Sentry isn't getting (tracing, logs or both), so an agent that doesn't open the checklist can still tell the user what's missing.
Codex treats instructions in tool output as untrusted and won't turn on more
data collection unless the user asks. It does pass a fact on to the user, so
under Codex the hint asks it to say what's off and offer to turn it on.
The hint says what Sentry won't show for an error (the request, the logs) and that the checklist shows what to change. An agent that reads the setting names in the hint treats them as a choice it already made and doesn't open the checklist. An agent that sees the hint only in its last command passes this sentence on to the user.

The Codex line keeps its first sentence, because the rest of that line refers to the settings by name.
"Not the request or logs around them" isn't accurate. An error event has request data, and log records still arrive as breadcrumbs. Agents also read "the request" as request data and pointed at send_default_pii. The hint says what tracing and logs add: the request trace, and logs beyond breadcrumbs.
The hint printed every time an agent ran the app with tracing or logs off, so a project that chose errors only got it in every agent session. It prints when the file that calls init() changed in the last hour and git says the change is this user's: uncommitted, or committed here. A fresh checkout or a teammate's commit doesn't count. Without git, the file's modified time decides.
Without git the SDK can't tell whose change it is, so a recently copied project could get the hint for an hour. Ten minutes still covers an agent that writes the setup and then runs the app.
subprocess.run() has no capture_output or text arguments on Python 3.6, so the git call raised at exit there. The exit hook also runs inside capture_internal_exceptions, so a failure in it can't reach the app.

The hint tests run the script in-process, so coverage sees the code, and they don't require the hint to be the first thing on stderr: free-threaded Python prints a GIL warning before it.
The forms for one product off named the other products ("errors and logs", "errors and traces"). Naming the products is what makes an agent skip the checklist, so each form is the full sentence with the other product's part left out.
@wedamija wedamija changed the title [DO NOT MERGE] feat: Point coding agents to a setup checklist feat: Point coding agents to a setup checklist Nov 19, 2026
@wedamija
wedamija marked this pull request as ready for review November 19, 2026 00:38
@wedamija
wedamija requested a review from a team as a code owner November 19, 2026 00:38

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit bb4ebed. Configure here.

Comment thread sentry_sdk/_agent_hint.py Resolved
Comment thread sentry_sdk/SETUP_CHECKLIST.md Resolved

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant