fix: don't attach empty heap profiles to Sentry memory alerts (APP-5199) - #14754
Draft
warp-agent-staging[bot] wants to merge 1 commit into
Draft
fix: don't attach empty heap profiles to Sentry memory alerts (APP-5199)#14754warp-agent-staging[bot] wants to merge 1 commit into
warp-agent-staging[bot] wants to merge 1 commit into
Conversation
The macOS heap profile path shells out to the bundled `pprof` helper and only checks its exit status before reading the output file. When `pprof` exits 0 without writing anything usable, the read yields zero bytes that propagate as `Ok(vec![])`, get attached to the Sentry event as `heap-profile.pb`, and are reported as a success. The resulting alert fires with an attachment that has no allocation samples, so it cannot be triaged, and nothing in the logs signals that the capture failed. Treat an empty dump as an error, surfacing pprof's own stderr so the cause is diagnosable, and skip the Sentry report entirely when no usable profile exists rather than claiming one was sent. Co-Authored-By: Oz <oz-agent@warp.dev>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sentry memory alerts from macOS have been arriving with a
heap-profile.pbattachment that has no content, which makes them impossible to triage — there are no allocation call stacks to analyze. The client has a silent-failure path that produces exactly that outcome.In
app/src/profiling.rs:dump_jemalloc_heap_profile_inner(non-Linux branch) shells out to the bundledpprofhelper (Contents/Helpers/pprof) againsthttp://127.0.0.1:9277/debug/pprof/heap. It checks onlyoutput.status.success()and thenstd::fs::reads the output path. Ifpprofexits 0 without writing anything usable — local profiling HTTP server unreachable, error body returned, helper missing or quarantined — the read yields an emptyVec<u8>that propagates asOk(vec![]).dump_jemalloc_heap_profileattaches that buffer unconditionally, captures the"Excessive memory usage detected"message, and logs"Sent heap profile to Sentry". The alert fires with a useless attachment and nothing indicates the capture failed.Linux is unaffected: it dumps in-process via
dump_jemalloc_pprof_bytes(jemalloc_pprof), with no helper binary, HTTP round-trip, or output file involved.This change:
dump_jemalloc_heap_profile_inner, includingpprof's own stderr in the error so the cause is diagnosable (and applies the same guard to the Linux in-process dump);dump_heap_profile_to_diskfrom writing a zero-byte profile file.Evidence
Sentry issue 7259255054 ("Excessive memory usage detected", 3,471 events / 2,467 users). Two consecutive events each carry exactly one
heap-profile.pb:815020b5f71a461387bf9d42a663185b→ attachment23766597477, reported size 11,590 Bc36ffdaf68ac4bc7b0526ccdb862e5bf→ attachment23766706070, reported size 8,576 BBoth hash to SHA1
da39a3ee5e6b4b0d3255bfef95601890afd80709— the SHA1 of empty content — and neither yields a usable payload. Affected build:mac_stable_release, releasev0.2026.07.29.09.05.stable_02. The spikes themselves are real (memory_breakdown: ~10.28 GB resident, ~10.04 GB footprint, ~17 MB GPU).One caveat, stated plainly: Sentry reports non-zero sizes for attachments whose content hashes as empty. Had the client uploaded zero bytes, the reported size would be 0. So a Sentry-side ingestion/storage problem may also be in play, and this client path may not be the sole mechanism behind those two specific attachments. The silent-failure path fixed here is a genuine defect either way — it can emit an empty profile and label it a success — but it should not be assumed to fully explain the observed empty attachments until the ingestion side is checked.
Linked Issue
APP-5199. Unblocks APP-4849, which tracks the repeated triage runs on Sentry 7259255054 that dead-end because the profile is unusable.
Testing
Validated in a Linux cloud environment, which constrains what could be exercised — stated explicitly rather than implied:
cargo check -p warp --features heap_usage_tracking— passes (app crate type-checked).cargo clippy -p warp --features heap_usage_tracking -- -D warnings— passes with no lints.rustfmt --checkwith the repo's config (.rustfmt.toml,imports_granularity=Module,group_imports=StdExternalCrate) — clean.Not run: the macOS branch of
dump_jemalloc_heap_profile_inneriscfg-gated out on Linux, so it was not compiled here (no macOS toolchain/SDK available). It was reviewed by inspection;outputremains in scope at the new bail and the error uses an inlined format arg. A macOS build is needed to confirm.Not added: no unit test. The failing path is a subprocess invocation of a bundled helper inside a
cfg-gated async function; testing it meaningfully would require injecting thepprofinvocation behind a seam, which is a larger refactor than this fix warrants. Happy to add it if reviewers prefer.I have manually tested my changes locally with
./script/runAgent Mode
CHANGELOG-NONE
Conversation: https://staging.warp.dev/conversation/8b0bcf02-f31f-4d8c-836f-ff051ff0e906
Run: https://oz.staging.warp.dev/runs/019fd445-8892-779e-93d2-f4f49c912df2
This PR was generated with Oz.