Skip to content

Trigger the speedlab memory gate from branch pipelines - #963

Draft
rkennke wants to merge 3 commits into
mainfrom
rkennke/memory-gate-trigger
Draft

rkennke wants to merge 3 commits into
mainfrom
rkennke/memory-gate-trigger

Conversation

@rkennke

@rkennke rkennke commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?:

It adds a manual memory-gate job to the GitLab branch pipelines. The job triggers speedlab's memory gate (DataDog/speedlab#105, merged) to compare this branch's java-profiler build with main's on spring-petclinic's memory. Speedlab posts a warn-only verdict on the PR (DataDog/speedlab#109).

Piece Change
deploy.sh After verifying the snapshot it published, writes MEMORY_GATE_CANDIDATE=snapshot:<that build> (write_memory_gate_candidate in includes.sh). The write cannot fail the job.
deploy-artifact Exposes that file as a dotenv report.
.gitlab/memory-gate/.gitlab-ci.yml One manual trigger job:
- SPEEDLAB_TRIGGER=java-profiler-memory-gate
- PROFILER_CANDIDATE = the recorded build
- PROFILER_BASELINE=branch:main
- UPSTREAM_* for the comment
It sets inherit: variables: false, so no global variable leaks into speedlab. It reuses .benchmarks-skip-rules, plus main and tags.
TestingGuide.md A tier row and a short section.

Motivation:

This makes the low-noise memory measurement from the variance investigation available on PRs, so a profiler change's memory overhead is visible before merge.

Additional Notes:

  • Draft until DataDog/speedlab#109 merges: the trigger targets speedlab's rkennke/memory-gate-trigger branch. It switches to main once Clean up the sanitizer runs #109 merges.
  • Pinning limit: the candidate is the build deploy-artifact verified. Two pushes to one branch that publish at nearly the same time can make it record the other push's build; this is documented.
  • Without an open PR, deploy-artifact publishes nothing (prepare.sh's CANCELLED). Rules cannot see that dotenv variable, so the job is still offered, but it has no candidate and speedlab rejects the run.
  • Expected warning: where no snapshot is published (tags, no PR), deploy-artifact's upload logs a "no matching files" warning for memory-gate.env.
  • Review: the agentic review (/code-review, three rounds) led to the current design.
    • It replaced a separate resolve job, which re-read the snapshot metadata. That approach had a branch-name regex bug and could pin a stale build or turn PR pipelines red.
    • It added the non-fatal write and the shared skip rules.
    • Not changed: copyright headers (.gitlab/** is exempt per AGENTS.md), and an integration test of the deploy.sh call, which cannot run outside CI.

How to test the change?:

  • Unit tests: bash .gitlab/scripts/tests/includes_test.sh covers the helper: it writes the build's selector, and rejects an empty build without writing.
  • End to end, in progress on this PR: run a branch pipeline, play memory-gate, then check that speedlab runs only the gate jobs and posts the verdict here.

For Datadog employees:

  • If this PR touches code that signs or publishes builds or packages, or handles
    credentials of any kind, I've requested a security review (run the dd:platform-security-review
    skill, or file a request via the PSEC review form).
    bewaire also runs automatically on every PR.
  • This PR doesn't touch any of that.
  • JIRA: PROF-16185

🤖 Generated with Claude Code

rkennke and others added 3 commits November 9, 2026 19:21
A manual memory-gate job triggers speedlab's memory gate with this
branch's published snapshot against main's. memory-gate:resolve pins the
candidate to the timestamped build deploy-artifact published, so a later
push cannot change what is measured, and passes it on as a dotenv
variable. The trigger job inherits no global variables, so only its own
reach speedlab. Speedlab posts the warn-only verdict on the PR. Neither job
runs on main, tags or release branches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
deploy-artifact already finds and verifies the snapshot build it publishes,
so deploy.sh now records it as MEMORY_GATE_CANDIDATE in a dotenv report
(write_memory_gate_candidate in includes.sh), and the memory-gate trigger
job reads it from there. This replaces memory-gate:resolve, which re-read
the snapshot metadata in every PR pipeline: it could pin another push's
build from stale metadata, rejected branch names with regex characters,
and could turn the pipeline red although the gate is manual. The trigger
job now also skips merge-queue branches, scheduled and merge-request
pipelines and jdk-integration triggers, like the benchmarks jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…k skip rules

The publish has already succeeded when deploy.sh records the candidate, so a
failed write now only warns instead of failing deploy-artifact. The trigger
job reuses .benchmarks-skip-rules (including its CANCELLED and merge-queue
rules) and checks main by CI_COMMIT_REF_NAME, which is set for triggered
pipelines too. The docs state that two near-simultaneous pushes to one
branch can record the other push's build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

Your PR has warnings. Please review the issues below.

🚦 12 Pipeline jobs failed

DDCI - DataDog/java-profiler | renaissance-benchmarks: [fj-kmeans, 25] — 🔄 Retry may pass, looks flaky

View more details · View in DDCI

DDCI - DataDog/java-profiler | renaissance-benchmarks: [future-genetic, 25] — 🔄 Retry may pass, looks flaky

View more details · View in DDCI

DDCI - DataDog/java-profiler | renaissance-benchmarks: [naive-bayes, 25] — 🔄 Retry may pass, looks flaky

View more details · View in DDCI

View all 12 failed jobs.

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7383d3e | Docs | View more details | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Scan-Build Report

User:runner@runnervmwuk1x
Working Directory:/home/runner/work/java-profiler/java-profiler/ddprof-lib/src/test/make
Command Line:make -j4 all
Clang Version:Ubuntu clang version 21.1.8 (6ubuntu1)
Date:Mon Nov 9 20:38:54 2026

Bug Summary

Bug TypeQuantityDisplay?
All Bugs2
Logic error
Dereference of null pointer1
Unused code
Dead initialization1

Filter Results by File

Reports

Bug Group Bug Type ▾ File Function/Method Line Path Length
Unused codeDead initializationreferenceChains.cppapplyOomRamp4971
Logic errorDereference of null pointerperfEvents_linux.cppunregisterThread72026

@dd-octo-sts

dd-octo-sts Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

CI Test Results

Run: #42694637627 | Commit: 187e805 | Duration: 21m 21s (longest job)

✅ All 32 test jobs passed

Status Overview

JDK glibc-aarch64/debug glibc-amd64/debug musl-aarch64/debug musl-amd64/debug
8 - ✅ - -
8-ibm - ✅ - -
8-j9 ✅ ✅ - -
8-librca - - ✅ ✅
8-orcl - ✅ - -
11 - ✅ - -
11-j9 ✅ ✅ - -
11-librca - - ✅ ✅
17 ✅ ✅ - -
17-graal ✅ ✅ - -
17-j9 ✅ ✅ - -
17-librca - - ✅ ✅
21 ✅ ✅ - -
21-graal ✅ ✅ - -
21-librca - - ✅ ✅
25 ✅ ✅ - -
25-graal ✅ ✅ - -
25-librca - - ✅ ✅

Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled

Summary: Total: 32 | Passed: 32 | Failed: 0


Updated: 2026-11-09 21:02:19 UTC

@dd-octo-sts

dd-octo-sts Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

✅ All 40 integration tests passed

📊 Dashboard · 👷 Pipeline · 📦 7383d3e4

@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Speedlab memory gate

No significant change (pipeline): spring-petclinic's anonymous memory changed by +0.3 MiB (95% CI [-1.6, +2.1], 6 pairs).

java-profiler build
baseline 1.53.0-20261109.105029-52
candidate 1.53.0-rkennke_memory-gate-trigger-20261109.211620-1

Warn-only. A regression is a 95% interval above zero with a mean of at least 3 MiB. Changes in retained free memory show only in part; see speedlab's docs/memory-gate.md.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant