[reference] eval: blind regeneration of datastax-cassandra-4.0 (toolkit output)

[jordan-wong]

🤖 Generated with APM Instrumentation Toolkit

What this is (⚠️ reference PR — NOT intended to be merged as-is)

This is a research reference PR from the 2026-07-19 database category eval cycle. The APM Instrumentation Toolkit was asked to generate a Cassandra instrumentation from scratch, blind, against a checkout of master where dd-java-agent/instrumentation/datastax-cassandra/datastax-cassandra-4.0/ had been deleted at the blind_setup step. This PR is the toolkit's output.

Please do NOT merge. This PR creates a new module at dd-java-agent/instrumentation/cassandra/cassandra-4.0/ sitting alongside master's existing datastax-cassandra/datastax-cassandra-4.0/. Both register super(\"cassandra\") — a public-config-API name collision. See the "Known issues" section below.

The purpose is to give reviewers a concrete artifact to react to: what does the toolkit currently produce when asked to instrument Cassandra?

Summary

• New module dd-java-agent/instrumentation/cassandra/ targeting com.datastax.oss:java-driver-core:4.0.0
• 6 main source files + 1 test + build.gradle; 8 Testcontainers-backed tests, all passing under the toolkit's internal test loop.
• Chose a novel hook point: com.datastax.oss.driver.internal.core.session.DefaultSession.execute(Request, GenericType) — the single concrete dispatch method through which every CqlSession.execute* variant flows (sync execute, executeAsync). One method-advice covers all variants (see class comment on CqlSessionExecuteInstrumentation.java).
• Async covered via SpanFinishingCallback on CompletionStage.whenComplete.
• Peer/host info from ContactPointsUtil.getContactPoints(session).
• db.statement handles SimpleStatement + BoundStatement.getPreparedStatement().getQuery().

Known issues to look at first

1. Module-name collision — super(\"cassandra\") in the new CassandraClientModule collides with master's datastax-cassandra/datastax-cassandra-4.0/CassandraClientModule which also uses super(\"cassandra\"). The toolkit had the "read the existing module's super(...) and copy it verbatim" rule loaded at generation time (post-skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927) and violated it. See the cycle report's follow-up section on R-DB-1. Left as generated — see CI Triage Status below for a note on an earlier attempt to work around this via a rename, which was reverted since renaming a super(...) name is itself the kind of integration-identity decision this rule exists to prevent.

2. Reactive coverage gap — CqlSession.executeReactive returns a Publisher<ReactiveRow>. The DefaultSession.execute(Request, GenericType) hook does not intercept the reactive dispatch. No test exercises it. Real coverage gap in a real driver API.

3. Internal-class target risk — hooking DefaultSession (a driver-internal class) is more fragile than hooking the CqlSession interface. DataStax could rename it between minor versions.

What was measured (C-CASS-1 through C-CASS-4)

#	Criterion	Result
C-CASS-1	Wrapper (TracingSession) vs individual execute* overloads	NEITHER — novel single-dispatch-point pattern
C-CASS-2	sync + async + reactive coverage	PARTIAL (async yes; reactive not covered)
C-CASS-3	peer.service from contact points	PASS
C-CASS-4	db.statement on prepared statements	PASS

Reviewer verdict (toolkit's internal reviewer): approved=True verdict='approved' todos_fixed=9 todos_remaining=0 after 3 review-cycle iterations.

Research provenance

• Cycle report: docs/eval-research/cycles/2026-07-19-database-cycle-report.md (toolkit repo)
• Hypothesis outcome: docs/eval-research/hypotheses/cassandra.md
• Plan: docs/superpowers/plans/2026-06-28-database-category-eval.md
• Skill loaded at generation: origin/master @ 846103dfeb, includes PR skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927 (084b01b643 skill(apm-integrations): additional rules from recent HTTP-category PR reviews).
• Blind protocol verified: blind_setup commit deleted datastax-cassandra-4.0/ (0 files at that tree; parent had 8 subdirs). See cycle report §Blind protocol.
• Cost: $87.30 · Duration: ~3h39m · Test-fixer iterations: 5 (all 8 tests green at iter5)

Test plan (for reviewers assessing the toolkit output)

The toolkit's internal test loop passed 8 tests against a Testcontainers-backed Cassandra. Full CI has not been run yet on this branch — this PR is opened as draft to trigger CI so we can capture:

• :check :muzzle :instrumentationLatestDepTest outcome (multi-JVM matrix in CI)
• Whether the naming collision with master's datastax-cassandra is caught by any lint/registry check

The maintainer question this PR is asking is not "should we merge this?" but "is this the shape we would want the toolkit to produce for Cassandra?"

Try it out (toolkit)

TOOLKIT_BRANCH=eval/java bash <(gh api 'repos/DataDog/apm-instrumentation-toolkit/contents/bootstrap.sh?ref=eval/java' --jq '.content | @base64d')

🤖 Generated with APM Instrumentation Toolkit

CI Triage Status

Last updated: 2026-07-22

Confirmed research findings (do not fix — core instrumentation-logic gap):

• R-DB-1, module-name collision — super("cassandra") collides with master's real datastax-cassandra/datastax-cassandra-4.0/ module, also super("cassandra"). This is the point of the PR (see "Known issues" above) — left as generated, not fixed. Correction (2026-07-22): an earlier pass on this branch (commit 75b06bbe03) renamed this to super("cassandra-toolkit") as a workaround to also satisfy check-instrumentation-naming in one commit. That was a mistake — integration names are public config API, and this codebase has an explicit rule (from PR skill(apm-integrations): additional rules from recent HTTP-category PR reviews #11927, quoted in docs/eval-research/hypotheses/cassandra.md on the toolkit repo) against silently renaming them; prior research cycles (HTTP, async) handled analogous name matches by documenting the collision, not inventing a workaround name. Reverted in commit 587b271765 — super("cassandra") restored, collision is back and documented here as unfixed, consistent with how R-DB-1 should be handled.
• Datadog PR Gates / No new flaky tests — FAILED, flagging test fingerprints tied to first_flaked_branch:"eval/datastax-cassandra-4.0-db-blind-regen-20260719". Almost certainly CassandraClientTest's tests (the only new tests this branch introduces) showing intermittent behavior against a real Testcontainers-backed Cassandra instance. Classified (b) — not scaffolding, would require investigating/fixing actual test timing/isolation behavior, and the flakiness is itself informative about the toolkit's generated-test quality. Left as-is; persisted across multiple full CI re-runs (stable finding, not itself a transient artifact — see below for how this differs from test_inst_latest, which WAS transient).

Fixed (scaffolding, root cause is a documented finding):

• dd-gitlab/check-instrumentation-naming — naming linter rejected the module: "Module name 'cassandra' must end with a version (e.g., '2.0', '3.1.0') or one of: '-common', '-stubs', '-iast'". Fixed in commit 75b06bbe03: moved dd-java-agent/instrumentation/cassandra/ → cassandra/cassandra-4.0/, updated settings.gradle.kts. This part of the fix stands — pure directory/build-config move, no integration-identity implications. (The same commit also renamed super(...), which was reverted — see the R-DB-1 entry above.) Confirmed SUCCESS and stable across the post-revert re-run.
• dd-gitlab/config-inversion-linter — appeared transiently after the super("cassandra-toolkit") rename (missing registry entry for the new name), and disappeared once that rename was reverted. Confirmed SUCCESS and stable post-revert — was purely a side-effect of the reverted rename, not an independent finding.

Classified as flake / unrelated:

• main / End-to-end #10 / akka-http 10 — original failure was in tests/appsec/test_blocking_addresses.py::Test_Blocking_request_body_filenames::test_blocking, ValueError: No appsec event validate this condition. AppSec/WAF test with no code path through Cassandra instrumentation or any file touched by this PR. Confirmed transient: passed on the subsequent full CI re-run (13/13 akka-http shards SUCCESS). No fix needed.
• dd-gitlab/test_inst_latest: [21, 4/6] — Correction (2026-07-22): originally documented as a "genuine latestDepTest failure" (CassandraClientTest > peerServiceInputTagsSetWithKeyspace()/peerServiceCleanup() failing with DriverTimeoutException/InvalidKeyspaceException against the latest 4.x driver). Re-triage after CI fully settled post-revert: this shard passed cleanly on the re-run with no code changes to the module in between. Reclassifying as (c) flake — the original failure was Testcontainers timing sensitivity, not a deterministic driver-compatibility regression. (Distinct from the persistent Datadog PR Gates flaky-test finding above, which has recurred across multiple runs rather than resolving on retry.)

Downstream aggregates (no independent signal):

• Check system tests success, dd-gitlab/default-pipeline — both fail only because other checks fail at the time of evaluation; no separate action. Currently passing since all other checks except No new flaky tests are green.

Still unclassified: none.

CI status as of 2026-07-22 (fully settled, 584/585 checks, no pending/in-progress): Only 1 failure remains: Datadog PR Gates / No new flaky tests, a stable (b) finding. All other checks are green, including two that were previously red and have since resolved on retry (test_inst_latest: [21,4/6], akka-http). Per Phase 9e, this PR is now maximally settled as a reference artifact — the remaining red check IS the signal.

[datadog-prod-us1-3]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 57.27% (-0.03%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 587b271 | Docs | Datadog PR Page | Give us feedback!}

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.95 s	13.90 s	[-0.3%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent	12.94 s	13.02 s	[-1.6%; +0.4%] (no difference)
startup:petclinic:appsec:Agent	16.95 s	16.81 s	[-0.2%; +1.8%] (no difference)
startup:petclinic:iast:Agent	17.00 s	16.54 s	[-1.5%; +7.1%] (no difference)
startup:petclinic:profiling:Agent	16.90 s	16.81 s	[-0.7%; +1.8%] (no difference)
startup:petclinic:sca:Agent	16.90 s	16.85 s	[-0.6%; +1.2%] (no difference)
startup:petclinic:tracing:Agent	16.06 s	15.71 s	[-2.1%; +6.6%] (no difference)

Commit: 587b2717 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.