Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ versioning.

### Changed

- Go tests can explicitly skip duplicate nogo analysis with `nogo = False` while
disabled Test Optimization retains the ordinary analysis policy. An optional
adapter records original SDK/nogo settings for consumers whose rules_go already
restores them on non-Go edges; upstream consumers keep the existing path.

- Opt-in test batching stores identical common enrichment tags once in the
request metadata. Per-event process IDs and varying target metadata are
preserved, and the emitted body is validated before upload. Batching remains
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1979,6 +1979,15 @@ Target metadata records the instrumentation in
`bazel.go.sdk.name`, and `bazel.go.sdk.version`. See
[Go target metadata](docs/Configuration_Reference.md#go-target-metadata).

Go analysis follows the workspace's nogo policy by default. A workspace that
already analyzes the same sources in its ordinary build can set `nogo = False`
on its instrumented tests. Disabled sync exports retain the ordinary analysis
policy. A separate `restore_non_go_settings` opt-in supports consumers whose
rules_go already restores the original SDK/nogo configuration on non-Go edges;
it requires that capability and stays off for unpatched upstream rules_go.
See [Go analysis and non-Go configuration](docs/Configuration_Reference.md#go-analysis-and-non-go-configuration)
for the contract and compatibility limits.

If the test runtime needs runtime-visible source files for metadata such as
`test.source.end`, enable source staging explicitly:

Expand Down
32 changes: 32 additions & 0 deletions docs/Configuration_Reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,8 @@ accepts:
| `stage_sources` | `False` | Stage direct `srcs` and `embedsrcs` in runtime runfiles so the test can open its source files for metadata such as `test.source.end`. Sets `rundir` to `.` unless the caller sets `rundir` |
| `ci_visibility_enabled` | `True` | Set `DD_CIVISIBILITY_ENABLED=true` in the test environment. Set `False` only when the caller owns that variable |
| `enable_test_binary_linker_optimization` | `True` | Add `-s -w` to the hidden test's `gc_linkopts` where rules_go does not already strip (see below) |
| `nogo` | `True` | Retain the configured nogo analysis. `False` skips analysis for the enabled instrumented test and its Go dependency closure. It does not disable analysis for a plain test when the sync export is disabled |
| `restore_non_go_settings` | `False` | Record incoming SDK/nogo settings for a consumer-patched rules_go that restores them on non-Go edges. Non-configurable; requires the contract described below |
| `go_test_rule` | rules_go `go_test` | Override for tests and low-level experiments; keep repository policy in a wrapper outside the macro |
| `**kwargs` | | Forwarded to `go_test` (`srcs`, `deps`, `embed`, `data`, `env`, `tags`, ...) |

Expand Down Expand Up @@ -279,6 +281,36 @@ Their `$(location)`/`$(rlocationpath)` inputs are resolved from the raw Go test'
configured dependencies, including `data` selected by `race` or `pure`. Data
binaries retain their `x_defs` and selected Go configuration.

### Go analysis and non-Go configuration

`nogo = True` preserves the workspace's existing analysis policy, including an
opt-out already selected by the caller. If the ordinary build analyzes the same
sources, the central wrapper can pass `nogo = False` for the instrumented copy.
This skips nogo for its Go dependencies too; it is not the per-package `no-nogo`
tag. A disabled sync export still creates an ordinary test with its normal
analysis policy. This option uses rules_go's publicly visible
`//go/private:bootstrap_nogo` implementation setting, verified across the
supported 0.59.0–0.65.0 range.

`restore_non_go_settings = True` is an additional opt-in for a workspace whose
rules_go already restores SDK/nogo settings on non-Go dependency edges. It
requires both `//go/toolchain:original_sdk_name` and
`//go/private:original_bootstrap_nogo`, plus transitions that restore their
JSON-encoded caller values and clear the records on those edges. Defining the
labels alone does not implement restoration. Upstream rules_go 0.59.0–0.65.0
does not provide this contract; leave the attribute false there. An unsupported
opt-in fails analysis on the missing settings instead of silently claiming
restoration.

The adapter records the incoming values before selecting `sdk_name = "ddto"`
or skipping nogo. Existing records survive repeated transition application.
The consumer's rules_go owns restoring them for native, data or proto
dependencies; the adapter does not patch rules_go or change SDK registration.
Caller-owned Go builds reached through those restored edges retain their normal
analysis policy, including embedded libraries requested by metadata helpers.
Both modes retain the `ddtest_go` test language. All SDK version, mirror,
experiment and variant settings keep their existing configuration surface.

### Explicit single-service Go descriptor

`dd_topt_go_test` also accepts a local-static `topt_data` descriptor for a
Expand Down
1 change: 1 addition & 0 deletions modules/go/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ exports_files(
"topt_go_ddtest.bzl",
"topt_go_ddtest_extension.bzl",
"topt_go_ddtest_repository.bzl",
"topt_go_ddtest_restore.bzl",
"topt_go_extension.bzl",
"topt_go_stdlib.bzl",
"topt_go_workspace.bzl",
Expand Down
32 changes: 26 additions & 6 deletions modules/go/tests/ddtest/fixtures.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"""Fixtures and analysis tests of the ddtest instrumentation."""

load("@bazel_skylib//lib:unittest.bzl", "analysistest", "asserts", "unittest")
load("//:topt_go_ddtest.bzl", "argument_words_for_tests", "ddtest_transition_impl_for_tests")
load("//:topt_go_ddtest.bzl", "argument_words_for_tests", "ddtest_restore_transition_impl_for_tests", "ddtest_transition_impl_for_tests")

# The stub sync repository's export, as the macro tests use it.
TOPT_DATA = {
Expand Down Expand Up @@ -40,11 +40,31 @@ foreign_mini_failure_test = analysistest.make(

def _ddtest_transition_test_impl(ctx):
env = unittest.begin(ctx)
asserts.equals(
env,
{"@io_bazel_rules_go//go/toolchain:sdk_name": "ddto"},
ddtest_transition_impl_for_tests(None, None),
)
sdk = "@io_bazel_rules_go//go/toolchain:sdk_name"
skip = "@io_bazel_rules_go//go/private:bootstrap_nogo"
original_sdk = "@io_bazel_rules_go//go/toolchain:original_sdk_name"
original_skip = "@io_bazel_rules_go//go/private:original_bootstrap_nogo"

# The stdlib warmup also uses this transition, without a nogo attribute.
asserts.equals(env, {sdk: "ddto", skip: False}, ddtest_transition_impl_for_tests({skip: False}, struct()))
for nogo in [False, True]:
for incoming_skip in [False, True]:
attr = struct(nogo = nogo)
incoming = {skip: incoming_skip}
expected = {sdk: "ddto", skip: incoming_skip or not nogo}
values = ddtest_transition_impl_for_tests(incoming, attr)
asserts.equals(env, expected, values)
asserts.equals(env, values, ddtest_transition_impl_for_tests(values, attr))

for incoming_sdk in ["", "custom-sdk", "ddto"]:
for saved_sdk, saved_skip in [("", ""), (json.encode("parent-sdk"), "false")]:
incoming = {sdk: incoming_sdk, skip: incoming_skip, original_sdk: saved_sdk, original_skip: saved_skip}
restored = ddtest_restore_transition_impl_for_tests(incoming, attr)
expected = dict(values)
expected[original_sdk] = saved_sdk or (json.encode(incoming_sdk) if incoming_sdk != "ddto" else "")
expected[original_skip] = saved_skip or (json.encode(incoming_skip) if values[skip] != incoming_skip else "")
asserts.equals(env, expected, restored)
asserts.equals(env, restored, ddtest_restore_transition_impl_for_tests(restored, attr))
return unittest.end(env)

ddtest_transition_test = unittest.make(_ddtest_transition_test_impl)
Expand Down
106 changes: 73 additions & 33 deletions modules/go/topt_go_ddtest.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ load(":topt_go_ddtest_repository.bzl", "DDTEST_RELEASE", "MINI_REPO")

_BAZEL_TARGET_METADATA_OUTPUT = "bazel_target_metadata.json"
_SDK_NAME_SETTING = "@io_bazel_rules_go//go/toolchain:sdk_name"
_SKIP_NOGO_SETTING = "@io_bazel_rules_go//go/private:bootstrap_nogo"
_ORIGINAL_SDK_NAME_SETTING = "@io_bazel_rules_go//go/toolchain:original_sdk_name"
_ORIGINAL_SKIP_NOGO_SETTING = "@io_bazel_rules_go//go/private:original_bootstrap_nogo"
_MINI_MODULE = "github.com/DataDog/ddtest/ddto"
_ARG_COUNT = "_DD_TOPT_WRAPPER_ARG_COUNT"
_ARG_PREFIX = "_DD_TOPT_WRAPPER_ARG_"
Expand All @@ -30,15 +33,46 @@ ToptGoTestArgsInfo = provider(
# Mini's testopt library, which the raw test depends on.
MINI_TESTOPT = Label("@%s//testopt" % MINI_REPO)

def _ddtest_transition_impl(_settings, _attr):
return {_SDK_NAME_SETTING: DDTEST_RELEASE.sdk_name}
def _ddtest_transition_impl(settings, attr):
# This publicly visible implementation setting exists throughout the tested
# rules_go range. Do not override a caller that already disabled analysis.
values = {
_SDK_NAME_SETTING: DDTEST_RELEASE.sdk_name,
_SKIP_NOGO_SETTING: settings[_SKIP_NOGO_SETTING] or not getattr(attr, "nogo", True),
}
return values

def _record_original_settings(settings, values):
for key, original_key in [
(_SDK_NAME_SETTING, _ORIGINAL_SDK_NAME_SETTING),
(_SKIP_NOGO_SETTING, _ORIGINAL_SKIP_NOGO_SETTING),
]:
original = settings[original_key]
if values[key] != settings[key] and not original:
original = json.encode(settings[key])
values[original_key] = original

def _ddtest_restore_transition_impl(settings, attr):
values = _ddtest_transition_impl(settings, attr)
_record_original_settings(settings, values)
return values

ddtest_transition_impl_for_tests = _ddtest_transition_impl
ddtest_restore_transition_impl_for_tests = _ddtest_restore_transition_impl

ddtest_transition = transition(
implementation = _ddtest_transition_impl,
inputs = [],
outputs = [_SDK_NAME_SETTING],
inputs = [_SKIP_NOGO_SETTING],
outputs = [_SDK_NAME_SETTING, _SKIP_NOGO_SETTING],
)

# Referencing these labels is safe until the restored transition is applied.
# Consumers must explicitly select it and supply rules_go's restoration support.
_restored_settings = [_SDK_NAME_SETTING, _SKIP_NOGO_SETTING, _ORIGINAL_SDK_NAME_SETTING, _ORIGINAL_SKIP_NOGO_SETTING]
_ddtest_restore_transition = transition(
implementation = _ddtest_restore_transition_impl,
inputs = _restored_settings,
outputs = _restored_settings,
)

# --- A single Mini --------------------------------------------------------
Expand Down Expand Up @@ -362,32 +396,38 @@ def _ddtest_go_test_impl(ctx):
providers.append(testing.ExecutionInfo(requirements = execution_requirements))
return providers

ddtest_go_test = rule(
implementation = _ddtest_go_test_impl,
attrs = {
"_windows_launcher": attr.label(
default = "//tools/testwrapper",
executable = True,
cfg = "target",
),
"actual": attr.label(
mandatory = True,
executable = True,
cfg = ddtest_transition,
aspects = [mini_check_aspect, _test_args_aspect],
doc = "The hidden raw go_test, built with the instrumented SDK.",
),
"metadata": attr.label(
mandatory = True,
allow_single_file = True,
cfg = ddtest_transition,
doc = "Bazel-owned target metadata copied next to emitted test payloads.",
),
"_allowlist_function_transition": attr.label(
default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),
"_windows_constraint": attr.label(default = "@platforms//os:windows"),
},
test = True,
doc = "The public test target of a Go test instrumented by ddtest.",
)
def ddtest_go_test_rule(restore_non_go_settings = False):
"""Construct the wrapper rule; its exporting module keeps the ddtest_go_test name."""
configuration = _ddtest_restore_transition if restore_non_go_settings else ddtest_transition
return rule(
implementation = _ddtest_go_test_impl,
attrs = {
"_windows_launcher": attr.label(
default = "//tools/testwrapper",
executable = True,
cfg = "target",
),
"actual": attr.label(
mandatory = True,
executable = True,
cfg = configuration,
aspects = [mini_check_aspect, _test_args_aspect],
doc = "The hidden raw go_test, built with the instrumented SDK.",
),
"metadata": attr.label(
mandatory = True,
allow_single_file = True,
cfg = configuration,
doc = "Bazel-owned target metadata copied next to emitted test payloads.",
),
"_allowlist_function_transition": attr.label(
default = "@bazel_tools//tools/allowlists/function_transition_allowlist",
),
"_windows_constraint": attr.label(default = "@platforms//os:windows"),
"nogo": attr.bool(default = True, doc = "Whether to retain the configured nogo analysis for the instrumented build."),
},
test = True,
doc = "The public test target of a Go test instrumented by ddtest.",
)

ddtest_go_test = ddtest_go_test_rule()
12 changes: 12 additions & 0 deletions modules/go/topt_go_ddtest_restore.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Unless explicitly stated otherwise all files in this repository are licensed under
# the Apache 2.0 License.
#
# This product includes software developed at Datadog
# (https://www.datadoghq.com/) Copyright 2025-Present Datadog, Inc.

"""Explicit adapter for rules_go with original SDK/nogo restoration settings."""

load(":topt_go_ddtest.bzl", "ddtest_go_test_rule")

# Export the same name so Bazel's test language remains ddtest_go in both modes.
ddtest_go_test = ddtest_go_test_rule(restore_non_go_settings = True)
16 changes: 15 additions & 1 deletion modules/go/topt_go_test.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ load(
)
load("@io_bazel_rules_go//go:def.bzl", "go_test")
load("//:topt_go_ddtest.bzl", "MINI_TESTOPT", "ddtest_go_test", "topt_go_ddtest_registration")
load("//:topt_go_ddtest_restore.bzl", ddtest_go_restore_test = "ddtest_go_test")
load("//:topt_go_infer.bzl", "topt_go_bazel_metadata", "topt_go_payloads_selector")

_service_mapping_entries = service_mapping_entries
Expand Down Expand Up @@ -316,6 +317,8 @@ def dd_topt_go_test(
# behavior. Production go_binary targets are unaffected because this
# macro only forwards flags to its hidden raw go_test target.
enable_test_binary_linker_optimization = True,
nogo = True,
restore_non_go_settings = False,
**kwargs):
"""Define a Go test with Datadog Test Optimization support.

Expand Down Expand Up @@ -355,6 +358,13 @@ def dd_topt_go_test(
in Bazel modes where rules_go is not already stripping and the user did
not explicitly request debug/no-strip behavior. Set false to restore
caller-provided linker flags exactly if this optimization causes issues.
nogo: Keep the configured nogo analysis for the instrumented build by
default. Set false only when another build analyzes the same sources.
Disabled Test Optimization exports retain the ordinary test's policy.
restore_non_go_settings: Non-configurable boolean. Set true only with a
consumer-patched rules_go that restores sdk_name and bootstrap_nogo
from original_sdk_name and original_bootstrap_nogo on non-Go edges.
Upstream rules_go 0.59.0-0.65.0 does not implement that contract.
**kwargs: Forwarded to underlying go_test (e.g., srcs, deps, data, tags, ...).
The raw test also gets Mini's `testopt` library and the registration
file of its package.
Expand All @@ -367,6 +377,8 @@ def dd_topt_go_test(
if topt_data == None or not _is_dict(topt_data):
fail_with_prefix("dd_topt_go_test", "topt_data is required and must be the dict from @<repo>//:export.bzl (single-service) or the aggregator mapping")
_reject_removed_orchestrion_attrs(kwargs)
if type(restore_non_go_settings) != "bool":
fail_with_prefix("dd_topt_go_test", "restore_non_go_settings must be a non-configurable boolean")
if go_test_rule == None:
fail_with_prefix("dd_topt_go_test", "go_test_rule override cannot be None")

Expand Down Expand Up @@ -664,9 +676,11 @@ def dd_topt_go_test(
**kwargs
)

ddtest_go_test(
wrapper_rule = ddtest_go_restore_test if restore_non_go_settings else ddtest_go_test
wrapper_rule(
name = name,
actual = ":" + raw_name,
metadata = ":" + metadata_name,
nogo = nogo,
**wrapper_kwargs
)
8 changes: 8 additions & 0 deletions tools/agent-skills/go-test-optimization-onboarding/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,14 @@ Go wrapper, treat it as a consumer-specific integration:
- Preserve repository-local wrapper policy such as tags, scheduling, Docker
defaults, platform constraints, and flaky-test behavior in the local helper
layer.
- Keep nogo enabled by default. Set `nogo = False` only for an enabled
instrumented copy whose sources the ordinary build also analyzes. Verify a
deliberate analyzer violation still fails the ordinary build. Disabled sync
exports must retain that analysis even when the wrapper passes this option.
- Leave `restore_non_go_settings = False` with upstream rules_go. Enable it only
after verifying the consumer's original SDK/nogo setting and restoration
transition contract in the configuration reference. Adding settings alone
cannot restore dependency configurations; never patch rules_go implicitly.
- Preserve the caller's `DD_SERVICE`, including an unset value, inherited
environment and `select()`. Do not inject the reporting service into the test.
The uploader validates it against the target sidecar and sync context; legacy
Expand Down
Loading
Loading