Upgrade Shadow plugin to 9.4.2#11730
Conversation
* Adapt custom ShadowJar wiring to the lazy API. Configuration lists now use providers around `configurations.named(...)`, and Shadow resolves `Configuration` values directly. GradleUp/shadow#1044 GradleUp/shadow#1045 * Keep filtering explicit. Shadow 9 changed `DependencyFilter` overloads, so shared filters run in Shadow `dependencies` blocks. Empty module/version fields no longer act as broad wildcards, so group-wide excludes use full group/module/version regex notation. That keeps dependencies meant to stay external out of shaded jars; otherwise downstream jars can get partly relocated API copies, such as SLF4J, and lose their real binding. https://gradleup.com/shadow/configuration/dependencies/#using-regex-patterns-to-filter-dependencies GradleUp/shadow#1328 * Keep `mergeServiceFiles` users on `DuplicatesStrategy.INCLUDE` before transformers run, then drop unrelated duplicate resources with file matching. Shadow 9 honors duplicate strategies during transforms. GradleUp/shadow#1233 GradleUp/shadow#1617 * Disable automatic Multi-Release manifest inheritance for the agent jar to keep the previous manifest shape. GradleUp/shadow#1239 GradleUp/shadow#1675 * Note the relocation output change. Shadow 9.2+ rewrites more descriptor-shaped string constants, which explains the observed `LoggerRuntime` LDC descriptor change. GradleUp/shadow#1714 GradleUp/shadow#1731 * Apply the same API and resource handling changes to smoke-test modules that run in the main Gradle build.
There was a problem hiding this comment.
LGTM from CIVis side! AFAIK we don't use JaCoCo's LoggerRuntime at all, we rely on jacoco for parsing/analyzing coverage reports, and our smoke tests already cover the expected behavior for the instrumentation. So agreed it shouldn't be a regression
| filesMatching([ | ||
| // agent-installer also publishes this Java 11 source-set output at runtime. | ||
| 'datadog/trace/agent/tooling/bytebuddy/DDJava9ClassFileTransformer*.class', | ||
| // agent-installer also publishes this Java 25 source-set output at runtime. | ||
| 'datadog/trace/agent/tooling/servicediscovery/MemFDUnixWriterFFM*.class', | ||
| // aggregate instrumentation includes some classes directly and via jars. | ||
| 'datadog/trace/instrumentation/**/*.class', | ||
| // same aggregate duplication, for Java 11 exception profiling advice. | ||
| 'datadog/exceptions/instrumentation/**/*.class', | ||
| ]) { | ||
| duplicatesStrategy = DuplicatesStrategy.EXCLUDE | ||
| } |
There was a problem hiding this comment.
note: I asked codex to give me the list of the duplicate classes
Shadow Instrumentation Duplicate Classes
Generated from a temporary :dd-java-agent:instrumentation:shadowJar run with duplicate handling forced to INCLUDE.
The verbose origin check showed no meaningful difference between duplicate origins: class-output/runtime-artifact pairs have identical origin hashes, and the servlet5 runtime-artifact/runtime-file-dependency pairs are also identical. The list is therefore grouped by owning module.
- Duplicate class paths: 140
- Origin hash mismatches: 0
:dd-java-agent:agent-installer
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 3
-
datadog/trace/agent/tooling/bytebuddy/DDJava9ClassFileTransformer.class -
datadog/trace/agent/tooling/servicediscovery/MemFDUnixWriterFFM$Lazy.class -
datadog/trace/agent/tooling/servicediscovery/MemFDUnixWriterFFM.class
:dd-java-agent:instrumentation:datadog:profiling:exception-profiling
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 2
-
datadog/exceptions/instrumentation/ExclusionAdvice.class -
datadog/exceptions/instrumentation/ThrowableInstanceAdvice.class
:dd-java-agent:instrumentation:graal:graal-native-image-20.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 1
-
datadog/trace/instrumentation/graal/nativeimage/DeleteFieldAdvice.class
:dd-java-agent:instrumentation:java:java-lang:java-lang-22.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 3
-
datadog/trace/instrumentation/java/lang/jdk22/CaptureLibraryNameAdvice.class -
datadog/trace/instrumentation/java/lang/jdk22/CaptureSymbolAddressAdvice.class -
datadog/trace/instrumentation/java/lang/jdk22/DownCallWrapAdvice.class
:dd-java-agent:instrumentation:java:java-net:java-net-11.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 9
-
datadog/trace/instrumentation/httpclient/BodyHandlerWrapper$BodySubscriberWrapper.class -
datadog/trace/instrumentation/httpclient/BodyHandlerWrapper.class -
datadog/trace/instrumentation/httpclient/CompletableFutureWrapper.class -
datadog/trace/instrumentation/httpclient/HeadersAdvice.class -
datadog/trace/instrumentation/httpclient/HttpHeadersInjectAdapter.class -
datadog/trace/instrumentation/httpclient/JavaNetClientDecorator.class -
datadog/trace/instrumentation/httpclient/ResponseConsumer.class -
datadog/trace/instrumentation/httpclient/SendAdvice.class -
datadog/trace/instrumentation/httpclient/SendAsyncAdvice.class
:dd-java-agent:instrumentation:java:java-nio-1.8
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 3
-
datadog/trace/instrumentation/directbytebuffer/AllocateDirectAdvice.class -
datadog/trace/instrumentation/directbytebuffer/MemoryMappingAdvice.class -
datadog/trace/instrumentation/directbytebuffer/NewDirectByteBufferAdvice.class
:dd-java-agent:instrumentation:jetty:jetty-client:jetty-client-10.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 4
-
datadog/trace/instrumentation/jetty_client10/JettyClientDecorator.class -
datadog/trace/instrumentation/jetty_client10/SendAdvice.class -
datadog/trace/instrumentation/jetty_client10/SendContextPropagationAdvice.class -
datadog/trace/instrumentation/jetty_client10/SpanFinishingCompleteListener.class
:dd-java-agent:instrumentation:jetty:jetty-client:jetty-client-12.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 8
-
datadog/trace/instrumentation/jetty_client12/CallbackWrapper.class -
datadog/trace/instrumentation/jetty_client12/HeadersInjectAdapter.class -
datadog/trace/instrumentation/jetty_client12/JettyClientDecorator.class -
datadog/trace/instrumentation/jetty_client12/RequestCreateAdvice.class -
datadog/trace/instrumentation/jetty_client12/SendAdvice.class -
datadog/trace/instrumentation/jetty_client12/SendContextPropagationAdvice.class -
datadog/trace/instrumentation/jetty_client12/SpanFinishingCompleteListener.class -
datadog/trace/instrumentation/jetty_client12/WrapListenerAdvice.class
:dd-java-agent:instrumentation:jetty:jetty-server:jetty-server-10.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 18
-
datadog/trace/instrumentation/jetty10/DispatchableAdvice.class -
datadog/trace/instrumentation/jetty10/ExtractAdapter$Request.class -
datadog/trace/instrumentation/jetty10/ExtractAdapter$Response.class -
datadog/trace/instrumentation/jetty10/ExtractAdapter.class -
datadog/trace/instrumentation/jetty10/HandleAdvice$ContextTrackingAdvice.class -
datadog/trace/instrumentation/jetty10/HandleAdvice.class -
datadog/trace/instrumentation/jetty10/HandleExceptionAdvice.class -
datadog/trace/instrumentation/jetty10/JettyCommitResponseHelper.class -
datadog/trace/instrumentation/jetty10/JettyDecorator$OnResponse.class -
datadog/trace/instrumentation/jetty10/JettyDecorator.class -
datadog/trace/instrumentation/jetty10/JettyOnCommitBlockingHelper$CloseCallback.class -
datadog/trace/instrumentation/jetty10/JettyOnCommitBlockingHelper.class -
datadog/trace/instrumentation/jetty10/ResetAdvice.class -
datadog/trace/instrumentation/jetty10/SendResponseCbAdvice.class -
datadog/trace/instrumentation/jetty10/ServerHandleAdvice.class -
datadog/trace/instrumentation/jetty10/SetContextPathAdvice.class -
datadog/trace/instrumentation/jetty10/SetRequestedSessionIdAdvice.class -
datadog/trace/instrumentation/jetty10/SetServletPathAdvice.class
:dd-java-agent:instrumentation:jetty:jetty-server:jetty-server-11.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 11
-
datadog/trace/instrumentation/jetty11/ExtractAdapter$Request.class -
datadog/trace/instrumentation/jetty11/ExtractAdapter$Response.class -
datadog/trace/instrumentation/jetty11/ExtractAdapter.class -
datadog/trace/instrumentation/jetty11/JettyDecorator.class -
datadog/trace/instrumentation/jetty11/JettyServerAdvice$ContextTrackingAdvice.class -
datadog/trace/instrumentation/jetty11/JettyServerAdvice$HandleAdvice.class -
datadog/trace/instrumentation/jetty11/JettyServerAdvice$ResetAdvice.class -
datadog/trace/instrumentation/jetty11/JettyServerAdvice.class -
datadog/trace/instrumentation/jetty11/RequestURIDataAdapter.class -
datadog/trace/instrumentation/jetty11/SetContextPathAdvice.class -
datadog/trace/instrumentation/jetty11/SetRequestedSessionIdAdvice.class
:dd-java-agent:instrumentation:jetty:jetty-server:jetty-server-12.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 12
-
datadog/trace/instrumentation/jetty12/ExtractAdapter$Request.class -
datadog/trace/instrumentation/jetty12/ExtractAdapter$Response.class -
datadog/trace/instrumentation/jetty12/ExtractAdapter.class -
datadog/trace/instrumentation/jetty12/JettyDecorator.class -
datadog/trace/instrumentation/jetty12/JettyRunnableWrapper.class -
datadog/trace/instrumentation/jetty12/JettyServerAdvice$ContextTrackingAdvice.class -
datadog/trace/instrumentation/jetty12/JettyServerAdvice$HandleAdvice.class -
datadog/trace/instrumentation/jetty12/JettyServerAdvice$ResetAdvice.class -
datadog/trace/instrumentation/jetty12/JettyServerAdvice.class -
datadog/trace/instrumentation/jetty12/RequestURIDataAdapter.class -
datadog/trace/instrumentation/jetty12/ResolveRequestedSessionIdAdvice.class -
datadog/trace/instrumentation/jetty12/SetContextPathAdvice.class
:dd-java-agent:instrumentation:kafka:kafka-clients-3.8
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 29
-
datadog/trace/instrumentation/kafka_clients38/AvroSchemaExtractor.class -
datadog/trace/instrumentation/kafka_clients38/ConstructorAdvice.class -
datadog/trace/instrumentation/kafka_clients38/ConsumerCoordinatorAdvice.class -
datadog/trace/instrumentation/kafka_clients38/DDOffsetCommitCallback.class -
datadog/trace/instrumentation/kafka_clients38/IterableAdvice.class -
datadog/trace/instrumentation/kafka_clients38/IteratorAdvice.class -
datadog/trace/instrumentation/kafka_clients38/KafkaConsumerInfo.class -
datadog/trace/instrumentation/kafka_clients38/KafkaConsumerInstrumentationHelper.class -
datadog/trace/instrumentation/kafka_clients38/KafkaDecorator.class -
datadog/trace/instrumentation/kafka_clients38/KafkaProducerCallback.class -
datadog/trace/instrumentation/kafka_clients38/LegacyConstructorAdvice.class -
datadog/trace/instrumentation/kafka_clients38/ListAdvice.class -
datadog/trace/instrumentation/kafka_clients38/MetadataUpdate22AndAfterAdvice.class -
datadog/trace/instrumentation/kafka_clients38/MetadataUpdateBefore22Advice.class -
datadog/trace/instrumentation/kafka_clients38/NoopTextMapInjectAdapter.class -
datadog/trace/instrumentation/kafka_clients38/OffsetCommitCallbackInvokerAdvice.class -
datadog/trace/instrumentation/kafka_clients38/PayloadSizeAdvice.class -
datadog/trace/instrumentation/kafka_clients38/ProducerAdvice.class -
datadog/trace/instrumentation/kafka_clients38/ProducerConstructorAdvice.class -
datadog/trace/instrumentation/kafka_clients38/ProducerContextPropagationAdvice.class -
datadog/trace/instrumentation/kafka_clients38/RecordsAdvice.class -
datadog/trace/instrumentation/kafka_clients38/TextMapExtractAdapter.class -
datadog/trace/instrumentation/kafka_clients38/TextMapInjectAdapter.class -
datadog/trace/instrumentation/kafka_clients38/TextMapInjectAdapterInterface.class -
datadog/trace/instrumentation/kafka_clients38/TracingIterable.class -
datadog/trace/instrumentation/kafka_clients38/TracingIterableDelegator.class -
datadog/trace/instrumentation/kafka_clients38/TracingIterator.class -
datadog/trace/instrumentation/kafka_clients38/TracingList.class -
datadog/trace/instrumentation/kafka_clients38/TracingListIterator.class
:dd-java-agent:instrumentation:micronaut:micronaut-http-server-netty:micronaut-http-server-netty-4.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 8
-
datadog/trace/instrumentation/micronaut/v4_0/ChannelAcceptAdvice.class -
datadog/trace/instrumentation/micronaut/v4_0/ChannelAcceptAdvice2.class -
datadog/trace/instrumentation/micronaut/v4_0/CreateDefaultErrorResponseAdvice.class -
datadog/trace/instrumentation/micronaut/v4_0/EncodeHttpResponseAdvice.class -
datadog/trace/instrumentation/micronaut/v4_0/EncodeHttpResponseAdvice2.class -
datadog/trace/instrumentation/micronaut/v4_0/EncodeHttpResponseAdvice3.class -
datadog/trace/instrumentation/micronaut/v4_0/HandleRouteMatchAdvice.class -
datadog/trace/instrumentation/micronaut/v4_0/MicronautDecorator.class
:dd-java-agent:instrumentation:servlet:jakarta-servlet-5.0
-
Duplicate source: runtime artifact + runtime file dependency
-
Duplicate class paths: 6
-
datadog/trace/instrumentation/servlet5/AbstractServletInputStreamWrapper.class -
datadog/trace/instrumentation/servlet5/BufferedReaderWrapper.class -
datadog/trace/instrumentation/servlet5/HttpServletGetInputStreamAdvice.class -
datadog/trace/instrumentation/servlet5/HttpServletGetReaderAdvice.class -
datadog/trace/instrumentation/servlet5/Servlet31InputStreamWrapper.class -
datadog/trace/instrumentation/servlet5/ServletBlockingHelper.class
:dd-java-agent:instrumentation:spring:spring-webmvc:spring-webmvc-6.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 18
-
datadog/trace/instrumentation/springweb6/ControllerAdvice.class -
datadog/trace/instrumentation/springweb6/ErrorHandlerAdvice.class -
datadog/trace/instrumentation/springweb6/FilterInjectingAdvice.class -
datadog/trace/instrumentation/springweb6/HandleMatchAdvice.class -
datadog/trace/instrumentation/springweb6/HandlerMappingAdvice.class -
datadog/trace/instrumentation/springweb6/HandlerMappingResourceNameFilter$BeanDefinition.class -
datadog/trace/instrumentation/springweb6/HandlerMappingResourceNameFilter.class -
datadog/trace/instrumentation/springweb6/InterceptorPreHandleAdvice.class -
datadog/trace/instrumentation/springweb6/OrderedServletPathRequestFilter$BeanDefinition.class -
datadog/trace/instrumentation/springweb6/OrderedServletPathRequestFilter.class -
datadog/trace/instrumentation/springweb6/PairList.class -
datadog/trace/instrumentation/springweb6/PathMatchingHttpServletRequestWrapper.class -
datadog/trace/instrumentation/springweb6/RenderAdvice.class -
datadog/trace/instrumentation/springweb6/ResourceNameFilterMappingAdvice.class -
datadog/trace/instrumentation/springweb6/ServletPathFilterInjectingAdvice.class -
datadog/trace/instrumentation/springweb6/ServletRequestURIAdapter.class -
datadog/trace/instrumentation/springweb6/SpringWebHttpServerDecorator.class -
datadog/trace/instrumentation/springweb6/WrapContinuableResultAdvice.class
:dd-java-agent:instrumentation:websocket:jetty-websocket:jetty-websocket-10.0
-
Duplicate source: class output + runtime artifact
-
Duplicate class paths: 5
-
datadog/trace/instrumentation/websocket/jetty10/MethodHandleWrappers.class -
datadog/trace/instrumentation/websocket/jetty10/WebSocketAdvices$MessageSinkAdvice.class -
datadog/trace/instrumentation/websocket/jetty10/WebSocketAdvices$OpenClose10Advice.class -
datadog/trace/instrumentation/websocket/jetty10/WebSocketAdvices$OpenClose9Advice.class -
datadog/trace/instrumentation/websocket/jetty10/WebSocketAdvices.class
What Does This Do
Upgrades the Gradle Shadow plugin from 8.3.9 to 9.4.2 in the main build.
The build wiring is updated for Shadow 9 APIs and behavior:
ShadowJarconfiguration inputs where practicalMotivation
Shadow 9 brings several build-pipeline fixes that are useful for this project:
Migrate
ShadowJarto using lazy properties GradleUp/shadow#1044Resolve
Configurationdirectly inDependencyFilterGradleUp/shadow#1045Refactor file visiting logic in StreamAction GradleUp/shadow#1233
Default
duplicatesStrategyback toEXCLUDEGradleUp/shadow#1617Merge dependency and project overloads in
DependencyFilterGradleUp/shadow#1328Support relocating list of types in
RelocatorRemapperGradleUp/shadow#1714Update
RelocatorRemapperclass pattern to cover more Java method descriptors GradleUp/shadow#1731Additional Notes
bric3/jardiffwas used between a master 11646d3 agent jar and this this branch's agent jar using ASM textifier output.The difference is in a string constant inside JaCoCo’s
LoggerRuntime, not inPatchLogger.javaitself.Shadow 9.2+ started relocating descriptor-shaped string constants. So this
LDCchanged:That comes from the
org.jacoco.core.runtime.LoggerRuntimeclass included via:dd-java-agent:agent-ci-visibility.Before Shadow 9, relocation had already changed the method owner to
datadog/trace/bootstrap/PatchLogger, but the descriptor string still said the method returnedjava.util.logging.Logger. That describes a method that does not exist:PatchLogger.getLogger(String):Logger. Shadow 9 rewrites the descriptor too, so generated bytecode calls the real method:PatchLogger.getLogger(String):PatchLogger, which matchesdatadog.trace.bootstrapPatchLoggerin:dd-java-agent:agent-bootstrap.https://github.com/DataDog/dd-trace-java/blob/987a0eba81a1121ea037005640cd0f407aaf5389/dd-java-agent/agent-bootstrap/src/main/java/datadog/trace/bootstrap/PatchLogger.java#L16-L19
In my opinion this looks like a correctness fix for that generated-accessor path, not a behavioral regression. It is also aligned with the agent relocation rule that intentionally redirects
java.util.logging.LoggertoPatchLoggerto avoid JUL initialization during premain:https://github.com/DataDog/dd-trace-java/blob/987a0eba81a1121ea037005640cd0f407aaf5389/dd-java-agent/build.gradle#L114-L116
Coverage-wise:
The Maven and Gradle smoke tests cover the real CI Visibility JaCoCo integration path: they enable the JaCoCo plugin version and assert received coverages/reports.
dd-trace-java/dd-smoke-tests/maven/src/test/java/datadog/smoketest/MavenSmokeTest.java
Lines 84 to 102 in 11646d3
dd-trace-java/dd-smoke-tests/gradle/src/test/java/datadog/smoketest/GradleDaemonSmokeTest.java
Lines 96 to 112 in 11646d3
That path uses the external JaCoCo agent runtime, which our instrumentation targets via
org.jacoco.agent.rt.internal...:dd-trace-java/dd-java-agent/instrumentation/jacoco-0.8.9/src/main/java/datadog/trace/instrumentation/jacoco/InstrumenterInstrumentation.java
Lines 28 to 35 in 11646d3
The shaded
org.jacoco.coredependency inagent-ci-visibilityappears used for parsing/analyzing coverage data and reports, not forLoggerRuntime:dd-trace-java/dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/coverage/report/JacocoCoverageProcessor.java
Lines 233 to 234 in 11646d3
dd-trace-java/dd-java-agent/agent-ci-visibility/src/main/java/datadog/trace/civisibility/coverage/line/LineCoverageStore.java
Lines 27 to 29 in 11646d3
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issue/merge.Jira ticket: [PROJ-IDENT]