[NativeAOT] Enable workload-provided linker for NDK-free builds#11303
Open
sbomer wants to merge 21 commits into
Open
[NativeAOT] Enable workload-provided linker for NDK-free builds#11303sbomer wants to merge 21 commits into
sbomer wants to merge 21 commits into
Conversation
Add NativeAotBuildTests as the dedicated test class for NativeAOT-specific build tests. First test verifies that NativeAOT builds fail when the NDK is not available, by clearing AndroidNdkDirectory via build parameters. This test will be flipped to assert success when the workload pack ships its own linker and sysroot files, removing the NDK dependency. Context: #10697 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add crtbegin_so.o and crtend_so.o to the NativeAOT runtime pack. These were only included for CoreCLR. They are needed for NDK-free linking where the workload pack provides all linker dependencies. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NativeAOT links dotnet/runtime's libSystem.Native.a which was compiled against API 24 and references symbols like __gnu_strerror_r (API 23+) and __write_chk (API 24+). The existing redist/ directory has API 21 stubs which lack these symbols. Add a separate redist-nativeaot/ directory with CRT objects and system lib stubs from the API 24 sysroot. The NativeAOT runtime pack sources from this directory instead of the shared redist/ (which stays at API 21 for Mono/CoreCLR compatibility). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add libc++_static.a, libc++abi.a, libclang_rt.builtins, and libunwind.a to the NativeAOT pack. These were only in the CoreCLR pack but NativeAOT needs them for the workload linker path (no NDK). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an internal opt-in path for NativeAOT builds to link using workload/runtime-pack-provided native artifacts (CRT, sysroot stubs, toolchain libs) so NativeAOT can build without a locally installed Android NDK, and updates pack/prep and tests to support this.
Changes:
- Introduces
_AndroidUseWorkloadNativeLinkerinMicrosoft.Android.Sdk.NativeAOT.targetsto source linker inputs from the runtime pack and to disable debug-section compression when using the workload linker. - Extends
make prepare/pack creation to include NativeAOT-specific API-24 CRT + system stub libs (redist-nativeaot/) and additional toolchain libraries in the NativeAOT runtime pack. - Adds NativeAOT build tests to validate behavior with and without NDK depending on the opt-in flag, plus a builder switch to skip injecting
AndroidNdkDirectoryin test response files.
Show a summary per file
| File | Description |
|---|---|
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/Builder.cs | Adds a test-only switch to skip emitting AndroidNdkDirectory into the generated response file. |
| src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/NativeAotBuildTests.cs | Adds coverage for NativeAOT builds with no NDK (expected fail by default; expected success with workload linker opt-in). |
| src/Xamarin.Android.Build.Tasks/Tasks/LinkNativeAotSharedLibrary.cs | Plumbs a new task property to control --compress-debug-sections based on linker capabilities. |
| src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.NativeAOT.targets | Implements the opt-in workload-linker path and runtime-pack-based resolution for CRT/stubs/toolchain libs. |
| build-tools/xaprepare/xaprepare/Steps/Step_Android_SDK_NDK.cs | Copies API-24 CRT + system stubs into a new redist-nativeaot/ directory for NativeAOT runtime pack consumption. |
| build-tools/create-packs/Microsoft.Android.Runtime.proj | Includes redist-nativeaot/ assets and additional toolchain libraries in the NativeAOT runtime pack. |
Copilot's findings
- Files reviewed: 7/7 changed files
- Comments generated: 2
Neither ILC nor the CoreCLR/Mono link paths pass --compress-debug-sections. This was added without justification and created unnecessary complexity for the workload linker path (whose LLD lacks zlib support). Tracked in #11304 for future enablement across all runtimes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add XML doc comment for SkipNdkDirectory property - Clarify test comment: default path still requires NDK Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Enable ProcessRuntimePackLibraryDirectories for NativeAOT by removing the != NativeAOT condition and adding the NativeAOT package prefix to IsInSupportedRuntimePack. This reuses the existing CoreCLR/Mono mechanism to remove sysroot stub .so files (libc.so, libdl.so, etc.) from ResolvedFileToPublish so they don't end up in the APK. Also make DSOWrapperGenerator.GetConfig skip gracefully when libarchive-dso-stub.so is not found in the runtime pack directory, since the NativeAOT pack doesn't include this CoreCLR/Mono-specific file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
sbomer
force-pushed
the
dev/sbomer/nativelink-workload
branch
from
b15ac05 to
bcccc62
Compare
…overy Setting AndroidNdkDirectory to empty is not sufficient because ResolveSdks has a fallback chain (SDK ndk-bundle dir, environment variables, PATH) that discovers the NDK regardless of the input hint. Override _AndroidNdkDirectory (the internal resolved path) via /p: which has highest MSBuild precedence and cannot be overridden by task outputs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The NDK path is already passed via /p:AndroidNdkDirectory in Builder.cs for every test build, making the project-level property set by SetPublishAot redundant. The duplicate property also prevented SkipNdkDirectory from working correctly, since the project-level property still fed ResolveSdks. Changes: - Remove the androidNdkPath parameter from SetPublishAot - Update all three callers to use the simplified signature Assisted-by: Claude:claude-opus-4.6 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
# Conflicts: # external/xamarin-android-tools
ResolveSdks always discovers NDK through its fallback chain (SDK dir, env vars, PATH), so neither /p:_AndroidNdkDirectory= nor SkipNdkDirectory on Builder.cs could prevent NDK resolution on CI agents with NDK installed. Add a _SkipNdkResolution property that clears _AndroidNdkDirectory after ResolveSdks runs. Tests pass /p:_SkipNdkResolution=true to reliably simulate a missing NDK. Remove the unused SkipNdkDirectory flag from Builder.cs since _SkipNdkResolution handles this at the MSBuild level. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
…nker When _AndroidUseWorkloadNativeLinker is enabled, SetNdkPathForIlc should use AndroidBinUtilsDirectory instead of _NdkBinDir to add tools like llvm-objcopy and llvm-ar to PATH. Without this, ILC's SetupOSSpecificProps fails on machines where these tools aren't in the system PATH (e.g., macOS CI agents) because the NDK bin directory is empty. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
The task is no longer NDK-specific — it adds any toolchain bin directory to PATH for ILC. Rename the class to SetIlcToolchainPath and the parameter from NdkBinDirectory to ToolchainBinDirectory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
Comment on lines
+233
to
+237
| // NativeAOT only supports 64-bit ABIs and links against dotnet/runtime's | ||
| // libSystem.Native.a which was compiled targeting a higher API level. Copy a | ||
| // second set of CRT/system lib stubs from the higher API level sysroot for the | ||
| // NativeAOT runtime pack. | ||
| bool is64BitAbi = abi == "arm64-v8a" || abi == "x86_64"; |
Member
There was a problem hiding this comment.
we will be adding 32-bit arm runtimepack (armel) soon: dotnet/runtime#125712 - what will need to change once it is available? is there something we can do here to make it easier to revisit this part of the codebase in near future?
Member
Author
There was a problem hiding this comment.
I looked into this and found the SupportNativeAOT metadata here:
android/build-tools/scripts/Ndk.projitems.in
Lines 19 to 63 in d3d2b5d
However, I think #11331 will remove the need for the separate file copy and this can go away entirely.
ILC tools (llvm-ar, llvm-objcopy) were resolved via bare names on PATH, silently falling through to system-installed versions. This caused failures on macOS CI where the tools aren't on the system PATH, and could silently pick up wrong versions on machines that happen to have them installed. Use full paths from either the NDK bin directory or AndroidBinUtilsDirectory (when using the workload linker) so there is no PATH fallback. If the tool doesn't exist at the expected location, the build fails immediately. Also add llvm-ar to the workload binutils pack (Configurables.cs, unix-binutils.projitems, create-installers.targets) so it is available for the workload linker path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
Override ILC's LinkNative target as a no-op in Microsoft.Android.Sdk.After.targets. With NativeLib=static, LinkNative only runs llvm-ar to create a .a archive from the ILC-compiled .o file. We never consume that .a — _AndroidLinkNativeAotSharedLibrary links the .o directly into a .so. The override must be in After.targets because: 1. MSBuild uses the last target definition (last definition wins) 2. The ILC NuGet package targets are imported after the workload SDK targets 3. After.targets is imported after both, so its definition takes precedence This eliminates the need for llvm-ar (not shipped in the workload toolchain), CppLibCreator, and the LinkerFlavor workaround for dotnet/runtime#126978. Reverts the llvm-ar additions to binutils packaging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
Override ILC's LinkNative target as a no-op in Microsoft.Android.Sdk.After.targets. With NativeLib=static, LinkNative only runs llvm-ar to create a .a archive from the ILC-compiled .o file. We never consume that .a — _AndroidLinkNativeAotSharedLibrary links the .o directly into a .so. The override must be in After.targets because: 1. MSBuild uses the last target definition (last definition wins) 2. The ILC NuGet package targets are imported after the workload SDK targets 3. After.targets is imported after both, so its definition takes precedence This eliminates the need for llvm-ar (not shipped in the workload toolchain), CppLibCreator, and the LinkerFlavor workaround for dotnet/runtime#126978. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
Extract the inline LinkNative no-op target from After.targets into a dedicated Microsoft.Android.Sdk.NativeAOT.After.targets file. This keeps After.targets as a pure import list and puts the NativeAOT-specific ILC override in its own file, consistent with the pattern of separating runtime-specific logic. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
…tivelink-workload # Conflicts: # src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.After.targets # src/Xamarin.Android.Build.Tasks/Microsoft.Android.Sdk/targets/Microsoft.Android.Sdk.NativeAOT.targets
… dependency Two fixes: 1. native.targets: Add redist-nativeaot glob to NativeAOT _CopyToPackDirs so crtbegin_so.o, crtend_so.o, and system lib stubs are included in the NativeAOT runtime pack. Previously only *.a was globbed, missing *.o files. 2. NativeAOT.After.targets: Preserve DependsOnTargets=$(LinkNativeDependsOn) on the LinkNative override so IlcCompile still runs. Without this, the ILC compilation was skipped entirely, leaving NativeLibrary items empty. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Assisted-by: Claude:claude-opus-4.6-1m
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[NativeAOT] Enable workload-provided linker for NDK-free builds
Adds an opt-in
_AndroidUseWorkloadNativeLinkerproperty that allows NativeAOT builds to use the linker and sysroot files from the workload runtime pack instead of requiring the Android NDK. This is the foundation for removing the NDK dependency from NativeAOT builds (issue #10697).What changed
Workload linker opt-in (
Microsoft.Android.Sdk.NativeAOT.targets)_AndroidUseWorkloadNativeLinker=true, the NativeAOT link step resolves CRT objects, system lib stubs, and toolchain libraries from the runtime pack instead of the NDK.NativeAOT-specific redist directory (
Step_Android_SDK_NDK.cs)redist-nativeaot/duringmake prepare.libSystem.Native.areferences symbols introduced at that level (__gnu_strerror_r,__write_chk,preadv,pwritev,stderr).redist/.Runtime pack contents (
Microsoft.Android.Runtime.proj)crtbegin_so.o,crtend_so.o) and system lib stubs fromredist-nativeaot/(API 24).libc++_static.a,libc++abi.a,libclang_rt.builtins,libunwind.a) that were previously only in the CoreCLR pack.Tests (
NativeAotBuildTests.cs)BuildNativeAot_WithoutNdk_Fails: Verifies that NativeAOT builds fail without NDK when the workload linker is not enabled.BuildNativeAot_WithWorkloadLinker_WithoutNdk: Verifies that NativeAOT builds succeed with the workload linker and no NDK installed.Context
This is the third step in the NDK-free NativeAOT linking series:
The
_AndroidUseWorkloadNativeLinkerproperty is underscore-prefixed (internal) for now. A future PR will make it the default.Contributes to #10697.