[2.x] Tracking issue for RID-related binary loading issues on Linux

[Exanite]

Summary

This is an issue I'm opening specifically to track the other open issues related to the RID-related binary loading issue. Solving this issue should solve the related issues.

The symptom of this issue is that Silk 2 fails to find binaries located in the runtimes folder.
Moving the binary out and next to the built executable is a common workaround for this issue.

At its core, the problem is the following:

// linux-x64
Console.WriteLine(System.Runtime.InteropServices.RuntimeInformation.RuntimeIdentifier);

// vanilla.2.0-x64 - This is what Silk 2 uses
Console.WriteLine(Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment.GetRuntimeIdentifier());

vanilla... and many other RIDs obtained from the PlatformAbstractions call don't exist as part of the RID graph found here: https://github.com/dotnet/runtime/blob/main/src/libraries/Microsoft.NETCore.Platforms/src/runtime.json

Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment.GetRuntimeIdentifier() is called here:

Silk.NET/src/Core/Silk.NET.Core/Loader/DefaultPathResolver.cs

Line 357 in 57e0f86

foreach (var rid in GetAllRuntimeIds(RuntimeEnvironment.GetRuntimeIdentifier(), DependencyContext.Default))

Steps to reproduce

For a given affected distro, run the code above and see what RIDs are produced. The second RID is the one that Silk uses. If that RID is not part of the RID graph, then that's why the binary fails to be found as part of the runtimes folder.

Comments

There's a lot of workarounds and fixes suggested in these issues, but I think a better fix is that we make use of the System.Runtime.InteropServices.RuntimeInformation.RuntimeIdentifier value instead.

From what I remember, modern .NET is no longer including per-distro RIDS anymore, so it makes sense that these distro RIDs are not part of the graph.

I also remember seeing some #if preprocessors related to the RuntimeInformation code, so I'm guessing that the RuntimeInformation APIs are not available on .NET Standard or something. If my guess is wrong, then this fix is easy and we just use both so we preserve the old behavior and fallback to the new.

My investigation of this issue can be found here: https://discord.com/channels/521092042781229087/607634593201520651/1540594381956972695

Edit: Apparently this exact underlying issue was mentioned here and probably not investigated further: https://discord.com/channels/521092042781229087/607634593201520651/1331176382339743755

Directly related issues

Solving this issue should close all of the following issues:

• Silk.net window fails #2601 (and technically unresolved, Package native binaries with nuget package #2599)
• [Linux] runtimes/<rid>/native/ not probed on Ubuntu — _linuxRiDs fallback list missing "ubuntu" (regression of #652) #2571
• RuntimesFolderResolver fails on Ubuntu #2568
• Silk.NET.WebGPU can't load native library on linux-x64 #2465
  • RID/distro not specified, but very likely to be this issue

Fedora is part of the RID graph, but only versions v39 and before. Latest is v44.
Ubuntu is part of the RID graph, but only versions v23 and before. Latest is v26.

Vanilla OS is another report I got from a user in Discord (https://discord.com/channels/521092042781229087/607634593201520651/1540577683195568178), which is also not part of the RID graph.

Once this issue is fixed and a new version of Silk is published, we should go through these issues, ask if the issue is fixed, and close the issues after a week or so if resolved or no response.

[Perksey]

We likely want to try both RuntimeEnvironment.GetRuntimeIdentifier() and RuntimeInformation.RuntimeIdentifier for the best chance of success.

It also wouldn't hurt to add a resolver that uses AppContext.GetData("NATIVE_DLL_SEARCH_DIRECTORIES") which is available on newer runtimes and again gives us the best chance of finding the native binaries. Notably, this gives us the path we need on newer runtimes right away, but it isn't available on all versions and all variants, meaning we still need the other resolvers.