Bundler 4.0.1 trying to find gem for every platform

[jcoyne]

Describe the problem as clearly as you can

When running a rails generator within a Bundle.with_unbundled_env we get this error: (see https://github.com/projectblacklight/blacklight/actions/runs/20079840341/job/57603875568?pr=3783). This was not a problem when using bundler 2.

       rails  importmap:install
/opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:356:in 'Bundler::Resolver#raise_not_found!': Could not find gems matching 'sqlite3 (>= 2.1)' valid for all resolution platforms (aarch64-linux-gnu, aarch64-linux-musl, arm-linux-gnu, arm-linux-musl, arm64-darwin, x86_64-darwin, x86_64-linux-gnu, x86_64-linux-musl, x86_64-linux) in locally installed gems. (Bundler::GemNotFound)

The source contains the following gems matching 'sqlite3 (>= 2.1)':
  * sqlite3-2.8.1-x86_64-linux-gnu
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:450:in 'block in Bundler::Resolver#prepare_dependencies'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:425:in 'Hash#each'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:425:in 'Enumerable#filter_map'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:425:in 'Bundler::Resolver#prepare_dependencies'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:63:in 'Bundler::Resolver#setup_solver'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/resolver.rb:28:in 'Bundler::Resolver#start'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/definition.rb:755:in 'Bundler::Definition#start_resolution'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/definition.rb:350:in 'Bundler::Definition#resolve'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/definition.rb:668:in 'Bundler::Definition#materialize'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/definition.rb:240:in 'Bundler::Definition#specs'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/definition.rb:312:in 'Bundler::Definition#specs_for'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/runtime.rb:18:in 'Bundler::Runtime#setup'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler.rb:165:in 'Bundler.setup'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/setup.rb:32:in 'block in <top (required)>'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/ui/shell.rb:177:in 'Bundler::UI::Shell#with_level'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/ui/shell.rb:122:in 'Bundler::UI::Shell#silence'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/gems/3.4.0/gems/bundler-4.0.1/lib/bundler/setup.rb:32:in '<top (required)>'
	from <internal:/opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/3.4.0/rubygems/core_ext/kernel_require.rb>:136:in 'Kernel#require'
	from <internal:/opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/3.4.0/rubygems/core_ext/kernel_require.rb>:136:in 'Kernel#require'
	from /opt/hostedtoolcache/Ruby/3.4.7/x64/lib/ruby/3.4.0/rubygems.rb:1417:in '<top (required)>'
	from <internal:gem_prelude>:2:in 'Kernel#require'
	from <internal:gem_prelude>:2:in '<internal:gem_prelude>'

Did you try upgrading rubygems & bundler?

Yes

Post steps to reproduce the problem

Run any build in

Which command did you run?

rails importmap:install

What were you expecting to happen?

the generator runs

What happened instead?

see stacktrace from above.

Replace this with the actual result you got. Paste the output of your command here.

If not included with the output of your command, run bundle env and paste the output below

[jcoyne]

Here is the engine_cart test suite showing the same problem: https://github.com/cbeer/engine_cart/actions/runs/20080636540/job/57606563007?pr=132

[Edouard-chin]

I think it's the same issue as #9130, can you try the solution I mentioned in the comment.

[jcoyne]

@Edouard-chin yes, that does appear to be the same. I'm not sure which solution you are referring to. Use Rails main branch?

[Edouard-chin]

#9130 (comment)

Wrap whatever code creates a bundle install subprocess with Bundle.with_unbundled_env.
I believe in your project what runs bundle install is in the engine_cart library, so this would need to be fixed here https://github.com/cbeer/engine_cart/blob/b89464ce94b3c168c296e9ec893482b4ca673e74/Rakefile#L22

[jrochkind]

@Edouard-chin If we have a project that tries to for instance run rails new or rails generate something (a generator that will call bundle install, as some do) -- then it's actually rails that is running bundle install, although we are calling rails.

Are we out of luck unless using a version of Rails that fixes this? Or wrapping our own call to rails in with_unbundled_env should work? We thought we already were wrapping in with_unbundled_env in necessary places, as mentioned in description of this issue, although it is possible we missed a place.

This is feeling enormously confusing to me.

[Edouard-chin]

I opened a fix in the engine_cart gem. The issue wasn't related to #9130. But the issue isn't related to the 4.0.1 release, the engine_cart test suite is broken since Bundler 2.5.0