Perform the install triggered by auto_install in a subprocess

[meganemura]

What was the end-user or developer problem that led to this PR?

With auto_install enabled, require "bundler/setup" (and bundle exec) can fail with a Gem::LoadError the first time gems are missing:

Automatically installing missing gems.
Fetching gem metadata from https://rubygems.org/...
Installing rake 13.4.2
Installing openssl 3.2.1 with native extensions
Bundle complete! 2 Gemfile dependencies, 2 gems now installed.
/path/to/lib/bundler/runtime.rb:328:in 'Bundler::Runtime#check_for_activated_spec!': You have already activated openssl 3.2.0, but your Gemfile requires openssl 3.2.1. Since openssl is a default gem, you can either remove your dependency on it or try updating to a newer version of bundler that supports openssl as a default gem. (Gem::LoadError)

The automatic install itself succeeds, so running the same command again works. That makes it look unreproducible, and both suggestions in the error message are dead ends: openssl is usually a transitive dependency that can't be dropped from the Gemfile, and the bundler in use is already current.

Reproduction, on a Ruby whose default openssl differs from the pinned version:

ruby -v                 # verified with ruby 3.3.1 and ruby 4.0.5
gem list -e openssl     # the pin below must differ from this

WORK="$(mktemp -d)"; cd "$WORK"; mkdir gems
export GEM_HOME="$WORK/gems" GEM_PATH="$WORK/gems"
unset BUNDLE_GEMFILE RUBYOPT

cat > Gemfile <<'EOF'
source "https://rubygems.org"
gem "openssl", "3.2.1"   # any released version != this Ruby's default openssl
gem "rake"               # any gem at all, it only has to be missing
EOF

bundle lock
BUNDLE_AUTO_INSTALL=true ruby -e 'require "bundler/setup"; puts "OK"'

Expected OK, got the Gem::LoadError above. What does not reproduce it:

Condition	Result
bundle exec <binstub>, gems missing	same Gem::LoadError (binstubs are Kernel.loaded in process)
bundle exec ruby -e ..., gems missing	passes, ruby is Kernel.execed so activation is reset
second run, gems now installed	passes
auto_install disabled	Bundler::GemNotFound, which is clear and actionable
plain bundle install	always fine, it never calls Bundler.setup

Only auto_install, which chains an install and a Bundler.setup in one process, breaks.

What is your fix for the problem, implemented in this PR?

Why it happens

lib/bundler/setup.rb calls Bundler.auto_install before Bundler.setup, and Bundler.auto_install runs CLI::Install in the same process:

# lib/bundler.rb
def auto_install
  return unless Bundler.settings[:auto_install]

  begin
    definition.specs
  rescue GemNotFound, GitError
    ui.info "Automatically installing missing gems."
    reset!
    CLI::Install.new({}).run   # in process: talks HTTPS, so requires `openssl`
    reset!
  end
end

1. The install talks to the gem source over HTTPS, which requires openssl.
2. Bundler.setup has not run yet, so no Gemfile requirement is in effect. RubyGems activates the newest installed version, which is the default gem while the locked one is still missing.
3. Activation is one version per process and cannot be undone. reset! only drops Bundler's Definition, not RubyGems' activated specs.
4. Bundler.setup then activates the lockfile's version and Runtime#check_for_activated_spec! raises.

bundle exec reaches the same code through lib/bundler/cli.rb (AUTO_INSTALL_CMDS).

Bundler normally avoids this class of conflict by vendoring what it needs (lib/bundler/vendor/: net-http-persistent, uri, securerandom, thor, …). openssl is a C extension and can't be vendored, so it is the hole left in that defense.

The fix

Run the install in a forked child, so the process that boots the application never activates gems that were only needed to install. bundler/inline already does exactly this, for exactly this reason:

# lib/bundler/inline.rb
# When possible we do the install in a subprocess because to install
# gems we need to require some default gems like `securerandom` etc
# which may later conflict with the Gemfile requirements.

Platforms without fork (Windows, JRuby) keep installing in process, as in bundler/inline.

Differences from bundler/inline, and why

• The child is wrapped in with_friendly_errors. The handler exe/bundle wraps around the parent cannot see a failure in the child: fork's block runs under rb_protect and the child exits without unwinding into the parent's frames. Verified: with an enclosing rescue Exception around the fork call, the rescue never fires and the child prints the exception itself. Without wrapping, a failed automatic install under bundle exec would degrade from a friendly message and a proper status code to a raw backtrace and exit 1.

• The child ends with exit!, after flushing by hand. fork { ... } exits the child normally, which runs the at_exit handlers of the program that is booting — they would run once in the install child and once in the real process. exit! skips them, but it also skips flushing, hence the explicit flush.

• The parent flushes before forking, so the child cannot inherit and re-emit buffered output.

• No Gem.load_yaml before the fork. bundler/inline calls it, but here it would defeat the fix: on Ruby 3.2, Gem.load_yaml requires psych unconditionally, activating the default psych in the parent, which is the very conflict this PR removes. Measured on Ruby 3.2.4 / RubyGems 3.4.19 with the new spec below:

  lib/bundler.rb	result
  before this PR	Gem::LoadError: already activated psych 5.0.1, but your Gemfile requires psych 999
  fork + Gem.load_yaml	same Gem::LoadError
  fork only (this PR)	passes

  The NameError that call is meant to prevent did not occur, because Bundler.setup has already loaded the real Psych by the time YAML is loaded. bundler/inline can afford the pre-activation because it recovers by re-resolving with the activated version; that is not an option here, since Bundler.setup must activate what the lockfile says.

• No $VERBOSE = nil in the child. Nothing in this path emits warnings under ruby -w, and silencing them would hide warnings from a real bundle install triggered during application boot.

Tests

• spec/bundler/bundler_spec.rb: the install runs in a subprocess, and a failed install exits with the child's status code.
• spec/runtime/setup_gems_spec.rb: an automatic install of a bundle that locks a default gem to another version. openssl can't be used here because the suite stubs HTTPS, so it uses a psych 999 fixture, the same stand-in spec/runtime/inline_spec.rb already uses. This one reproduces the bug in the system-rubygems-bundler job, where RubyGems is old enough to require psych while installing.

Make sure the following tasks are checked

• Describe the problem / feature
• Write tests for features and bug fixes
• Write code to solve the problem
• Make sure you follow the current code style and write meaningful commit messages without tags