Bundler::GemNotFound when starting running rails commands when Gemfile has changed dependencies

[technicalpickles]

Let's say hypothetically, I was away at a conference for a week, and get back on Monday to pull the latest changes and start work. I've found that running rails server, or really any subcommand (I was trying rails db:drop), you end up getting a stacktrace because some gems couldn't be found:

❯ bin/rails db:drop
You're missing some gems. Run `bundle install` to fix it.
Traceback (most recent call last):
        21: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/bin/spring:49:in `<main>'
        20: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/client.rb:30:in `run'
        19: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/client/command.rb:7:in `call'
        18: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/client/server.rb:9:in `call'
        17: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        16: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        15: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/server.rb:9:in `<top (required)>'
        14: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        13: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
        12: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/commands.rb:4:in `<top (required)>'
        11: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/gems/2.7.0/gems/spring-4.0.0/lib/spring/commands.rb:33:in `<module:Spring>'
        10: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
         9: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:85:in `require'
         8: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/setup.rb:20:in `<top (required)>'
         7: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/ui/shell.rb:88:in `silence'
         6: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/ui/shell.rb:136:in `with_level'
         5: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/setup.rb:20:in `block in <top (required)>'
         4: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler.rb:151:in `setup'
         3: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/runtime.rb:18:in `setup'
         2: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/definition.rb:239:in `specs_for'
         1: from /Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/definition.rb:191:in `specs'
/Users/josh.nichols/.asdf/installs/ruby/2.7.5/lib/ruby/site_ruby/2.7.0/bundler/definition.rb:486:in `materialize': Could not find aws-partitions-1.588.0, vite_ruby-3.1.1 in any of the sources (Bundler::GemNotFound)

bin/rails is the generated binstub for spring:

#!/usr/bin/env ruby
load File.expand_path("spring", __dir__)
APP_PATH = File.expand_path('../config/application', __dir__)
require_relative "../config/boot"
require "rails/commands"

That would make sense it load spring, and try to start a server. The place it falls down is here:

spring/lib/spring/commands.rb

Lines 31 to 33 in 184ecda

	# If the config/spring.rb contains requires for commands from other gems,
	# then we need to be under bundler.
	require "bundler/setup"

Finding that, I saw that ~/.spring.rb is loaded. I was able to add this to that file and get better error:

❯ cat ~/.spring.rb
begin
        require "bundler/setup"
rescue Bundler::GemNotFound
        $stderr.puts "Missing gems from your Gemfile. Trying running `bundle install` to fix!"
        exit 1
end

❯ rails runner "puts 'hi'"                                                     
Missing gems from your Gemfile. Trying running `bundle install` to fix!

Interestingly, bin/spring server shows a colorful error with the info you need:

❯ bin/spring server
Could not find aws-partitions-1.588.0, vite_ruby-3.1.1 in any of the sources
Run `bundle install` to install missing gems.

[technicalpickles]

Looked more closely at bundler/setup, and see where it rescues and prints a better message
https://github.com/rubygems/rubygems/blob/20e9b96b92664fb86f11ee4f76233145a5dea5ab/bundler/lib/bundler/setup.rb#L9-L18

  if STDOUT.tty? || ENV["BUNDLER_FORCE_TTY"]
    begin
      Bundler.ui.silence { Bundler.setup }
    rescue Bundler::BundlerError => e
      Bundler.ui.error e.message
      Bundler.ui.warn e.backtrace.join("\n") if ENV["DEBUG"]
      if e.is_a?(Bundler::GemNotFound)
        Bundler.ui.warn "Run `bundle install` to install missing gems."
      end
      exit e.status_code
    end
  else
    Bundler.ui.silence { Bundler.setup }
  end

I think then the difference can be explained by bin/spring server still running as a TTY, whereas calling bin/rails is starting the spring server in the background.

Knowing that an environment variable can force it, this seems to get the error I'm looking for:

❯ BUNDLER_FORCE_TTY=1 bin/rails db:drop
Could not find aws-partitions-1.588.0, vite_ruby-3.1.1 in any of the sources
Run `bundle install` to install missing gems.

[rafaelfranca]

I'm not sure what is the issue here. Is the fact that you see a backtrace instead of a formatted error?

[technicalpickles]

@rafaelfranca that's right. These are CLI tools used by developers, and would prefer to give a formatted message with directions rather than a stack trace that isn't super clear how to fix.

[technicalpickles]

@rafaelfranca awesome, thanks!