Test failures on fresh project checkout

[mttkay]

There are 3 tests that fail locally after a fresh checkout of the project on master:

$ bundle exec rspec
...
Finished in 5.1 seconds (files took 0.3487 seconds to load)
157 examples, 3 failures

Failed examples:

rspec ./spec/license_scout/dependency_manager/bundler_spec.rb:134 # LicenseScout::DependencyManager::Bundler#dependencies bundler 2.x project returns an array of Dependencies found in the directory
rspec ./spec/license_scout/dependency_manager/cargo_spec.rb:86 # LicenseScout::DependencyManager::Cargo#dependencies cargo project returns an array of Dependencies found in the directory
rspec ./spec/license_scout/dependency_manager/mix_spec.rb:80 # LicenseScout::DependencyManager::Mix#dependencies returns an array of Dependencies found in the directory

testoutput.log

I'm not sure what the bundler related errors refer to. I ran bundle install so all dependencies as declared by Gemfile are installed:

$ bundle install
...
Bundle complete! 10 Gemfile dependencies, 83 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.

It appears that some tests require Bundler 2.x, but the project was bundled with Bundler 1.7 as per Gemfile.lock:

BUNDLED WITH
   1.17.2

My understanding is that whatever version is specified here is the required version of Bundler when interacting with this repository.

By the time I ran these tests, my Bundler version matched the BUNDLED_WITH version:

$ bundle -v    
Bundler version 1.17.2

[mttkay]

If I understand these tests correctly they rely on platform / host specifics to execute. That is not a good idea and will cause problems with setups that isolate Ruby dependencies and binaries such as rbenv or rvm. These specs test scenarios with both Bundler 1 and 2 and rely on them to co-exist. Since this project only requires Bundler 1.x as per Gemfile.lock, this is not going to work when license_scout dependencies are sandboxed.

Tests should be repeatable and isolated from their execution environment. Perhaps the solution here is to rely on vendored bundlers instead, so that you can execute them predictably across any environment?