Bundler installation fails on Ruby 2.1 due to RubyGems.org dependency API deprecation

[philr]

Ensure the following before filing this issue

• I verified it reproduces with the latest version with - uses: ruby/setup-ruby@v1 (see Versioning policy)

• I tried to reproduce the issue locally by following the workflow steps (including all commands done by ruby/setup-ruby, except for Downloading Ruby & Extracting Ruby),
  and it did not reproduce locally (if it does reproduce locally, it's not a ruby/setup-ruby issue)

Are you running on a GitHub-hosted runner or a self-hosted runner?

GitHub-hosted runner

Link to the failed workflow job (must be a public workflow job, so the necessary information is available)

https://github.com/tzinfo/tzinfo/actions/runs/4657561879/jobs/8242275341

Any other notes?

I can reproduce this issue locally with the same versions of Ruby and RubyGems. I think this still counts as an issue because it makes Ruby 2.1 with bundler unusable through setup-ruby.

The RubyGems.org dependency API is being deprecated and removed, see: https://blog.rubygems.org/2023/02/22/dependency-api-deprecation.html, https://blog.rubygems.org/2023/04/07/dependency-api-deprecation-delayed.html, rubygems/rubygems.org#3477 and rubygems/rubygems.org#3692.

There's a brownout period today (April 10) where the dependencies API is returning 404 responses. This causes the gem install bundler -v ~> 1.0 command issued under 'Installing Bundler' to fail with Ruby 2.1 on Ubuntu:

Installing Bundler
  Bundler 2 requires Ruby 2.3+, using Bundler 1 on Ruby <= 2.2
  /opt/hostedtoolcache/Ruby/2.1.9/x64/bin/gem install bundler -v ~> 1.0
  ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
      bad response Not Found 404 (https://api.rubygems.org/api/v1/dependencies?gems=bundler)
  Took   5.04 seconds
Error: The process '/opt/hostedtoolcache/Ruby/2.1.9/x64/bin/gem' failed with exit code 1

and Windows:

Installing Bundler
  Bundler 2 requires Ruby 2.3+, using Bundler 1 on Ruby <= 2.2
  C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\Ruby\2.1.9\x64\bin\gem.bat install bundler -v "~> 1.0""
  ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
      bad response Not Found 404 (https://api.rubygems.org/api/v1/dependencies?gems=bundler)
  Took  12.19 seconds
Error: The process 'C:\hostedtoolcache\windows\Ruby\2.1.9\x64\bin\gem.bat' failed with exit code 1

Other Ruby versions appear to be unaffected.

[dentarg]

I think this can be worked around by using bundler: none (hard to test now that there's no brownout going on)

setup-ruby/action.yml

Lines 17 to 23 in 55283cc

	bundler:
	description: |
	The version of Bundler to install. Either 'Gemfile.lock' (the default), 'default', 'latest', 'none', or a version number (e.g., 1, 2, 2.1, 2.1.4).
	For 'Gemfile.lock', the version of the BUNDLED WITH section from the Gemfile.lock if it exists. If the file or section does not exist then the same as 'default'.
	For 'default', if the Ruby ships with Bundler 2.2+ as a default gem, that version is used, otherwise the same as 'latest'.
	For 'latest', the latest compatible Bundler version is installed (Bundler 2 on Ruby >= 2.3, Bundler 1 on Ruby < 2.3).
	For 'none', nothing is done.

setup-ruby/index.js

Lines 91 to 94 in 55283cc

	if (inputs['bundler'] !== 'none') {
	bundlerVersion = await common.measure('Installing Bundler', async () =>
	bundler.installBundler(inputs['bundler'], rubygemsInputSet, lockFile, platform, rubyPrefix, engine, version))
	}

---

2.2 failed too at https://github.com/ruby/setup-ruby/actions/runs/4658840660/jobs/8253314480?pr=494#step:3:24 (segfault though 😦) but 1.9 and 2.0 didn't fail, but looks to me they should have failed too 🤔 Is it the RubyGems version that matter? (Really old versions does not know about the "Dependency API"?)

[dentarg]

Seems related: rubygems/rubygems.org#3698

[philr]

The Test bundler: 1.x for old Ruby job from that run also failed with an error that's possibly related (again with 2.2):

Installing Bundler
  /opt/hostedtoolcache/Ruby/2.2.10/x64/bin/gem install bundler -v 1.16.6
  ERROR:  While executing gem ... (RuntimeError)
      Marshal.load reentered at marshal_load
  Took   0.60 seconds

That rubygems.org bug looks like exactly the same issue.

I've searched through the rubygems commit history looking for references to the dependencies API. It was first used by v2.0.0 (rubygems/rubygems@01c296c). The reference was finally removed in v3.2.3 (rubygems/rubygems@6b5d465). There are changes to how it used and how API errors are handled along the way.

The Ruby 2.1 build is using rubygems v2.2.5. That version uses the API in Source#dependency_resolver_set and APISet.

When installing bundler with Ruby 2.1 today (no brownout) Source#dependency_resolver_set gets called first and then an APISet gets created.

Source#dependency_resolver_set tests the API with a HEAD request and then creates an APISet only if successful. APISet then performs GET requests. If the HEAD request fails, it falls back to a path that doesn't involve the dependency API (IndexSet) and still manages to install bundler.

Perhaps the brownout was returning a 200 OK or 204 No Content for HEAD requests, whilst returning 404 Not Found for GET requests? If that's the case then the removal of the dependencies API might resolve this issue.

[dentarg]

Perhaps the brownout was returning a 200 OK or 204 No Content for HEAD requests, whilst returning 404 Not Found for GET requests?

Going by the tests added in rubygems/rubygems.org#3477 that looks like a possible explanation... @indirect or @segiddins probably knows more?

[segiddins]

Perhaps the brownout was returning a 200 OK or 204 No Content for HEAD requests, whilst returning 404 Not Found for GET requests?

... I think that's likely, let me check our CDN

[eregon]

Is there anything we can do about this in setup-ruby?
Maybe always using Bundler 1.12+ on Ruby 2.1, and maybe on 1.9 & 2.0 too?
I'd love to have a PR from RubyGems folks about this since I guess they know best the envisioned solution for those old Rubies.

[indirect]

We fixed an issue with the server responses during the brownout that we believe will correct the fallback behavior in older Bundler versions. We’ll find out for sure in the next brownout in a couple of days. 👍🏻

…

---- On Apr 15, 2023, at 6:49 AM, Benoit Daloze ***@***.***> wrote: Is there anything we can do about this in setup-ruby? Maybe always using Bundler 1.12+ on Ruby 2.1, and maybe on 1.9 & 2.0 too? I'd love to have a PR from RubyGems folks about this since I guess they know best the envisioned solution for those old Rubies. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>

[philr]

Thanks @indirect and @segiddins.

There's another brownout today (2023-04-17). I can confirm GET and HEAD requests are both now returning 404 Not Found:

$ curl -s -o /dev/null -w "%{http_code}\n" https://api.rubygems.org/api/v1/dependencies?gems=bundler
404
$ curl --head -s -o /dev/null -w "%{http_code}\n" https://api.rubygems.org/api/v1/dependencies?gems=bundler
404

Ruby 2.1 is now able to install bundler: https://github.com/tzinfo/tzinfo/actions/runs/4657567472/jobs/8382560503

The issue that @dentarg spotted with Ruby 2.2 (Marshal.load reentered at marshal_load) has also now been independently raised as #496.

[simi]

@philr can this be closed?

[philr]

Yes, I've not seen any issues with Ruby 2.1 since the fixes were made to make the API requests return a 404.