-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bundler 2.2.3 generated Gemfile.lock on a mac cannot be used on linux out of the box #4269
Comments
Yes, this is correct. Sorry, I've been going back and forth here trying to get a correct (and as fast as previously) resolution, while keeping this thing usable, but yeah right now if you want to develop under a platform and deploy to a different one (linux in this example), you'd need to run I could try restoring the RUBY platform as a fallback but you wouldn't get any platform specific gems unless you explicitly add the platform you want to deploy to. |
Thanks for the ultra fast response! WorkaroundI think in the short term I'm going to try to upgrade Heroku customers to Bundler 2.2.3 which provides a warning for this out of the box.
Then customers will be unblocked in the short term (provided I don't have to rollback due to a yet-to-be-found regression). Future musingsAs a feature of bundler, I want to be able to either have bundler default to linux as a secondary platform or have some way of specifying it globally. This change in behavior with 2.2.3 means a lot of existing tutorials for deploying Ruby apps now no longer work. Ideally, I would like to get them back to being valid as many are not maintained and will never be updated. For those that can be updated, it would be ideal to not have to add an extra step. I'm happy to help brainstorm here or provide support if I can. |
Right, I think I can probably restore the old behaviour as a last resort without losing any of the good stuff of locking the specific platform in the lockfile. I'll work on it tomorrow and keep you posted. |
Thanks! We have code on the Heroku buildpack that deletes the Gemfile.lock if we see it's locked to the windows platform. It would be exciting to get rid of that functionality and get deterministic deploys for devs using windows. I appreciate you working on this area. |
Hi @schneems! So, I've been sleeping on this issue, and I think what we have now is quite good to be honest. Let me try to explain why we did this, and why I believe it's a good thing. ContextIn previous bundler versions, bundler didn't consider platforms for resolution at all. What bundler would previously do is to resolve dependencies without considering platforms, and then at installation time, pick up platform specific variants with the same version as the resolved version if they exist. That approach had several problems:
Resolving for the specific running platform and recording the exact resolution in the lockfile fixes the above issues. Current situation
Please let me know what you think. |
I think that's reasonable to keep this in there. As you mentioned it's more specific and safer. I am interested though in polishing the "out of the box" experience. I'm thinking to how bundler was added to Ruby by default so the annoyance of hitting the In my ideal world, a project with a Spitballing some ideas:
Do any of those sound remotely appealing or workable? What I'm striving for is that if a developer does every step to create an app and deploy it "by the book" that it works "out of the box". Like this sentiment: https://twitter.com/searls/status/1344277289840873472. If not I would be interested to hear more about your considerations. Also if you prefer we could move this over to slack or a zoom/hangout chat if you want higher fidelity. I'm happy to bounce ideas around if you think it could be helpful. |
Also some feedback from a customer who develops on Linux already. I asked if this fixed their issue:
They responded:
|
@schneems I'll respond to your comments as soon as I can, but if you can report that issue separately in the mean time, I appreciate it. |
Can do. Thanks! |
Hi @schneems. I'm not opposed to adding some configuration so that users can specify the set of platforms they want to support. I had the idea of adding a We currently have the issue though that the resolver can handle platforms correctly, but at the same time, resolving for multiple platforms by default is sometimes very inefficient. That's why we currently resolve only for the current platform and if the user wants to add more platforms, for example, to get ready for deploying under a different platform that the one they are developing on, then she can add them manually. I'm currently working on fixing these issues so that I can at least restore resolving for the Until I can fix these issues, would adding the deployment platform on the fly to the lockfile if not present work for you? Basically, run |
That's do-able. We've got some deploy freezes right now, but I can work something up. My concern is that a temporary workaround ends up being permanent. Like the same way we have to strip out I think adding this platform on our end defeats the security and resolution-correctness purpose you mentioned before as they can get different versions in prod than at development time. I would perhaps be easier to just not enable Options on my endHere's some brainstorming about options I can take moving forwards with the consequences (good and bad) listed below each. For reference deploying, CodeTriage takes 2m0.919s which also includes running a release phase Do nothing (zeroeth option)
Turn off "deployment mode" on Heroku
Auto-add x86_64-linux before bundle install on Heroku
Upgrade bundler to 2.2.5 for all users on Heroku
Detect
|
Reading the comments, I am running into the same issue even after running the bundler lock command in several variations (/w ruby, /wo ruby) doesnt matter, still get the same error: remote: -----> Building on the Heroku-20 stack gemfile.lock: running Not sure what I am missing here, any thoughts? |
We had this symptom and it was actually caused by our Gemfile lock having
CLRF line endings which wasn't being parsed by the build back correctly.
However i believe that has now been fixed by the build pack maintainer
…On Sat, 6 Mar 2021, 01:30 Tom Dulin, ***@***.***> wrote:
Reading the comments, I am running into the same issue even after running
the bundler lock command in several variations (/w ruby, /wo ruby) doesnt
matter, still get the same error:
remote: -----> Building on the Heroku-20 stack
remote: -----> Ruby app detected
remote: -----> Installing bundler 2.2.11
remote: -----> Removing BUNDLED WITH version in the Gemfile.lock
remote: -----> Compiling Ruby/Rails
remote: -----> Using Ruby version: ruby-3.0.0
remote: -----> Installing dependencies using bundler 2.2.11
remote: Running: BUNDLE_WITHOUT='development:test'
BUNDLE_PATH=vendor/bundle BUNDLE_BIN=vendor/bundle/bin BUNDLE_DEPLOYMENT=1
bundle install -j4
remote: Your bundle only supports platforms ["x86_64-darwin-20"] but your
local platform
remote: is x86_64-linux. Add the current platform to the lockfile with bundle
lock remote: --add-platform x86_64-linux and try again.
remote: Bundler Output: Your bundle only supports platforms
["x86_64-darwin-20"] but your local platform
remote: is x86_64-linux. Add the current platform to the lockfile with bundle
lock remote: --add-platform x86_64-linux and try again.
remote:
remote: !
remote: ! Failed to install gems via Bundler.
remote: !
remote: ! Push rejected, failed to compile Ruby app.
gemfile.lock:
...
PLATFORMS
x86_64-linux
...
running
ruby 2.6
rails 3.0.0
bundler 2.2.11
Not sure what I am missing here, any thoughts?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#4269 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAJB2CV73TCRNAMHEWHIQDTCGARVANCNFSM4V52ZOPA>
.
|
See rubygems/rubygems#4269 for why we have to do this
This fixed for me on macOS Catalina |
Like so many others, we dev on Mac and deploy to Linux. I've been running Are there plans to add desired platforms to the Gemfile so it's automatic? |
We got devs on mac and linux and deploy to heroku. Our solution has been to |
Think I'm running into a similar issue. It started 3 days ago, after updating bundle, which moved my project from
I'm on MacOS Big Sur 11.2.2, and Heroku still installs then removes all my gems:
Any ideas would be appreciated, otherwise I'll just downgrade using these instructions. |
Upgraded from rails 6.0.3 to 'rails', '~> 6.1', '>= 6.1.3.2' The downgrade instructions gregschwartz mentioned above didn't work for me. |
possible solution from rubygems/rubygems#4269
My GemFile.lock had only
Running |
its works for me. great! |
…ctions bundle install rubygems/rubygems#4269
Just want to say here that this whole thing was pretty bad. Seems like adding this level of granularity of the platform to the Gemfile.lock was just a bad idea. Dependencies became an order of magnitude more complicated unnecessarily and only cause problems for us and didn't solve anything. The issue we are having is that the dependency tree for grpc seems to be broken for at least one platform - alpine. And so we have to build with something else that happens to fill in the right dependency, which, I think at this time might be the m1 mac. This is kinda a silly thing that just seems to come out of this gross overcomplication. |
This seems to be required if developing on a different platform to the one being deployed to. See: rubygems/rubygems#4269 (comment) Added by running: `bundle lock --add-platform x86_64-linux`
Added platform check - the way bundler works with platforms has changed over the years. See how it [works](https://prathamesh.tech/2021/04/18/bundler-2-2-3-and-deployment-of-ruby-apps/) [now](https://www.moncefbelyamani.com/understanding-the-gemfile-lock-file/#platforms) x how [it used to work](rubygems/rubygems#4269 (comment)). TLDR: One Gemfile.lock can contain dependencies for multiple platforms and the way they will be resolved at deploy/install time changed with bundler version 2.2.0 and 2.2.3. I decided to implement it the simplest way - support only Gemfile.lock with only one platform specified - ruby. This way, we will be able to support all bundler versions and it will be simpler to parse the resulting file (scancode-toolkit isn't able to parse files with multiple platform versions of one package). Added check for Git reference (40 character hash). For GEM dependencies, semantic versioning is recommended, [but not enforced](https://guides.rubygems.org/patterns/), therefore the only thing that's validated is if the version isn't None (similarly for PATH). Signed-off-by: Milan Tichavský <[email protected]>
#5700 should fix this issue, feel free to give it a try! |
Maybe I'm missing something completely here but... Why are the proposed fixes centered around changing the Perhaps another option to consider is that I get it that people want things to be "automatic" but for anyone that needs to support multiple OS and architecture in projects that are not ruby (as I do) this is frequently a build configuration matter that simply becomes a different target in a Makefile or the addition of a step in a CI config.
Right now I support arm64 and x86_64 targets for both macOS and Linux across multiple projects. In those projects, a simple |
Describe the problem as clearly as you can
Here's an example repo: https://github.com/schneems/bundler_223_fails_deploy. It has one dependency
rake
and the Gemfile.lock was generated by runningbundle install
on my mac. When I try to use this Gemfile/Gemfile.lock combo it fails:Then
What were you expecting to happen?
I expect that a Gemfile.lock produced on mac can also work on linux
What actually happened?
The gem appears to install, but then it cannot be loaded.
Extra context
Either adding
ruby
to thePLATFORMS
section orx86_64-linux
fixes the issue, but this is not immediately clear.If not included with the output of your command, run
bundle env
and paste the output belowbundle env (on my mac)
## EnvironmentBundler Build Metadata
Bundler settings
Gemfile
Gemfile
Gemfile.lock
bundle env (on docker)
## EnvironmentBundler Build Metadata
Bundler settings
Gemfile
Gemfile
Gemfile.lock
The text was updated successfully, but these errors were encountered: