-
Notifications
You must be signed in to change notification settings - Fork 15.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fill out JRuby support #7923
Fill out JRuby support #7923
Conversation
Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). 📝 Please visit https://cla.developers.google.com/ to sign. Once you've signed (or fixed any issues), please reply here with What to do if you already signed the CLAIndividual signers
Corporate signers
ℹ️ Googlers: Go here for more info. |
I'm working on getting my company to sign the CLA, hopefully will be able to get that done today. The codespell failure doesn't look to be related to anything I changed. |
Sorry about the delay on the CLA we're still working on taking care of it. |
@googlebot I signed it! |
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
CLAs look good, thanks! ℹ️ Googlers: Go here for more info. |
Is there anything I can do to keep this moving along? |
Pinging every couple of days as per @acozzette 's comment above :) Would love to see this get moved through. |
Just sending in a ping to see if this has gotten on anybody's radar yet. |
@rdubya Sorry for the delay again. We talked about this at our team meeting today. The thing we've been hesitating about is that the JRuby protobuf implementation has not been very actively maintained the past few years, and no one one currently on the team is very familiar with it. I believe it was originally written by JRuby enthusiasts from outside the protobuf team and never really worked on much by our team itself. We can try to review this PR but it's just tricky because no one on the team is knowledgeable enough about the JRuby code to really do a thorough review. We were thinking it would be ideal if we could perhaps split off the JRuby implementation into its own separate Git repo that could be owned and maintained by members of the community such as yourself. What do you think? |
Hey @acozzette, Thanks for the feedback. I think splitting it off would probably cause some issues since the JRuby version needs to be pushed with the same gem name so that dependencies can work correctly. How would you see that interaction going if it were a separate repo? @enebo or @headius : any thoughts on maintaining a JRuby specific repo? We could also potentially move more of the code into ruby so there are less pieces that are written in c or java, but I don't know what other implications that might have. Not having JRuby support locks JRuby users out of several of Google's gems because it is a dependency for them. The gRPC gem is unfortunately another hinderance there but this is step one of getting full support. I got permission to do this as a way to hopefully kickstart some better support going forward, but I don't think my company sees enough value in it to fund me maintaining a whole project. Our workaround thus far has been to implement the REST api calls we need since most of our use cases have only needed a couple of api calls. |
@rdubya That is a good point about the JRuby gem needing to have the same name, and I don't know of any good solution to that. |
There are a couple test failures but they should be easy to fix just by adding |
from = get_msg_proto2 | ||
data = A::B::Proto2::TestMessage.encode(from) | ||
to = A::B::Proto2::TestMessage.decode(data) | ||
# This doesn't work for proto2 on JRuby because there is a nested required message. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that the plain Ruby implementation isn't validating required fields during parsing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I didn't dig into the c-ruby side too much on it but I think it tracked down to it not checking things in the same order. There is a required field that is the same type as the message that contains it and, iirc, it turns into an infinite loop.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That makes sense, it's just that the fact that this test was passing for C Ruby makes me think it is not validating required fields, which is a separate problem that we should probably fix at some point. This is a very weird test message because it's impossible to create a valid instance of it, given the recursive required field.
@rdubya Sorry about the delay and thanks for the pull request! |
Awesome, thank you for including it! |
Thanks much, @rdubya - I've been tracking #1594 for four years! Looking forward to seeing it finally squashed. @acozzette / @haberman - when can we expect to see a release that includes the JRuby gem again? It's been a while... CC: @blowmage |
@acozzette / @haberman / @TeBoring - Bumping my previous comment - where are we on a release that includes this change (and makes sure |
@JasonLunn Sorry for taking so long to get back to you, but yes I would like to set things up so that we release Java variants of the gem. We are planning to do a 3.15 release soon, so I will work on this next week and try to make sure it gets done for 3.15. |
I tried installing google-protobuf 3.15 still getting an error for JRuby 9.2.11.1 |
@haberman There seems to be some progress on this, however I'm also unable to install this gem. Any advice? gem install google-protobuf
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: no-rdoc
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: --no-rdoc
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: no-ri
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: --no-ri
Building native extensions. This could take a while...
Exception `Gem::InstallError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:99 - make failed, exit code 2
Exception `Gem::InstallError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:99 - make failed, exit code 2
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-77730-1ky5kuq.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-77730-1ky5kuq.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-77730-1ky5kuq.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
ERROR: Error installing google-protobuf:
ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-77730-1ky5kuq.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::SystemExitException' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/user_interaction.rb:375 - Exiting RubyGems with exit_code 1
Exception `Gem::SystemExitException' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/user_interaction.rb:375 - Exiting RubyGems with exit_code 1
|
@rdubya had similar results, thanks for getting back to me chanst@C02D963ZMD6T LEM % gem install google-protobuf --platform java
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: no-rdoc
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: --no-rdoc
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: no-ri
Exception `OptionParser::InvalidOption' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/optparse.rb:1739 - invalid option: --no-ri
Building native extensions. This could take a while...
Exception `Gem::InstallError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:99 - make failed, exit code 2
Exception `Gem::InstallError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:99 - make failed, exit code 2
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-61916-kjy1wf.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-61916-kjy1wf.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::Ext::BuildError' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/ext/builder.rb:153 - ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-61916-kjy1wf.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
ERROR: Error installing google-protobuf:
ERROR: Failed to build gem native extension.
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
/Users/chanst/.rvm/rubies/jruby-9.2.13.0/bin/jruby -I /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib -r ./siteconf20210422-61916-kjy1wf.rb extconf.rb
creating Makefile
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR=" clean
current directory: /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8/ext/google/protobuf_c
make "DESTDIR="
make: *** No rule to make target `/Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/include/ruby/ruby.h', needed by `protobuf.o'. Stop.
make failed, exit code 2
Gem files will remain installed in /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/gems/google-protobuf-3.15.8 for inspection.
Results logged to /Users/chanst/.rvm/gems/jruby-9.2.13.0@lem/extensions/universal-java-11/2.5.0/google-protobuf-3.15.8/gem_make.out
Exception `Gem::SystemExitException' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/user_interaction.rb:375 - Exiting RubyGems with exit_code 1
Exception `Gem::SystemExitException' at /Users/chanst/.rvm/rubies/jruby-9.2.13.0/lib/ruby/stdlib/rubygems/user_interaction.rb:375 - Exiting RubyGems with exit_code 1
|
@haberman I just checked the rubygems repo and it doesn't look like the jruby version of the gem got built/pushed. |
Sorry but I haven't yet had a chance to get this integrated into our release process, though it's still on my todo list. |
Is there like a link I can use to track the progress? I'm not blocked by the way. I found an alternative for my needs. **edited. The alternative approach didn't work. Waiting for the gem to be available :) |
@acozzette three months later... any updates? |
@rdubya - I am looking into a PR to address the build and release issue. However, when I pull the repo, despite being able to successfully build and install the gem (using the instructions as written in https://github.com/protocolbuffers/protobuf/tree/master/ruby) under Debian + JDK11 + rvm 1.29.12 + jruby-9.2.9.0, |
Since this is the rake target used by kokoro, this will close the gem release gap for JRuby Should help protocolbuffers#7923 by making sure JRuby is built and released as part of the normal cadence.
Since this is the rake target used by kokoro, this will close the gem release gap for JRuby Should help protocolbuffers#7923 by making sure JRuby is built and released as part of the normal cadence.
Shout out to @JasonLunn for moving this along. Looks like there has been some more activity on this in recent days, and we'll have the first working JRuby version published on RubyGems since 2017 soon! 🙌 |
The wait is over! My sincerest thanks to @rdubya @haberman @acozzette @perezd for helping me to make this possible. |
This gets the Ruby gem to be able to be used on JRuby. Almost all Ruby tests are passing with a couple minor changes. There are a significant number of compatibility tests that are failing, but I figured this would at least get something in place that could be expanded upon by others in the community.
This is related to the following issues: