Incompatibility with Ruby 2.2.0

[dougo]

I upgraded to Ruby 2.2.0, but now I get errors when I install the JSON gem:

bash-3.2$ gem install json
Fetching: json-1.8.1.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing json:
    ERROR: Failed to build gem native extension.

    /Users/dorleans/.rbenv/versions/2.2.0/bin/ruby -r ./siteconf20141225-66244-1qaxb2p.rb extconf.rb
creating Makefile

make "DESTDIR=" clean

make "DESTDIR="
compiling generator.c
In file included from generator.c:1:
./../fbuffer/fbuffer.h:175:47: error: too few arguments provided to function-like macro invocation
    VALUE result = rb_str_new(FBUFFER_PAIR(fb));
                                              ^
/Users/dorleans/.rbenv/versions/2.2.0/include/ruby-2.2.0/ruby/intern.h:793:9: note: macro 'rb_str_new' defined here
#define rb_str_new(str, len) __extension__ (    \
        ^
In file included from generator.c:1:
./../fbuffer/fbuffer.h:175:11: warning: incompatible pointer to integer conversion initializing 'VALUE' (aka 'unsigned long') with an expression of type 'VALUE (const char *, long)' [-Wint-conversion]
    VALUE result = rb_str_new(FBUFFER_PAIR(fb));
          ^        ~~~~~~~~~~
1 warning and 1 error generated.
make: *** [generator.o] Error 1

make failed, exit code 2

Gem files will remain installed in /Users/dorleans/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/gems/json-1.8.1 for inspection.
Results logged to /Users/dorleans/.rbenv/versions/2.2.0/lib/ruby/gems/2.2.0/extensions/x86_64-darwin-13/2.2.0-static/json-1.8.1/gem_make.out
bash-3.2$ ruby --version
ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-darwin13]
bash-3.2$ gem --version
2.4.5

I don't see anything about rb_str_new in the Ruby 2.2.0 change log, so maybe I just have something wrong with my environment? Any tips would be appreciated.

[kgrz]

Not an expert in C and can't figure out why the macro expansion is not working properly. I was able to push a small fix by avoiding the use of FBUFFER_PAIR and using FBUFFER_PTR and FBUFFER_LEN as arguments to the rb_str_new function and get the extension to compile: kgrz@e49894d

[Mehonoshin]

Hi! I'm having the same issue

[DomT4]

Yup, same thing here:

/usr/local/opt/ruby/bin/ruby -r ./siteconf20141226-34853-1i5xphz.rb extconf.rb
creating Makefile

make "DESTDIR=" clean

make "DESTDIR="
compiling generator.c
In file included from generator.c:1:
./../fbuffer/fbuffer.h:175:47: error: too few arguments provided to function-like macro invocation
    VALUE result = rb_str_new(FBUFFER_PAIR(fb));
                                              ^
/usr/local/Cellar/ruby/2.2.0/include/ruby-2.2.0/ruby/intern.h:793:9: note: macro 'rb_str_new' defined here
#define rb_str_new(str, len) __extension__ (    \
        ^
In file included from generator.c:1:
./../fbuffer/fbuffer.h:175:11: warning: incompatible pointer to integer conversion initializing 'VALUE' (aka 'unsigned long') with an expression of type 'VALUE (const char *, long)' [-Wint-conversion]
    VALUE result = rb_str_new(FBUFFER_PAIR(fb));
          ^        ~~~~~~~~~~
1 warning and 1 error generated.
make: *** [generator.o] Error 1

make failed, exit code 2

[dougo]

It looks like this is the commit that changed rb_str_new from a function declaration to a macro:
ruby/ruby@abac848#diff-d25389b5dda13be5e80e89f60e1cc716R779
If I remove the macro from my local copy of the Ruby 2.2.0 include, then it compiles successfully.

[dougo]

Here is some discussion about how the C preprocessor deals with scanning function-like macro arguments: https://gcc.gnu.org/onlinedocs/cpp/Argument-Prescan.html#Argument-Prescan
Although it says "Macro arguments are completely macro-expanded before they are substituted into a macro body", apparently the scan for commas happens before the argument is macro-expanded, which is where it raises the error. So aside from wrapping the macro in an actual C function (which would require declaring the two arguments, which would defeat the purpose of using the FBUFFER_PAIR macro), I think @kgrz's fix of not using FBUFFER_PAIR is the only way to do it.

[hsbt]

We(ruby-core) fixed our bundled json gems like forked. see diff https://github.com/zzak/json/compare/ruby-2.2

I think we should backport our patchs into this repository. Please wait until merge it and bump version 🙇

[hedgehog]

Any ETA on when a PR might be coming? e.g. days, weeks, undefined

[zzak]

I'm still trying to work out a patch to fix 1.8.7, since that is the only blocker I imagine it should be soon. Unless we just drop 1.8.7 support all-together.

[hedgehog]

I'm not that familiar with this gem's code base.... would it not be easier to add something to the backports gem (marcandre/backports) and have that 1.8.7 dependency rather than drop support?

[zzak]

The problem is the capi has changed, and this struct isnt available in 1.8.7: https://github.com/zzak/json/blob/ruby-2.2/ext/json/ext/generator/generator.c#L518

[hedgehog]

Ack. The backport may have to be a pure ruby implementation. Rather than pull the pin (either way) maybe a non-1.8.7 gem can be released as a workaround - this will also give time to assess the level of pain caused in the 1.8.7 community.

My 2c on 1.x.y:
I think we should be cautious about dropping 1.8.7 support for such an important gem, given that it is 1.x.y series currently supported.
1.x.y is only one major version away from where we are now, 2.m.n.
While ruby evolves quickly, it is doing so because it works really well which means people are starting to look at it for serious use cases, i.e production code that doesn't change rapidly, but where the hardware might - people buy time by leaving code (expensive to develop, good code that is) unchanged and upgrading the cheap parts of the system (hardware).

[san983]

Same issue here

[hedgehog]

Given ruby 1.8.7 has had its EOL announced. I think the most pragmatic approach to take is to replace it with rbx 1.3 which I've just seen is continuing 1.8.7 compatibility - but fixing bugs.

See here: http://rubini.us/2014/12/08/rubinius-1-3/

[24X7]

When is a real fix available?

[anatol]

I have the same issue. Browsing internet I found this file that looks interesting http://fossies.org/linux/misc/ruby-2.2.0.tar.gz/ruby-2.2.0/ext/json/fbuffer/fbuffer.h

They replaced the problematic line with

VALUE result = rb_str_new(FBUFFER_PTR(fb), FBUFFER_LEN(fb));

[anatol]

Indeed, after applying following patch the tests pass on my Linux Arch with ruby 2.2.0 installed

diff --git a/ext/json/ext/fbuffer/fbuffer.h b/ext/json/ext/fbuffer/fbuffer.h
index af74187..9524fb1 100644
--- a/ext/json/ext/fbuffer/fbuffer.h
+++ b/ext/json/ext/fbuffer/fbuffer.h
@@ -172,7 +172,7 @@ static FBuffer *fbuffer_dup(FBuffer *fb)

 static VALUE fbuffer_to_s(FBuffer *fb)
 {
-    VALUE result = rb_str_new(FBUFFER_PAIR(fb));
+    VALUE result = rb_str_new(FBUFFER_PTR(fb), FBUFFER_LEN(fb));
     fbuffer_free(fb);
     FORCE_UTF8(result);
     return result;

[DouglasAllen]

It's not that I need json-1.7.7.gem
But gem install devcenter requires it.
Let's get them updated for Ruby 2.2.0
Maybe Heroku can try bson and get things working faster yet or will this cost us? ;-)
mongodb requires it so json could perhaps gain some performance with it also IMHO.
Anyway now that I'm sidetracked and up on a spur I can leave this parked line and and go find ragel and G2. Thanks Mario Bro. https://aur.archlinux.org/packages/?O=0&C=10&SeB=nd&K=ruby-json+1.7.7&outdated=&SB=n&SO=a&PP=50&do_Search=Go

[MaxGabriel]

I think this can be closed with the 1.8.2 release. I just tested and was able to install json without any problems on 2.2.0, where previously that machine ran into the too few arguments error.

[flori]

Thank you I am closing this issue.

[DomT4]

#229 (comment) seems to be back with Ruby 2.4.0 on macOS Sierra 10.12.4, Bundler 1.14.5. If I switch back to using Ruby 2.0.0p648 (which is the system default on macOS 10.12.4) have no such issue. Maybe I'm barking up the wrong tree since nobody else seems to have added a new comment here, but 🤷‍♂️.

[umarsheikh]

#229 issue is back with ruby 2.4.

compiling generator.c
In file included from generator.c:1:0:
../fbuffer/fbuffer.h: In function ‘fbuffer_to_s’:
../fbuffer/fbuffer.h:175:47: error: macro "rb_str_new" requires 2 arguments, but only 1 given
../fbuffer/fbuffer.h:175:20: warning: initialization makes integer from pointer without a cast [enabled by default]
generator.c: In function ‘generate_json’:
generator.c:832:25: error: ‘rb_cFixnum’ undeclared (first use in this function)
generator.c:832:25: note: each undeclared identifier is reported only once for each function it appears in
generator.c:834:25: error: ‘rb_cBignum’ undeclared (first use in this function)
generator.c: At top level:

An error occurred while installing json (1.7.7), and Bundler cannot continue.

[h0jeZvgoxFepBQ2C]

Have the same problem with ruby 2.3.4 on OSX Sierra.

[danman01]

Seems JSON version 1.8.3 does the trick http://fuzzyblog.io/blog/ruby/2016/10/11/what-to-do-when-bundle-install-fails-with-json-1-8-1.html