Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure strict mode is used for Oj parser. #1147

Merged
merged 1 commit into from
Aug 8, 2016

Conversation

edevil
Copy link
Contributor

@edevil edevil commented Aug 5, 2016

If we don't use strict mode strings that start with ":" are converted to symbols:

irb(main):002:0> Oj.load('{"log":"::1 - - [04/Aug/2016:16:15:13 +0000] \"OPTIONS * HTTP/1.0\" 200 126 \"-\" \"Apache/2.4.10 (Debian) PHP/5.6.24 (internal dummy connection)\"\n","stream":"stdout","time":"2016-08-04T16:15:13.726932176Z"}')
=> {"log"=>:":1 - - [04/Aug/2016:16:15:13 +0000] \"OPTIONS * HTTP/1.0\" 200 126 \"-\" \"Apache/2.4.10 (Debian) PHP/5.6.24 (internal dummy connection)\"\n", "stream"=>"stdout", "time"=>"2016-08-04T16:15:13.726932176Z"}

Using strict mode this does not happen:

irb(main):003:0> Oj.default_options = {bigdecimal_load: :float, :mode => :strict}
=> {:bigdecimal_load=>:float, :mode=>:strict}
irb(main):004:0> Oj.load('{"log":"::1 - - [04/Aug/2016:16:15:13 +0000] \"OPTIONS * HTTP/1.0\" 200 126 \"-\" \"Apache/2.4.10 (Debian) PHP/5.6.24 (internal dummy connection)\"\n","stream":"stdout","time":"2016-08-04T16:15:13.726932176Z"}')
=> {"log"=>"::1 - - [04/Aug/2016:16:15:13 +0000] \"OPTIONS * HTTP/1.0\" 200 126 \"-\" \"Apache/2.4.10 (Debian) PHP/5.6.24 (internal dummy connection)\"\n", "stream"=>"stdout", "time"=>"2016-08-04T16:15:13.726932176Z"}

Obtaining symbols instead of strings causes havoc for some downstream plugins.

@repeatedly
Copy link
Member

repeatedly commented Aug 5, 2016

Wow, I first knew this behaviour.
Good catch!

@repeatedly repeatedly self-assigned this Aug 5, 2016
@repeatedly
Copy link
Member

Tests are passed so exisiting json parsing works.

@tagomoris
Copy link
Member

I found that this change changes the process global default behavior, including behavior of formatter_json.
Now formatter_json raises exception when it dumps values with symbols, with message "Failed to dump Symbol Object to JSON in strict mode.".

@tagomoris
Copy link
Member

I re-checked the modes of Oj, and found that:

And, Oj only has process global default_options. Boo.
I think we must use compat mode, or remove Oj from entire Fluentd.

@tagomoris
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants