Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(parsers.json_v2): Reset parser before parsing (#13805) #14344

Merged
merged 3 commits into from
Nov 30, 2023

Conversation

agiilit-admin
Copy link
Contributor

Required for all PRs

resolves #13805

Running json_v2 parser leaves p.subPathResults dirty, affecting subsequent runs. This is a problem for input plugins reusing json_v2 parser instance instead of creating a new one.

Following the pattern of csv parser, added a Reset() method, which clears subPathResults. It is called in Init() and when starting Parse().

@telegraf-tiger
Copy link
Contributor

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

@telegraf-tiger telegraf-tiger bot added area/json json and json_v2 parser/serialiser related fix pr to fix corresponding bug plugin/parser 1. Request for new parser plugins 2. Issues/PRs that are related to parser plugins labels Nov 24, 2023
@agiilit-admin
Copy link
Contributor Author

!signed-cla

@Hipska Hipska added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Nov 27, 2023
@srebhan
Copy link
Member

srebhan commented Nov 27, 2023

@agiilit-admin how do you plan to use it? IMO the subpaths should be cleaned out on every call to Parse as we will always get a new document in that call!?!?

@srebhan srebhan self-assigned this Nov 27, 2023
@srebhan srebhan removed the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Nov 27, 2023
@agiilit-admin
Copy link
Contributor Author

@agiilit-admin how do you plan to use it? IMO the subpaths should be cleaned out on every call to Parse as we will always get a new document in that call!?!?

In CSV parser there is a separate ResetMode, and a conditional inside Parse():

// Reset the parser according to the specified mode
        if p.ResetMode == "always" {
                p.Reset()
        }

but since here I could not think of a use case where Reset() should not be called in Parse() (because, as you said, Parse() is always called with a new document), I left out the if and always call p.Reset() in Parse().

@srebhan
Copy link
Member

srebhan commented Nov 30, 2023

I think my comment was a bit misleading, sorry for that! What I mean is, why implement an exported Reset method if you can put the p.subPathResults = nil statement directly as a first line in Parse()? Maybe add a small comment that this is required to reset the state of the parser or something...

@srebhan
Copy link
Member

srebhan commented Nov 30, 2023

Oh and @agiilit-admin you need to sign the CLA as otherwise we cannot take your PR...

@agiilit-admin
Copy link
Contributor Author

Oh and @agiilit-admin you need to sign the CLA as otherwise we cannot take your PR...

I thought I did and posted the !signed-cla comment here as requested. Is there still something I should do?

@agiilit-admin
Copy link
Contributor Author

I think my comment was a bit misleading, sorry for that! What I mean is, why implement an exported Reset method if you can put the p.subPathResults = nil statement directly as a first line in Parse()? Maybe add a small comment that this is required to reset the state of the parser or something...

Probably it is just a learned predisposition: I don't think Parse() internal variables such as p.subPathResults should be visible via instance variables at all if Parse() is meant to be reused, but if they are, then there should be an instance method to revert the parser back to the state it was just after coming out of the constructor.

Anyway, comments added.

@srebhan
Copy link
Member

srebhan commented Nov 30, 2023

While I do agree with your thinking, in the current state this is useless to export Reset() as no plugin can use it without type-asserting the parser. Furthermore, it is fundamentally wrong to require the user of the parser to know about the internal workings! You always need to reset this state after parsing, there is no use-case to not reset subPathResults (contrary to CSV), so just do it and do not require others to care.

Therefore, please do not export the Reset() function! And in turn this then is a one-function-call wrapping function which should go away.

@telegraf-tiger
Copy link
Contributor

Copy link
Member

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding and fixing this issue @agiilit-admin! Very much appreciated!

@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Nov 30, 2023
@srebhan srebhan assigned powersj and unassigned srebhan Nov 30, 2023
@powersj powersj merged commit e17ee6d into influxdata:master Nov 30, 2023
22 of 23 checks passed
@github-actions github-actions bot added this to the v1.29.0 milestone Nov 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/json json and json_v2 parser/serialiser related fix pr to fix corresponding bug plugin/parser 1. Request for new parser plugins 2. Issues/PRs that are related to parser plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Weird behavior of json_v2 parser with multiple input lines and lots of decimal places
4 participants