Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] CSV parsing large number values overflow #126

Closed
revans2 opened this issue Jun 8, 2020 · 0 comments · Fixed by #4790
Closed

[BUG] CSV parsing large number values overflow #126

revans2 opened this issue Jun 8, 2020 · 0 comments · Fixed by #4790
Assignees
@andygrove
Labels
bug Something isn't working P2 Not required for release SQL part of the SQL/Dataframe plugin
Milestone
Feb 14 - Feb 25

Comments

@revans2
Copy link
Collaborator

revans2 commented Jun 8, 2020 •
edited
Loading

Describe the bug
When parsing large float number values in CSV they overflow to 'inf' or '-inf' instead of returning null as the CPU does.

For int numbers it is the same but the value literally overflows instead of becoming Inf or -Inf.

Steps/Code to reproduce bug
have a CSV file with a value too large to fit in the type being used.
@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify SQL part of the SQL/Dataframe plugin labels Jun 8, 2020
@sameerz sameerz added P2 Not required for release and removed ? - Needs Triage Need team to review and classify labels Aug 18, 2020
@revans2 revans2 mentioned this issue Apr 1, 2021
Open
38 tasks
@andygrove andygrove mentioned this issue Jan 31, 2022
Merged
7 tasks
@andygrove andygrove mentioned this issue Feb 15, 2022
Merged
4 tasks
@andygrove andygrove self-assigned this Feb 15, 2022
@andygrove andygrove added this to the Feb 14 - Feb 25 milestone Feb 15, 2022
tgravescs pushed a commit to tgravescs/spark-rapids that referenced this issue Nov 30, 2023
@nvauto
Merge pull request NVIDIA#126 from NVIDIA/bot-auto-merge-branch-22.04
4c07d07 
[auto-merge] bot-auto-merge-branch-22.04 to branch-22.06 [skip ci] [bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@andygrove andygrove

Labels
bug Something isn't working P2 Not required for release SQL part of the SQL/Dataframe plugin
Projects
None yet
Milestone
Feb 14 - Feb 25
Development

Successfully merging a pull request may close this issue.

Improve JSON and CSV parsing of integer values andygrove/spark-rapids
3 participants
@andygrove @revans2 @sameerz