Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] casting to string and list, and concat can cause overflow issues #6636

Closed
revans2 opened this issue Sep 27, 2022 · 2 comments
Closed
Assignees
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf P0 Must have for release reliability Features to improve reliability or bugs that severly impact the reliability of the plugin

Comments

@revans2
Copy link
Collaborator

revans2 commented Sep 27, 2022

Describe the bug
This is similar to #6598.

There are a number of places in cast where we don't think about the output size, and when going to a string or a list these sizes can be larger than the input was. This can result in overflows, and in the worst cases the algorithms can walk off the end of memory and corrupt things.

Generally I would like to see us go through all cast to string code and all concat code that deals with string, binary, arrays, or maps and test that we get appropriate errors without touching memory we should not. If we do run into error, which I think there will be many, then we need to either fix them of file follow on issues to fix them.

@revans2 revans2 added bug Something isn't working ? - Needs Triage Need team to review and classify P0 Must have for release reliability Features to improve reliability or bugs that severly impact the reliability of the plugin and removed ? - Needs Triage Need team to review and classify labels Sep 27, 2022
@ttnghia ttnghia changed the title [BUG] casting to sting and list, and concat can cause overflow issues [BUG] casting to string and list, and concat can cause overflow issues Sep 28, 2022
@revans2 revans2 added the cudf_dependency An issue or PR with this label depends on a new feature in cudf label Nov 7, 2022
@revans2
Copy link
Collaborator Author

revans2 commented Nov 7, 2022

I filed rapidsai/cudf#12087 for the string column issues. I will keep investigating others, especially for lists.

@revans2
Copy link
Collaborator Author

revans2 commented Dec 5, 2022

A number of changes have gone into CUDF to avoid memory corruption issues, but the longterm solution will be a part of #7257

@revans2 revans2 closed this as completed Dec 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cudf_dependency An issue or PR with this label depends on a new feature in cudf P0 Must have for release reliability Features to improve reliability or bugs that severly impact the reliability of the plugin
Projects
None yet
Development

No branches or pull requests

1 participant