Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Create Java string scalar from byte array #8233

Closed
jlowe opened this issue May 13, 2021 · 2 comments
Closed

[FEA] Create Java string scalar from byte array #8233

jlowe opened this issue May 13, 2021 · 2 comments
Labels
feature request New feature or request Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS

Comments

@jlowe
Copy link
Member

jlowe commented May 13, 2021

Is your feature request related to a problem? Please describe.
In Spark, strings are tracked with a UTF8String which is essentially a wrapper around a byte array. The Java Scalar only allows creating string scalars by passing a Java String, and creating that from a byte array requires copying the bytes to a String. Internally Scalar.fromString copies the String data to a byte array so it can be copied to the GPU.
Therefore when converting a Spark UTF8String into a Scalar there's two extra copies that are created that should be avoidable.

Describe the solution you'd like
Scalar should have a fromBytesAsString method that can take a byte array and convert it into a GPU string scalar.

@jlowe jlowe added feature request New feature or request Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS labels May 13, 2021
@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@jlowe
Copy link
Member Author

jlowe commented Nov 15, 2021

Fixed by #8294

@jlowe jlowe closed this as completed Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Development

No branches or pull requests

1 participant