Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Move the implementation of building an empty nested column to the native #8178

Open
firestarman opened this issue May 7, 2021 · 3 comments
Labels
feature request New feature or request Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS

Comments

@firestarman
Copy link
Contributor

Currently building an empty nested column is done in Java layer, the performance is not good, and it creates a lot of unnecessary Java objects.
We should move it to the native after figuring out a way to pass the nested data type to the native.

This requirement is from #8173 (comment)

@firestarman firestarman added feature request New feature or request Needs Triage Need team to review and classify labels May 7, 2021
@jlowe jlowe added Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS and removed 4 - Needs cuDF (Java) Reviewer Needs Triage Need team to review and classify labels May 7, 2021
@nvdbaranec
Copy link
Contributor

I believe we have everything we need to do this now. make_column_from_scalar() now supports structs and lists

@firestarman
Copy link
Contributor Author

firestarman commented May 25, 2021

I believe we have everything we need to do this now. make_column_from_scalar() now supports structs and lists

Thanks for the information, however seems this is not related to the make_column_from_scalar. Instead it is about creating an empty column according to the nested data type, not a scalar.

The key point for this issue is how to pass the nested data type to the native via JNI.

@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request Java Affects Java cuDF API. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Development

No branches or pull requests

4 participants