Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support posexplode in nested type columns. #6025

Closed
nartal1 opened this issue Aug 18, 2020 · 2 comments
Closed

[FEA] Support posexplode in nested type columns. #6025

nartal1 opened this issue Aug 18, 2020 · 2 comments
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS

Comments

@nartal1
Copy link
Member

nartal1 commented Aug 18, 2020

This looks like extension of #2975. Please let me know if we could merge this FEA in #2975 itself.

When processing a nested column, create a new row for each element with position in the given array or map column.
Spark API docs

Example:
val df = Seq(List(1,2,3),List(3,4,5)).toDF("Integers")
df.show
+---------+
| Integers|
+---------+
|[1, 2, 3]|
|[4, 5, 6]|
+---------+

df.select(posexplode($"Integers")).show
+---+---+
|pos|col|
+---+---+
| 0| 1|
| 1| 2|
| 2| 3|
| 0| 4|
| 1| 5|
| 2| 6|
+---+---+

@nartal1 nartal1 added feature request New feature or request Needs Triage Need team to review and classify libcudf Affects libcudf (C++/CUDA) code. labels Aug 18, 2020
@kkraus14 kkraus14 added Spark Functionality that helps Spark RAPIDS and removed Needs Triage Need team to review and classify labels Aug 18, 2020
@github-actions
Copy link

This issue has been marked rotten due to no recent activity in the past 90d. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

@nartal1
Copy link
Member Author

nartal1 commented Feb 17, 2021

This is duplicate of #6151. Closing.

@nartal1 nartal1 closed this as completed Feb 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. Spark Functionality that helps Spark RAPIDS
Projects
None yet
Development

No branches or pull requests

2 participants