[FEA] Support posexplode in nested type columns. #6025
Labels
feature request
New feature or request
libcudf
Affects libcudf (C++/CUDA) code.
Spark
Functionality that helps Spark RAPIDS
This looks like extension of #2975. Please let me know if we could merge this FEA in #2975 itself.
When processing a nested column, create a new row for each element with position in the given array or map column.
Spark API docs
Example:
val df = Seq(List(1,2,3),List(3,4,5)).toDF("Integers")
df.show
+---------+
| Integers|
+---------+
|[1, 2, 3]|
|[4, 5, 6]|
+---------+
df.select(posexplode($"Integers")).show
+---+---+
|pos|col|
+---+---+
| 0| 1|
| 1| 2|
| 2| 3|
| 0| 4|
| 1| 5|
| 2| 6|
+---+---+
The text was updated successfully, but these errors were encountered: