-
Notifications
You must be signed in to change notification settings - Fork 234
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add in the GpuArrayFilter command #10763
Conversation
Signed-off-by: Robert (Bobby) Evans <[email protected]>
build |
For performance numbers I ran the following query on Spark 3.4.2
On an a6000 running with 4 concurrent It took 36 seconds in the worst case, I ran it 3 times and the best was about 33 seconds. On a 32-core/64-thread Threadripper PRO 5975WX, using all 64 cores it took 688 seconds (and I didn't really want to run it again). That is a speedup of about 19x. |
sql-plugin/src/main/scala/com/nvidia/spark/rapids/higherOrderFunctions.scala
Show resolved
Hide resolved
build |
sql-plugin/src/main/scala/com/nvidia/spark/rapids/higherOrderFunctions.scala
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/higherOrderFunctions.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/scala/com/nvidia/spark/rapids/higherOrderFunctions.scala
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM other than copyright/indenting that @jlowe pointed out.
build |
The link check is failing because of #10768 |
this fixes #10760
I still need to do some performance comparisons/testing, but functionality wise it looks good. Will update it shortly.