Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unshim cache serializer and other 311+-all code [databricks] #5076

Merged
merged 8 commits into from
Mar 29, 2022

Conversation

jlowe
Copy link
Member

@jlowe jlowe commented Mar 28, 2022

Closes #5005. This performs the last remaining unshim work of #5005:

  • Moves the ParquetCachedBatchSerializer out of the shims package
  • Moves tests from the tests-spark310+ directory to the tests project and removes the tests-spark310+ project
  • Adds the ColumnarToRowTransition trait to GpuColumnarToRowExec and removes GpuColumnarToRowTransitionExec
  • Removes references to the sql-plugin/src/main/311+-all directory since no files remain after this PR
  • Moves the contents of dist/unshimmed-spark311.txt to dist/unshimmed-common-from-spark311.txt
  • Moves GpuJoinUtils out of the shims package.

Note that we could use Spark BuildSide instead of GpuBuildSide, but the wrapper here is very small, easy to maintain, and there are many references throughout the code to GpuBuildSide. We could do this as a followup if desired.

@jlowe jlowe added this to the Mar 21 - Apr 1 milestone Mar 28, 2022
@jlowe jlowe self-assigned this Mar 28, 2022
@jlowe
Copy link
Member Author

jlowe commented Mar 28, 2022

build

@jlowe
Copy link
Member Author

jlowe commented Mar 28, 2022

build

tgravescs
tgravescs previously approved these changes Mar 28, 2022
@jlowe
Copy link
Member Author

jlowe commented Mar 29, 2022

build

@jlowe jlowe merged commit 8ca7e98 into NVIDIA:branch-22.06 Mar 29, 2022
@jlowe jlowe deleted the unshim-31x-cache-serializer branch March 29, 2022 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants