Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Reuse partition vectors for arrow scan #931

Closed
jackylee-ch opened this issue May 23, 2022 · 1 comment
Closed

Reuse partition vectors for arrow scan #931

jackylee-ch opened this issue May 23, 2022 · 1 comment
Labels
enhancement New feature or request feature

Comments

@jackylee-ch
Copy link
Contributor

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Every time we read a arrow record batch from Parquet,we will use partitionValues to create partitionVectors and add them to record batch. Actually, the partition value won't change in the reading of one PartitionFile.

Describe the solution you'd like
We can create one partitionVectors for each PartitionFile and reuse it until task finished.

Describe alternatives you've considered
None

Additional context
None

@haojinIntel
Copy link
Collaborator

This issue seems to be fixed by #935. Thanks for your work, and we close it now.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request feature
Projects
None yet
Development

No branches or pull requests

2 participants