Avoid unecessary copies in Arrow IPC reader #2437
Labels
arrow
Changes to the arrow crate
enhancement
Any new improvement worthy of a entry in the changelog
good first issue
Good for newcomers
performance
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The Arrow IPC format is designed to avoid memory copies when moving data from one implementation to another. However, as noted by @tustvold on #2369 (comment), the arrow-rs ipc reader implementation is actually copying data unnecessarily
Describe the solution you'd like
In the ipc code, create a Buffer initially and rewrite the ipc implementation in terms of
Buffer
rather than&[u8]
(as the final output needs to be in aBuffer
Describe alternatives you've considered
N/A
Additional context
Came up in the context of #2369
Possibly also related to #189
The text was updated successfully, but these errors were encountered: