Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove arrow dependency in cuIO by implementing a host_buffer_source #14661

Closed
vuule opened this issue Dec 20, 2023 · 5 comments
Closed

Remove arrow dependency in cuIO by implementing a host_buffer_source #14661

vuule opened this issue Dec 20, 2023 · 5 comments
Assignees
Labels
cuIO cuIO issue improvement Improvement / enhancement to an existing function

Comments

@vuule
Copy link
Contributor

vuule commented Dec 20, 2023

Currently we use arrow::io::BufferReader to implement a zero-copy datasource for host buffers. This is a feature we could implement ourselves and remove the last of arrow dependency in cuIO.

A requirement for this source type would be to avoid copying the data when calling std::unique_ptr<datasource::buffer> host_read(size_t offset, size_t size). This is what device_buffer_source alrady does in its device_read implementation.

@ZelboK
Copy link
Contributor

ZelboK commented Dec 27, 2023

Can I take this

@vyasr
Copy link
Contributor

vyasr commented May 15, 2024

Related to #15193

@ZelboK
Copy link
Contributor

ZelboK commented Jul 25, 2024

I believe this should be closed per #15189 @vyasr @vuule

@jakirkham
Copy link
Member

Should we close this or is there more to do here?

@vyasr
Copy link
Contributor

vyasr commented Aug 16, 2024

Thanks for the ping @ZelboK! Yes, this was resolved by #15189.

@vyasr vyasr closed this as completed Aug 16, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in cuDF/Dask/Numba/UCX Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cuIO cuIO issue improvement Improvement / enhancement to an existing function
Projects
Status: Done
Development

No branches or pull requests

4 participants