Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support merge io read api by settings #3675

Closed
sundy-li opened this issue Nov 28, 2023 · 1 comment
Closed

support merge io read api by settings #3675

sundy-li opened this issue Nov 28, 2023 · 1 comment

Comments

@sundy-li
Copy link
Contributor

opendal could support batch read API that coalesces the ranges into merge ranges that send merged requests into storage.

Something like

  1. https://docs.rs/object_store/latest/src/object_store/lib.rs.html#581-588
  2. https://github.com/datafuselabs/databend/blob/a98335d33e7abfd34189e7f32c06ab34d53c64d0/src/query/storages/fuse/src/io/read/block/block_reader_merge_io_async.rs

Notice that the seek of some backends (eg: fs) is not costable, so it's better to read synchronized than join futures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants