Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added support to the Arrow C stream interface (read and write) #857

Merged
merged 7 commits into from
Feb 22, 2022

Conversation

jorgecarleitao
Copy link
Owner

@jorgecarleitao jorgecarleitao commented Feb 21, 2022

Motived by @wjones127 's suggestion here, this PR adds support to read from and write to the Arrow C stream interface.

The C stream interface is in essence FFI of an iterator of arrow arrays with a common data_type; it allows implementation A to request a new item from implementation B.

Also closes #859

@codecov
Copy link

codecov bot commented Feb 21, 2022

Codecov Report

Merging #857 (066723a) into main (b6a6780) will decrease coverage by 0.03%.
The diff coverage is 76.30%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #857      +/-   ##
==========================================
- Coverage   71.48%   71.45%   -0.04%     
==========================================
  Files         335      335              
  Lines       17897    17986      +89     
==========================================
+ Hits        12793    12851      +58     
- Misses       5104     5135      +31     
Impacted Files Coverage Δ
src/buffer/bytes.rs 51.35% <ø> (ø)
src/ffi/stream.rs 62.92% <62.92%> (ø)
src/ffi/schema.rs 68.47% <70.00%> (ø)
src/ffi/array.rs 81.09% <84.02%> (+21.09%) ⬆️
src/ffi/mod.rs 100.00% <100.00%> (ø)
src/compute/arithmetics/time.rs 26.60% <0.00%> (+0.91%) ⬆️
src/bitmap/utils/slice_iterator.rs 87.93% <0.00%> (+1.72%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6a6780...066723a. Read the comment docs.

@jorgecarleitao jorgecarleitao marked this pull request as ready for review February 21, 2022 21:11
@jorgecarleitao
Copy link
Owner Author

Big kudos to @pitrou for the write up of the C stream interface (see apache/arrow#8052); beautifully well written specification very easy to follow and with very little room for ambiguity.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Renamed Ffi_ArrowArray and Ffi_ArrowSchema
1 participant