Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MATLAB] Add IPC RecordBatchStreamReader MATLAB class #44923

Open
sgilmore10 opened this issue Dec 3, 2024 · 0 comments
Open

[MATLAB] Add IPC RecordBatchStreamReader MATLAB class #44923

sgilmore10 opened this issue Dec 3, 2024 · 0 comments
Assignees
@kevingurney
Labels
Component: MATLAB Type: enhancement

Comments

@sgilmore10
Copy link
Member

sgilmore10 commented Dec 3, 2024 •
edited
Loading

Describe the enhancement requested

To enable support for the IPC Streaming format in the MATLAB interface, we should add a RecordBatchStreamReader class.

This is a followup issue to #44922

Component(s)

MATLAB
@github-project-automation github-project-automation bot moved this to Backlog in Arrow MATLAB Dec 3, 2024
@sgilmore10 sgilmore10 mentioned this issue Dec 3, 2024
Merged
sgilmore10 added a commit that referenced this issue Dec 4, 2024
@sgilmore10 @kevingurney @kou
GH-44922: [MATLAB] Add IPC RecordBatchStreamFileWriter MATLAB class (…
2b5a802 
…#44925)

### Rationale for this change

To enable support for the IPC Streaming format in the MATLAB interface, we should add a `RecordBatchStreamWriter` class.

### What changes are included in this PR?

Added `arrow.io.ipc.RecordBatchStreamWriter` class.

**Example Usage:**

```matlab
>> city = ["Boston" "Seattle" "Denver" "Juno" "Anchorage" "Chicago"]';
>> daylength = duration(["15:17:01" "15:59:16" "14:59:14" "19:21:23" "14:18:24" "15:13:39"])';
>> matlabTable = table(city, daylength, VariableNames=["City", "DayLength"]);
>> recordBatch1 = arrow.recordBatch(matlabTable(1:4, :))
>> recordBatch2 = arrow.recordBatch(matlabTable(5:end, :));

>> writer = arrow.io.ipc.RecordBatchStreamWriter("daylight.arrow", recordBatch1.Schema);
>> writer.writeRecordBatch(recordBatch1);
>> writer.writeRecordBatch(recordBatch2);
>> writer.close();
```

### Are these changes tested?

Yes. I Parameterized the test cases in `test/arrow/io/ipc/tRecordBatchWriter.m` to test the behavior of both `arrow.io.ipc.RecordBatchFileWriter` AND `arrow.io.ipc.RecordBatchStreamWriter`.

### Are there any user-facing changes?

Yes. Users can now use `arrow.io.ipc.RecordBatchStreamWriter` to serialize `RecordBatch`es/`Table`s to the Arrow IPC Streaming format.

### Future Directions

1. #44923
* GitHub Issue: #44922

Lead-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Sarah Gilmore <[email protected]>
Co-authored-by: Kevin Gurney <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sarah Gilmore <[email protected]>
@kevingurney kevingurney changed the title [MATLAB] Add IPC RecordBatchStreamFileReader MATLAB class [MATLAB] Add IPC RecordBatchStreamReader MATLAB class Dec 4, 2024
@kevingurney kevingurney self-assigned this Dec 4, 2024
@github-actions github-actions bot mentioned this issue Dec 18, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kevingurney kevingurney

Labels
Component: MATLAB Type: enhancement
Projects
Arrow MATLAB
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kevingurney @sgilmore10