[WIP] Arrow (file) datasource

[Dandandan]

Which issue does this PR close?

• Waiting on Implement projection for arrow IPC Reader file / streams arrow-rs#1339
• add test
• linting
• change to use ipc file format

Closes #1857

Rationale for this change

Support reading arrow files.

❯ CREATE EXTERNAL TABLE t STORED AS ARROW LOCATION 'file.arrow';
0 rows in set. Query took 0.004 seconds.
❯ select * from t;
+----+-----+-------+
| f0 | f1  | f2    |
+----+-----+-------+
| 1  | foo | true  |
| 2  | bar |       |
| 3  | baz | false |
| 4  |     | true  |
+----+-----+-------+
4 rows in set. Query took 0.006 seconds.

What changes are included in this PR?

Are there any user-facing changes?

[matthewmturner]

Excited for this! I have a PR for writing arrow in #1893. I'll pause on that until this is done so that i can better test the write functionality. And ill align to your file names etc.

[alamb]

#2048 has the arrow upgrade

[matthewmturner]

hi @Dandandan - do you have an idea how close this PR to being ready?

[Dandandan]

hi @Dandandan - do you have an idea how close this PR to being ready?

Hey @matthewmturner I didn't put in the work yet to finalize the PR.

I think not that much work is needed, we should update the changes in DataFusion, and use the change in the new Arrow version, and add some more tests.
Probably won't have time the coming days, so feel free
if you want to have it merged soon.

[alamb]

marking as draft (so it is easer to see what PRs are waiting for review)

[matthewmturner]

apologies - i have been busy working on some other things so didnt follow up. I do hope to help get this over the line soon if @Dandandan doesnt get the chance.

[alamb]

No worries -- nothing to apologize for -- I am just trying to keep the reviews flowing :)

[alamb]

This PR is more than 6 month old, so closing it down for now to clean up the PR list. Please reopen if this is a mistake and you plan to work on it more