[FEAT] [CSV Reader] Bulk CSV reader + general CSV reader refactor

[clarkzinzow]

This PR adds support for bulk CSV reading to the native CSV reader, and integrates bulk CSV reading with MicroPartition as the default reading path.

Driveby Refactors

• Consolidated the execution-side configuration of CSV reading into 3 config classes: CsvConvertOptions, CsvParseOptions, and CsvReadOptions. This reduces the bloat of our execution-side code (and tests) by a good bit, and providing config objects that are transparently passed through the execution layer should make it easier to add more CSV configuration options in the future (less code to change). Note that these are currently not exposed to the query plan (logical or physical), although these might be moved into the FileFormatConfig enum once the old Table-based I/O path is removed.
• CSV reading has been refactored into a pipeline of stream transformations.
• A lot of repeated intermediate CSV reading code has been eliminated by making the Arrow2 CSV reader a trait object.
• Stats gathered during schema inference are now bundled into a CsvReadStats struct.

[codecov]

Codecov Report

Merging #1614 (16435d0) into main (06c2ccf) will increase coverage by 0.01%.
The diff coverage is 100.00%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1614      +/-   ##
==========================================
+ Coverage   85.00%   85.01%   +0.01%     
==========================================
  Files          55       55              
  Lines        5287     5291       +4     
==========================================
+ Hits         4494     4498       +4     
  Misses        793      793              

Files	Coverage Δ
daft/logical/schema.py	91.50% <100.00%> (+0.08%)	⬆️
daft/table/micropartition.py	89.58% <100.00%> (ø)
daft/table/schema_inference.py	98.14% <100.00%> (ø)
daft/table/table.py	84.01% <100.00%> (ø)
daft/table/table_io.py	95.51% <100.00%> (+0.08%)	⬆️

[samster25]

you can use the pyo3 get attribute instead i believe
https://pyo3.rs/v0.20.0/class?search=get

[clarkzinzow]

You unfortunately cannot use the field-wise get attribute within the context of a cfg_attr on the struct, and we can't use the get_all workaround since some of the getters aren't trivial, so we need to explicitly define getters for each field.

PyO3/pyo3#1003

[samster25]

You can try R: AsyncReader instead to drop the 'static requirement

[clarkzinzow]

AsyncReader is a struct, not a trait, and we're bounding the async-readable type that the AsyncReader is instantiated with. Although I might be misunderstanding your suggestion here. https://docs.rs/arrow2/latest/arrow2/io/csv/read_async/struct.AsyncReader.html

We need the R: AsyncRead bound on the inner async-readable for constructing the AsyncReader (link) and for reading from the reader with read_rows (link).