feat(connect): read/write → csv, write → json

[andrewgazelka]

No description provided.

[andrewgazelka]

• feat(connect): read/write → csv, write → json #3361 👈 (View in Graphite)
• feat(connect): add parquet support #3360
• [FEAT] connect: to_daft_* use ref instead of value #3355 : 2 other dependent PRs (#3357 , #3362 )
• [FEAT] connect: add alias support #3342 : 6 other dependent PRs (#3344 , #3345 , #3348 and 3 others)
• [FEAT] consolidate Spark session fixture into conftest.py #3341 : 3 other dependent PRs (#3309 , #3326 , #3343 )
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #3361 will degrade performances by 34.82%

_{Comparing andrew/connect-csv-json (59c7803) with main (ea8f8bd)}

Summary

⚡ 1 improvements
❌ 1 regressions
✅ 25 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	andrew/connect-csv-json	Change
⚡	test_iter_rows_first_row[100 Small Files]	345.2 ms	207.1 ms	+66.67%
❌	test_show[100 Small Files]	15.7 ms	24.1 ms	-34.82%

[codecov]

Codecov Report

Attention: Patch coverage is 75.00000% with 4 lines in your changes missing coverage. Please review.

Project coverage is 77.84%. Comparing base (ea8f8bd) to head (59c7803).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...t/src/translation/logical_plan/read/data_source.rs	71.42%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3361      +/-   ##
==========================================
+ Coverage   77.80%   77.84%   +0.03%     
==========================================
  Files         718      718              
  Lines       88247    88250       +3     
==========================================
+ Hits        68662    68698      +36     
+ Misses      19585    19552      -33     

Files with missing lines	Coverage Δ
src/daft-connect/src/op/execute/write.rs	80.00% <100.00%> (+1.95%)	⬆️
...t/src/translation/logical_plan/read/data_source.rs	81.08% <71.42%> (-3.30%)	⬇️

... and 4 files with indirect coverage changes

[andrewgazelka]

.

[raunakab]

Looks good to me. Tests seem straightforward too.