Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] Streaming physical writes for native executor #2992

Merged
merged 18 commits into from
Oct 31, 2024
+2,135 −421

if we move all file writer stuff to daft writer then the ray runner c…

bc9c7fa
Select commit
Loading
Failed to load commit list.
Merged

[FEAT] Streaming physical writes for native executor #2992

if we move all file writer stuff to daft writer then the ray runner c…
bc9c7fa
Select commit
Loading
Failed to load commit list.
Codecov / codecov/patch succeeded Oct 29, 2024 in 1s

82.43% of diff hit (target 78.30%)

View this Pull Request on Codecov

82.43% of diff hit (target 78.30%)

Annotations

Check warning on line 3 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L1-L3 

Added lines #L1 - L3 were not covered by tests

Check warning on line 7 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L5-L7 

Added lines #L5 - L7 were not covered by tests

Check warning on line 14 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L12-L14 

Added lines #L12 - L14 were not covered by tests

Check warning on line 18 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L18 

Added line #L18 was not covered by tests

Check warning on line 22 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L21-L22 

Added lines #L21 - L22 were not covered by tests

Check warning on line 35 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L32-L35 

Added lines #L32 - L35 were not covered by tests

Check warning on line 40 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L37-L40 

Added lines #L37 - L40 were not covered by tests

Check warning on line 44 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L44 

Added line #L44 was not covered by tests

Check warning on line 46 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L46 

Added line #L46 was not covered by tests

Check warning on line 50 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L48-L50 

Added lines #L48 - L50 were not covered by tests

Check warning on line 53 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L52-L53 

Added lines #L52 - L53 were not covered by tests

Check warning on line 56 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L55-L56 

Added lines #L55 - L56 were not covered by tests

Check warning on line 62 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L62 

Added line #L62 was not covered by tests

Check warning on line 65 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L64-L65 

Added lines #L64 - L65 were not covered by tests

Check warning on line 71 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L71 

Added line #L71 was not covered by tests

Check warning on line 75 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L74-L75 

Added lines #L74 - L75 were not covered by tests

Check warning on line 83 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L83 

Added line #L83 was not covered by tests

Check warning on line 93 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L92-L93 

Added lines #L92 - L93 were not covered by tests

Check warning on line 104 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L101-L104 

Added lines #L101 - L104 were not covered by tests

Check warning on line 108 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L106-L108 

Added lines #L106 - L108 were not covered by tests

Check warning on line 114 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L110-L114 

Added lines #L110 - L114 were not covered by tests

Check warning on line 118 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L117-L118 

Added lines #L117 - L118 were not covered by tests

Check warning on line 125 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L125 

Added line #L125 was not covered by tests

Check warning on line 134 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L133-L134 

Added lines #L133 - L134 were not covered by tests

Check warning on line 142 in daft/io/writer.py

See this annotation in the file changed.

@codecov codecov / codecov/patch

daft/io/writer.py#L139-L142 

Added lines #L139 - L142 were not covered by tests
View more details on Codecov