You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
// NOTE: If we add a non-snappy option, file name must change
let file_name = format!("part-{first_part}-{uuid_part}-{last_part}.snappy.parquet");
Also, should we consider using zstd by default? It produces smaller files than snappy, but is faster than gzip (the Iceberg default). It was one of the better ones it my tests of Parquet compression, and I've seen suggested in several other venues for Parquet tables.
Use Case
Related Issue(s)
The text was updated successfully, but these errors were encountered:
…eltaWriter (#1497)
# Description
Adds the capability to pass a configured WriterProperties to the
`RecordBatchWriter` and `DeltaWriter` similar to how the
`OptimizeBuilder` can be updated.
# Related Issue(s)
- closes#1469
- closes#1235
# Documentation
Description
We should probably make this configurable at the table level, if there's some standard for that.
Note: need to change the filename format:
delta-rs/rust/src/writer/utils.rs
Lines 96 to 97 in 8b85469
Also, should we consider using zstd by default? It produces smaller files than snappy, but is faster than gzip (the Iceberg default). It was one of the better ones it my tests of Parquet compression, and I've seen suggested in several other venues for Parquet tables.
Use Case
Related Issue(s)
The text was updated successfully, but these errors were encountered: