From 85089b1c7462561e364d8695866b34cdcea6e5f4 Mon Sep 17 00:00:00 2001 From: "R. Tyler Croy" Date: Wed, 1 May 2024 22:57:05 +0000 Subject: [PATCH] chore: update the changelog to include rust-v0.17.3 --- CHANGELOG.md | 62 ++++++++++++++++++++++++++++++++ dev/release/update_change_log.sh | 4 +-- 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 57226bfbce..e68641da21 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,67 @@ # Changelog +## [rust-v0.17.3](https://github.com/delta-io/delta-rs/tree/rust-v0.17.3) (2024-05-01) + +[Full Changelog](https://github.com/delta-io/delta-rs/compare/rust-v0.17.1...rust-v0.17.3) + +**Implemented enhancements:** + +- Limit concurrent ObjectStore access to avoid resource limitations in constrained environments [\#2457](https://github.com/delta-io/delta-rs/issues/2457) +- How to get a DataFrame in Rust? [\#2404](https://github.com/delta-io/delta-rs/issues/2404) +- Allow checkpoint creation when partion column is "timestampNtz " [\#2381](https://github.com/delta-io/delta-rs/issues/2381) +- is there a way to make writing timestamp\_ntz optional [\#2339](https://github.com/delta-io/delta-rs/issues/2339) +- Update arrow dependency [\#2328](https://github.com/delta-io/delta-rs/issues/2328) +- Release GIL in deltalake.write\_deltalake [\#2234](https://github.com/delta-io/delta-rs/issues/2234) +- Unable to retrieve custom metadata from tables in rust [\#2153](https://github.com/delta-io/delta-rs/issues/2153) +- Refactor commit interface to be a Builder [\#2131](https://github.com/delta-io/delta-rs/issues/2131) + +**Fixed bugs:** + +- Handle rate limiting during write contention [\#2451](https://github.com/delta-io/delta-rs/issues/2451) +- regression : delta.logRetentionDuration don't seems to be respected [\#2447](https://github.com/delta-io/delta-rs/issues/2447) +- Issue writing to mounted storage in AKS using delta-rs library [\#2445](https://github.com/delta-io/delta-rs/issues/2445) +- TableMerger - when\_matched\_delete\(\) fails when Column names contain special characters [\#2438](https://github.com/delta-io/delta-rs/issues/2438) +- Generic DeltaTable error: External error: Arrow error: Invalid argument error: arguments need to have the same data type - while merge data in to delta table [\#2423](https://github.com/delta-io/delta-rs/issues/2423) +- Merge on predicate throw error on date colum: Unable to convert expression to string [\#2420](https://github.com/delta-io/delta-rs/issues/2420) +- Writing Tables with Append mode errors if the schema metadata is different [\#2419](https://github.com/delta-io/delta-rs/issues/2419) +- Logstore issues on AWS Lambda [\#2410](https://github.com/delta-io/delta-rs/issues/2410) +- Datafusion timestamp type doesn't respect delta lake schema [\#2408](https://github.com/delta-io/delta-rs/issues/2408) +- Compacting produces smaller row groups than expected [\#2386](https://github.com/delta-io/delta-rs/issues/2386) +- ValueError: Partition value cannot be parsed from string. [\#2380](https://github.com/delta-io/delta-rs/issues/2380) +- Very slow s3 connection after 0.16.1 [\#2377](https://github.com/delta-io/delta-rs/issues/2377) +- Merge update+insert truncates a delta table if the table is big enough [\#2362](https://github.com/delta-io/delta-rs/issues/2362) +- Do not add readerFeatures or writerFeatures keys under checkpoint files if minReaderVersion or minWriterVersion do not satisfy the requirements [\#2360](https://github.com/delta-io/delta-rs/issues/2360) +- Create empty table failed on rust engine [\#2354](https://github.com/delta-io/delta-rs/issues/2354) +- Getting error message when running in lambda: message: "Too many open files" [\#2353](https://github.com/delta-io/delta-rs/issues/2353) +- Temporary files filling up \_delta\_log folder - increasing table load time [\#2351](https://github.com/delta-io/delta-rs/issues/2351) +- compact fails with merged schemas [\#2347](https://github.com/delta-io/delta-rs/issues/2347) +- Cannot merge into table partitioned by date type column on 0.16.3 [\#2344](https://github.com/delta-io/delta-rs/issues/2344) +- Merge breaks using logical datatype decimal128 [\#2343](https://github.com/delta-io/delta-rs/issues/2343) +- Decimal types are not checked against max precision/scale at table creation [\#2331](https://github.com/delta-io/delta-rs/issues/2331) +- Merge update+insert truncates a delta table [\#2320](https://github.com/delta-io/delta-rs/issues/2320) +- Extract `add.stats_parsed` with wrong type [\#2312](https://github.com/delta-io/delta-rs/issues/2312) +- Process fails without error message when executing merge [\#2310](https://github.com/delta-io/delta-rs/issues/2310) +- delta\_rs don't seems to respect the row group size [\#2309](https://github.com/delta-io/delta-rs/issues/2309) +- Auth error when running inside VS Code [\#2306](https://github.com/delta-io/delta-rs/issues/2306) +- Unable to read deltatables with binary columns: Binary is not supported by JSON [\#2302](https://github.com/delta-io/delta-rs/issues/2302) +- Schema evolution not coercing with Large arrow types [\#2298](https://github.com/delta-io/delta-rs/issues/2298) +- Panic in `deltalake_core::kernel::snapshot::log_segment::list_log_files_with_checkpoint::{{closure}}` [\#2290](https://github.com/delta-io/delta-rs/issues/2290) +- Checkpoint does not preserve reader and writer features for the table protocol. [\#2288](https://github.com/delta-io/delta-rs/issues/2288) +- Z-Order with larger dataset resulting in memory error [\#2284](https://github.com/delta-io/delta-rs/issues/2284) +- Successful writes return error when using concurrent writers [\#2279](https://github.com/delta-io/delta-rs/issues/2279) +- Rust writer should raise when decimal types are incompatible \(currently writers and puts table in invalid state\) [\#2275](https://github.com/delta-io/delta-rs/issues/2275) +- Generic DeltaTable error: Version mismatch with new schema merge functionality in AWS S3 [\#2262](https://github.com/delta-io/delta-rs/issues/2262) +- DeltaTable is not resilient to corrupted checkpoint state [\#2258](https://github.com/delta-io/delta-rs/issues/2258) +- Inconsistent units of time [\#2256](https://github.com/delta-io/delta-rs/issues/2256) +- Partition column comparison is an assertion rather than if block with raise exception [\#2242](https://github.com/delta-io/delta-rs/issues/2242) +- Unable to merge column names starting from numbers [\#2230](https://github.com/delta-io/delta-rs/issues/2230) +- Merging to a table with multiple distinct partitions in parallel fails [\#2227](https://github.com/delta-io/delta-rs/issues/2227) +- cleanup\_metadata not respecting custom `logRetentionDuration` [\#2180](https://github.com/delta-io/delta-rs/issues/2180) +- Merge predicate fails with a field with a space [\#2167](https://github.com/delta-io/delta-rs/issues/2167) +- When\_matched\_update causes records to be lost with explicit predicate [\#2158](https://github.com/delta-io/delta-rs/issues/2158) +- Merge execution time grows exponetially with the number of column [\#2107](https://github.com/delta-io/delta-rs/issues/2107) +- \_internal.DeltaError when merging [\#2084](https://github.com/delta-io/delta-rs/issues/2084) + ## [rust-v0.17.1](https://github.com/delta-io/delta-rs/tree/rust-v0.17.1) (2024-03-06) [Full Changelog](https://github.com/delta-io/delta-rs/compare/rust-v0.17.0...rust-v0.17.1) diff --git a/dev/release/update_change_log.sh b/dev/release/update_change_log.sh index d626804e3b..02592d7d2f 100755 --- a/dev/release/update_change_log.sh +++ b/dev/release/update_change_log.sh @@ -16,8 +16,8 @@ set -e LANGUAGE="rust" -SINCE_VERSION=${SINCE_VERSION:-"0.17.0"} -FUTURE_RELEASE=${FUTURE_RELEASE:-"0.18.0"} +SINCE_VERSION=${SINCE_VERSION:-"0.17.1"} +FUTURE_RELEASE=${FUTURE_RELEASE:-"0.17.3"} # only consider tags of the correct language if [ "$LANGUAGE" == "rust" ]; then