Skip to content

Commit

Permalink
chore(rust): update rust changelog for 0.7.0 (delta-io#1137)
Browse files Browse the repository at this point in the history
# Description

FYI I am merging `CHANGELOG.md` and `CHANGELOG-old.md`, since I think
people would rather scroll through all past version in one file.
(Otherwise, what's the benefit over looking at the Releases tab on
GitHub?) This merge happens manually, not in the script.

I went over and pruned the release log manually, removing any internal
facing issues or Python-specific pull requests (which don't get filtered
out since we only tag issues not PRs).

# Related Issue(s)

- closes delta-io#1035

# Documentation

<!---
Share links to useful documentation
--->
  • Loading branch information
wjones127 authored and chitralverma committed Mar 17, 2023
1 parent de12e5d commit ce4b72a
Show file tree
Hide file tree
Showing 5 changed files with 78 additions and 21 deletions.
1 change: 0 additions & 1 deletion CHANGELOG-old.md

This file was deleted.

54 changes: 53 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,58 @@
# Changelog

## [rust-v0.7.0](https://github.com/delta-io/delta-rs/tree/rust-v0.7.0) (2023-02-11)

[Full Changelog](https://github.com/delta-io/delta-rs/compare/rust-v0.6.0...rust-v0.7.0)

**Implemented enhancements:**

- Support FSCK REPAIR TABLE Operation [\#1092](https://github.com/delta-io/delta-rs/issues/1092)
- Expose the Delta Log in a DataFrame that's easy for analysis [\#1031](https://github.com/delta-io/delta-rs/issues/1031)
- Provide case-insensitive storage options in backend [\#999](https://github.com/delta-io/delta-rs/issues/999)
- Support local file path in CreateBuilder::with\_location\(\) [\#998](https://github.com/delta-io/delta-rs/issues/998)
- Save operational params in the same way with delta io [\#1054](https://github.com/delta-io/delta-rs/pull/1054) ([ismoshkov](https://github.com/ismoshkov))

**Fixed bugs:**

- DeltaTable DataFusion TableProvider does not support filter pushdown [\#1064](https://github.com/delta-io/delta-rs/issues/1064)
- DeltaTable DataFusion scan does not prune files properly [\#1063](https://github.com/delta-io/delta-rs/issues/1063)
- deltalake.DeltaTable constructor hangs in Jupyter [\#1093](https://github.com/delta-io/delta-rs/issues/1093)
- Transaction log JSON formatting issue when writing data via Python bindings [\#1017](https://github.com/delta-io/delta-rs/issues/1017)
- crates.io entry is missing link to rustdoc documentation [\#1076](https://github.com/delta-io/delta-rs/issues/1076)
- URL Registered with ObjectStore registry is different from url in DeltaScan [\#1018](https://github.com/delta-io/delta-rs/issues/1018)
- Not able to connect to Azure Storage with client id/secret [\#977](https://github.com/delta-io/delta-rs/issues/977)
- Deltalake 0.5 crate s3 feature dynamodb version mismatch [\#973](https://github.com/delta-io/delta-rs/issues/973)
- Overwrite mode does not work with Azure [\#939](https://github.com/delta-io/delta-rs/issues/939)
- Use Chrono without default features [\#914](https://github.com/delta-io/delta-rs/issues/914)
- `cargo test` does not run due to tls conflict [\#985](https://github.com/delta-io/delta-rs/issues/985)
- Azure SAS authorization fails with `<AuthenticationErrorDetail>Signature fields not well formed.` [\#910](https://github.com/delta-io/delta-rs/issues/910)

**Merged pull requests:**

- Make rustls default across all packages [\#1097](https://github.com/delta-io/delta-rs/pull/1097) ([wjones127](https://github.com/wjones127))
- Implement filesystem check [\#1103](https://github.com/delta-io/delta-rs/pull/1103) ([Blajda](https://github.com/Blajda))
- refactor: move vacuum command to operations module [\#1045](https://github.com/delta-io/delta-rs/pull/1045) ([roeap](https://github.com/roeap))
- feat: enable passing storage options to Delta table builder via DataFusion's CREATE EXTERNAL TABLE [\#1043](https://github.com/delta-io/delta-rs/pull/1043) ([gruuya](https://github.com/gruuya))
- feat: improve storage location handling [\#1065](https://github.com/delta-io/delta-rs/pull/1065) ([roeap](https://github.com/roeap))
- Fix to support UTC timezone [\#1022](https://github.com/delta-io/delta-rs/pull/1022) ([andrei-ionescu](https://github.com/andrei-ionescu))
- feat: harmonize and simplify storage configuration [\#1052](https://github.com/delta-io/delta-rs/pull/1052) ([roeap](https://github.com/roeap))
- feat: expose function to get table of add actions [\#1033](https://github.com/delta-io/delta-rs/pull/1033) ([wjones127](https://github.com/wjones127))
- fix: change unexpected field logging level to debug [\#1112](https://github.com/delta-io/delta-rs/pull/1112) ([houqp](https://github.com/houqp))
- fix: datafusion predicate pushdown and dependencies [\#1071](https://github.com/delta-io/delta-rs/pull/1071) ([roeap](https://github.com/roeap))
- fix: azure sas key url encoding [\#1036](https://github.com/delta-io/delta-rs/pull/1036) ([roeap](https://github.com/roeap))
- Add provisional workaround to support CDC \#1039 [\#1042](https://github.com/delta-io/delta-rs/pull/1042) ([Fazzani](https://github.com/Fazzani))
- improve debuggability of json ser/de errors [\#1119](https://github.com/delta-io/delta-rs/pull/1119) ([houqp](https://github.com/houqp))
- Add an example of writing to a delta table with a RecordBatch [\#1085](https://github.com/delta-io/delta-rs/pull/1085) ([rtyler](https://github.com/rtyler))
- minor: optimize partition lookup for vacuum loop [\#1120](https://github.com/delta-io/delta-rs/pull/1120) ([houqp](https://github.com/houqp))
- Add missing documentation metadata to Cargo.toml [\#1077](https://github.com/delta-io/delta-rs/pull/1077) ([johnbatty](https://github.com/johnbatty))
- add test for null\_count\_schema\_for\_fields [\#1135](https://github.com/delta-io/delta-rs/pull/1135) ([marijncv](https://github.com/marijncv))
- add test for min\_max\_schema\_for\_fields [\#1122](https://github.com/delta-io/delta-rs/pull/1122) ([marijncv](https://github.com/marijncv))
- add test for get\_boolean\_from\_metadata [\#1121](https://github.com/delta-io/delta-rs/pull/1121) ([marijncv](https://github.com/marijncv))
- add test for left\_larger\_than\_right [\#1110](https://github.com/delta-io/delta-rs/pull/1110) ([marijncv](https://github.com/marijncv))
- Add test for: to\_scalar\_value [\#1086](https://github.com/delta-io/delta-rs/pull/1086) ([marijncv](https://github.com/marijncv))
- Fix typo in delta-inspect [\#1072](https://github.com/delta-io/delta-rs/pull/1072) ([byteink](https://github.com/byteink))
- chore: update datafusion [\#1114](https://github.com/delta-io/delta-rs/pull/1114) ([roeap](https://github.com/roeap))

## [rust-v0.6.0](https://github.com/delta-io/delta-rs/tree/rust-v0.6.0) (2022-12-16)

[Full Changelog](https://github.com/delta-io/delta-rs/compare/rust-v0.5.0...rust-v0.6.0)
Expand Down Expand Up @@ -39,5 +92,4 @@
- test\(python\): add azure integration tests [\#912](https://github.com/delta-io/delta-rs/pull/912) ([wjones127](https://github.com/wjones127))



\* *This Changelog was automatically generated by [github_changelog_generator](https://github.com/github-changelog-generator/github-changelog-generator)*
2 changes: 1 addition & 1 deletion delta-inspect/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "delta-inspect"
version = "0.6.0"
version = "0.7.0"
authors = ["Qingping Hou <[email protected]>"]
homepage = "https://github.com/delta-io/delta.rs"
license = "Apache-2.0"
Expand Down
40 changes: 23 additions & 17 deletions dev/release/update_change_log.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,22 @@
#
# Usage:
# GITHUB_API_TOKEN=<TOKEN> ./update_change_log.sh
#
# Please edit the resulting change log to remove remaining irrelevant changes
# and to put interesting changes (features, bugfixes) above all the minor
# changes (depandabot updates).

set -e

LANGUAGE="rust"
SINCE_VERSION="0.5.0"
FUTURE_RELEASE="0.6.0"
SINCE_VERSION="0.6.0"
FUTURE_RELEASE="0.7.0"

# only consider tags of the correct language
if [ "$LANGUAGE" == "rust" ]; then
EXCLUDED_LANGUAGES_REGEX="python.*"
EXCLUDED_LANGUAGES_REGEX=".*python.*"
elif [ "$LANGUAGE" == "python" ]; then
EXCLUDED_LANGUAGES_REGEX="rust.*"
EXCLUDED_LANGUAGES_REGEX=".*rust.*"
else
echo "Language $LANGUAGE is invalid. Should be one of Python and Rust."
fi
Expand All @@ -28,23 +32,17 @@ SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"

OUTPUT_PATH="${SOURCE_TOP_DIR}/CHANGELOG.md"
OLD_OUTPUT_PATH="${SOURCE_TOP_DIR}/CHANGELOG-old.md"
HISTORIAL_PATH="${SOURCE_TOP_DIR}/CHANGELOG-old.md"

# remove header so github-changelog-generator has a clean base to append
sed -i '1d' "${OUTPUT_PATH}"
sed -i '1d' "${OLD_OUTPUT_PATH}"
# remove the github-changelog-generator footer from the old CHANGELOG.md
LINE_COUNT=$(wc -l <"${OUTPUT_PATH}")
sed -i "$(( $LINE_COUNT-3 )),$ d" "${OUTPUT_PATH}"
cp $OUTPUT_PATH $HISTORIAL_PATH

# Copy the previous CHANGELOG.md to CHANGELOG-old.md
echo '# Historical Changelog
' | cat - "${OUTPUT_PATH}" "${OLD_OUTPUT_PATH}" > "${OLD_OUTPUT_PATH}".tmp
mv "${OLD_OUTPUT_PATH}".tmp "${OLD_OUTPUT_PATH}"
# Remove header from historical change logs; will add back at the end.
sed -i.bak '1d' "${HISTORIAL_PATH}"

# use exclude-tags-regex to filter out tags used in the wrong language
pushd "${SOURCE_TOP_DIR}"
docker run -it --rm -e CHANGELOG_GITHUB_TOKEN="$GITHUB_API_TOKEN" -v "$(pwd)":/usr/local/src/your-app githubchangeloggenerator/github-changelog-generator \
docker run -it --rm -e CHANGELOG_GITHUB_TOKEN="$GITHUB_API_TOKEN" \
-v "$(pwd)":/usr/local/src/your-app githubchangeloggenerator/github-changelog-generator \
--user delta-io \
--project delta-rs \
--cache-file=.githubchangeloggenerator.cache \
Expand All @@ -56,4 +54,12 @@ docker run -it --rm -e CHANGELOG_GITHUB_TOKEN="$GITHUB_API_TOKEN" -v "$(pwd)":/u
--since-tag ${LANGUAGE}-v${SINCE_VERSION} \
--future-release ${LANGUAGE}-v${FUTURE_RELEASE}

sed -i.bak "s/\\\n/\n\n/" "${OUTPUT_PATH}"
# Remove footer from github-changelog-generator (we already have one at bottom)
LINE_COUNT=$(wc -l <"${OUTPUT_PATH}")
sed -i.bak "$(( $LINE_COUNT-3 )),$ d" "${OUTPUT_PATH}"

# Add historical change log back in
cat $HISTORIAL_PATH >> $OUTPUT_PATH

# Remove temporary files
rm $HISTORIAL_PATH
2 changes: 1 addition & 1 deletion rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "deltalake"
version = "0.6.0"
version = "0.7.0"
rust-version = "1.64"
authors = ["Qingping Hou <[email protected]>"]
homepage = "https://github.com/delta-io/delta.rs"
Expand Down

0 comments on commit ce4b72a

Please sign in to comment.