diff --git a/CHANGELOG b/CHANGELOG index cce79a8..efb7d1b 100644 --- a/CHANGELOG +++ b/CHANGELOG @@ -1,3 +1,28 @@ +Version 0.5.0 (2024-11-01) +-------------------------- +Disable asynchronous deletes for Hudi (#94) +Upgrade dependencies (#92) +DynamoDB runtime dependencies for Delta S3 multi-writer +PDP-1323 Handle initialization errors for GCP +PDP-1324 - Handle initialization errors for lake loader Delta Azure +Upgrade common-streams to 0.8.0-M5 (#83) +Delete files asynchronously (#82) +Upgrade common-streams 0.8.0-M4 (#81) +Avoid error on duplicate view name (#80) +Add option to exit on missing Iglu schemas (#79) +common-streams 0.8.x with refactored health monitoring (#78) +Create table concurrently with subscribing to stream of events (#77) +Iceberg fail fast if missing permissions on the catalog (#76) +Make alert messages more human-readable (#75) +Hudi loader should fail early if missing permissions on Glue catalog (#72) +Add alert & retry for delta/s3 initialization (#74) +Implement alerting and retrying mechanisms +Bump aws-hudi to 1.0.0-beta2 (#71) +Bump hudi to 0.15.0 (#70) +Allow disregarding Iglu field's nullability when creating output columns (#66) +Extend health probe to report unhealthy on more error scenarios (#69) +Fix bad rows resizing (#68) + Version 0.4.1 (2024-06-04) -------------------------- Fix missing s3 dependency for Delta diff --git a/README.md b/README.md index 75e521b..2fa42d6 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ This project contains applications required to load Snowplow data into Open Table Formats. -Lake Loader 0.4.1 supports [Delta](https://docs.delta.io/latest/index.html), [Iceberg](https://iceberg.apache.org/docs/latest/) and [Hudi](https://hudi.apache.org/docs/overview/) as output formats. +Lake Loader 0.5.0 supports [Delta](https://docs.delta.io/latest/index.html), [Iceberg](https://iceberg.apache.org/docs/latest/) and [Hudi](https://hudi.apache.org/docs/overview/) as output formats. Check out [the example config files](./config) for how to configure your lake loader. @@ -22,7 +22,7 @@ Basic usage: docker run \ -v /path/to/config.hocon:/var/config.hocon \ -v /path/to/iglu.json:/var/iglu.json \ - snowplow/lake-loader-azure:0.4.1 \ + snowplow/lake-loader-azure:0.5.0 \ --config /var/config.hocon \ --iglu-config /var/iglu.json ``` @@ -35,7 +35,7 @@ The GCP lake loader reads the stream of enriched events from Pubsub and writes t docker run \ -v /path/to/config.hocon:/var/config.hocon \ -v /path/to/iglu.json:/var/iglu.json \ - snowplow/lake-loader-gcp:0.4.1 \ + snowplow/lake-loader-gcp:0.5.0 \ --config /var/config.hocon \ --iglu-config /var/iglu.json ``` @@ -48,7 +48,7 @@ The AWS lake loader reads the stream of enriched events from Kinesis and writes docker run \ -v /path/to/config.hocon:/var/config.hocon \ -v /path/to/iglu.json:/var/iglu.json \ - snowplow/lake-loader-aws:0.4.1 \ + snowplow/lake-loader-aws:0.5.0 \ --config /var/config.hocon \ --iglu-config /var/iglu.json ``` @@ -79,7 +79,7 @@ Licensed under the [Snowplow Limited Use License Agreement][license]. _(If you a [build-image]: https://github.com/snowplow-incubator/snowplow-lake-loader/workflows/CI/badge.svg [build]: https://github.com/snowplow-incubator/snowplow-lake-loader/actions/workflows/ci.yml -[release-image]: https://img.shields.io/badge/release-0.4.1-blue.svg?style=flat +[release-image]: https://img.shields.io/badge/release-0.5.0-blue.svg?style=flat [releases]: https://github.com/snowplow-incubator/snowplow-lake-loader/releases [license]: https://docs.snowplow.io/limited-use-license-1.0