Version 1.9 Release Notes

Jump to bottom

David Freels Sr edited this page Oct 22, 2022 · 8 revisions

1.9.1

(#302) Support Spark 3.2
(#312) Add execution fork index to globals
(#313) Remove DStream Support in favor of Structured Streaming

1.9.2

(#320) Allow initial position to be set for KinesisDataConnector
(#322) BigQueryDataConnector uses a streaming writer now
(#326) metalus-delta now uses Deltalake 2.0.0 when running on Spark 3.2
(#328) Fixed issue when loading multiple files with Deltalake
(#329) Added streaming trigger options to the DataFrameWriterOptions object
(#330) Added new tags to the StepFunction annotation which will be added to the tags metadata for steps

1.9.3

(#337) Push fork information to child executions

1.9.4

Added Spark 3.3 support
Changes to the Metalus GCP jar for Pub/Sub