0.2.0-incubating

This is the second official apache release for Apache XTable, an incubating project under the Apache Software Foundation. Apache XTable™ (Incubating) is a cross-table converter for table formats that facilitates omni-directional interoperability across data processing systems and query engines. Currently, Apache XTable™ supports widely adopted open-source table formats such as Apache Hudi, Apache Iceberg, and Delta Lake.

Quick Start Guide

https://xtable.apache.org/docs/how-to

Features

Apache XTable™ (Incubating) provides users with the ability to translate metadata from one table format to another.

Apache XTable™ (Incubating) provides two sync modes, "incremental" and "full." The incremental mode is more lightweight and has better performance, especially on large tables. If there is anything that prevents the incremental mode from working properly, the tool will fall back to the full sync mode.

This sync provides users with the following:

1. Syncing of data files along with their column level statistics and partition metadata
2. Schema updates in the source are reflected in the target table metadata
3. Metadata maintenance for the target table formats.
   • For Hudi, unreferenced files will be marked as cleaned to control the size of the metadata table.
   • For Iceberg, snapshots will be expired after a configured amount of time.
   • For Delta, the transaction log will be retained for a configured amount of time.

Summarized Change Log

1. Updated LICENSE and NOTICE file contents based on guidance given by LEGAL-684.
2. We have addressed feedback on ASF best practices by IPMC member @pjfanning, all these issues were marked as release blockers and have been resolved.
3. Fix the edge case when handling non numeric values of double type in delta stats.
4. Add a UUID Type.
5. Improved downloads page and other refactoring efforts.
6. We have excluded xtable-utilities from the release process because it's a bundled jar containing non ASF compliant dependencies, we will include it back as part of 0.3.0 release. You won't find this module in the staged artifacts or in the source releases distributions for 0.2.0.
7. Upgrade dependency versions based on CVE-2024-47561

Source Distributions

• Source Release: Apache XTable 0.2.0-incubating Source Release (asc, sha512)

Maven Artifacts

• Maven Search: Apache XTable 0.2.0-incubating

What's Changed

• [297] Refactor config classes by @the-other-tim-brown in #480
• [MINOR] Moving to 0.2.0-SNAPSHOT on main branch. by @vinishjail97 in #515
• [DOCS] add releases page and modify README.md by @sagarlakshmipathy in #520
• [DOCS] added Downloads link to the site homepage by @sagarlakshmipathy in #522
• Update Downloads page to adhere to the ASF policy by @zabetak in #527
• Remove downloads page from website by @vinishjail97 in #534
• Set keep_files as false in GH action for site deploy to remove lingering pages by @vinishjail97 in #535
• [531] xtable poms are missing license details. by @rangareddy in #533
• [530] xtable-core pom has a hardcoded scala version for jackson-module-scala by @rangareddy in #537
• [536] Exclude xtable-utilities from mvn deploy by @vinishjail97 in #540
• [532] Add DISCLAIMER file to META-INF folder in jars by @rangareddy in #539
• [536] Update LICENSE and NOTICE files by @vinishjail97 in #541
• update import statements match Delta version upgrade in PR 349 by @zhen-d in #542
• Fix the edge case when handling non numeric values of double type in delta stats by @emilie-wang in #526
• [112] Add a UUID Type by @danielhumanmod in #518
• [546] Update Lombok Dependency Scope in Project by @vamsikarnika in #547
• [529] Build xtable with scala version(s) by @rangareddy in #544
• [551] Exclude xtable-utilities from release guide by @vinishjail97 in #552
• use avro 1.11.4 due to CVE-2024-47561 by @pjfanning in #559
• [561] Add timeout to Github Action by @the-other-tim-brown in #562
• [560] Increment Protobuf version due to vulnerability by @the-other-tim-brown in #557
• [564] mirror parent dependency scopes in child pom files by @the-other-tim-brown in #563
• [551] Remove dependencies in LICENSE and NOTICE files based on feedback in LEGAL-684 by @vinishjail97 in #566
• Update xtable-hadoop-defaults.xml reference by @emmanuel-ferdman in #568
• [551] Create separate LICENSE and NOTICE files for xtable-hudi-support-extensions by @vinishjail97 in #572

New Contributors

• @rangareddy made their first contribution in #533
• @zhen-d made their first contribution in #542
• @danielhumanmod made their first contribution in #518
• @vamsikarnika made their first contribution in #547
• @pjfanning made their first contribution in #559
• @emmanuel-ferdman made their first contribution in #568

Full Changelog: 0.1.0-incubating...0.2.0-incubating