From e80a9534eb40a8fbb70d8e9d8f1c0aabdb36372d Mon Sep 17 00:00:00 2001 From: Jerry Shao Date: Thu, 14 Nov 2024 18:15:32 +0800 Subject: [PATCH] Add download link and release note (#33) * Add download link and release note * Fix the trailing ws --- ...24-11-14-gravitino-0-7-0-release-notes.mdx | 189 ++++++++++++++++++ src/pages/downloads.mdx | 13 ++ 2 files changed, 202 insertions(+) create mode 100644 blog/2024-11-14-gravitino-0-7-0-release-notes.mdx diff --git a/blog/2024-11-14-gravitino-0-7-0-release-notes.mdx b/blog/2024-11-14-gravitino-0-7-0-release-notes.mdx new file mode 100644 index 000000000..fb834788a --- /dev/null +++ b/blog/2024-11-14-gravitino-0-7-0-release-notes.mdx @@ -0,0 +1,189 @@ +--- +title: Apache Gravitino 0.7.0 - strengthen the cloud support for Apache Gravitino™ (incubating) +slug: gravitino-0-7-0-release-notes +authors: [jerryshao] +tags: [apache,gravitino,metadata,multicloud] +--- + +Gravitino 0.7.0 is the second major release after entering the ASF. In this release, the community +mainly focuses on strengthening cloud support, to make Gravitino work better in the cloud environment. + +This release blog will briefly introduce the new features related to cloud support, as well as other +significant features and improvements. Please keep reading to learn more about what the community has worked on. + +## Cloud storage support for Gravitino + +As more and more users run their data stacks on the cloud and use cloud object storage, cloud +storage support becomes an imperative requirement. In this release, the community mainly focuses on +adding cloud storage support for Gravitino and makes sure Gravitino itself and its connectors/sources +can work smoothly with cloud storage. + +In this release: + +* Gravitino Iceberg REST catalog server now supports different cloud storages, including AWS S3, + Google GCS, Aliyun OSS. Users can simply configure it to make it work. +* Gravitino Fileset catalog now supports managing files (objects) stored in S3, GCS, and OSS. + Gravitino provides both server-side pluggable framework and client-side Java / Python GVFS + (Gravitino Virtual File System) SDK. Users can easily use their existing tools with the + Gravitino provided bundled packages to access the data in the cloud storage. Besides, + Gravitino also provides a pluggable framework for users to implement their own storage support. +* Gravitino’s Hive, Paimon, and Iceberg catalogs also adds and verifies the support with + different cloud storage in this release. +* Gravitino’s Spark, Trino connector also verifies to work with cloud storage. + +Overall, with 0.7.0 release Gravitino could generally support working with different cloud storages. +You can check our issue [#4396](https://github.com/apache/gravitino/issues/4396) to know more. Also, +we’re continuing to add more cloud storage support in the following releases, please stay tuned. + +## Credential vending support in Gravitino + +Besides the cloud storage support, credential vending support is also important for Gravitino, +especially to work with cloud storage. The traditional way of using AKSK is not convenient and safe, +with credential vending technology, Gravitino server will help users to get the temporary tokens for +authentication, which will significantly simplify the client side configurations and centralized the +authentications. + +In Gravitino 0.7.0, we introduce a framework to support Credential vending, also add S3 and GCS +token support. Besides, we integrated this framework in Gravitino Iceberg REST catalog service. So +users can smoothly access the Iceberg table on S3 and GCS with authentication. + +But, this is just the first step of credential vending, we will add more integrations with Gravitino, +like fileset support, connector support, etc, in the next release. + +For the details of credential vending, please check the issue [#4398](https://github.com/apache/gravitino/issues/4398) +and the [design document](https://docs.google.com/document/d/1fovK0ylSmI45ynrCPcnRZqzyPDn7DRNb_ExdbjVPq0k/edit?tab=t.0#heading=h.1cts6muko2j1). + +## Unified access control improvements + +In Gravitino 0.6.0, we introduced the alpha version of unified access control with Apache Ranger +support ([here](https://gravitino.apache.org/blog/gravitino-0-6-0-release-notes#introducing-the-unified-rbac-model-for-gravitino)), +but this feature still needs to improve a lot. In the version 0.7.0, we add lots of improvements +and fix bunches of bugs to make this access control end to end workable. Now, with the release of +0.7.0, the Gravitino unified access control can work well with Spark and Ranger to secure the table +from end to end. To see what we have fixed, please check out our issue [#4615](https://github.com/apache/gravitino/issues/4615). +You can also try our [playground](https://github.com/apache/gravitino-playground) to experience the +unified access control feature. + +## Centralized audit log support + +Thanks to the community, Gravitino now supports centralized audit log. With this feature enabled, +users can get the audit log in the centralized place, no matter they’re accessing tables or filesets from various sources. + +Gravitino’s audit log framework also supports to plugin different formatter and writer, so users +can implement their own log format and output destinations. + +Please see the issues [#4887](https://github.com/apache/gravitino/issues/4887) +and [#4021](https://github.com/apache/gravitino/issues/4021) to know more about Gravitino’s +centralized audit log. + +## New data sources support + +As a unified data catalog, the community always pursue the target of adding more data sources. In +this version, Gravitino adds two new data sources, one is [Apache Hudi](https://hudi.apache.org/), +another is [OceanBase](https://github.com/oceanbase/oceanbase). You can now use Gravitino to +manipulate Hudi and Oceanbase metadata in a unified manner. + +## Various core features + +Apart from the features listed above, this version also improves a lot in its core, here lists several important features: + +* Add PostgreSQL support for storage backend [#4101](https://github.com/apache/gravitino/issues/4101). + Gravitino already supports using MySQL, H2 as its backend metadata storage. In 0.7.0, the + community adds the PostgreSQL support to enlarge its adoption. +* Unify the catalog and metalake drop behavior [#5031](https://github.com/apache/gravitino/issues/5031). + In the previous version, we didn’t enforce the behavior of catalog and metadata drop operation. + In this version, we redefine its behavior and make it much safer to use. +* Manage the column in Gravitino [#4493](https://github.com/apache/gravitino/issues/4493). + In 0.7.0, we introduce the column entity in Gravitino, and can be managed by Gravitino versionly. + With this feature introduced, Gravitino now can support tagging on columns, and in future it + can support column level operations. +* Add event listener for Iceberg REST catalog server [#5204](https://github.com/apache/gravitino/issues/5204) + and support pre-event for event listener [#5112](https://github.com/apache/gravitino/issues/5112). + +## Other notable enhancements + +### Gravitino core + +* Supporting storing column metadata in Gravitino [#4493](https://github.com/apache/gravitino/issues/4493). +* Support pre-event for Gravitino [#5049](https://github.com/apache/gravitino/issues/5049). +* Unify drop metalake and catalog behavior [#5031](https://github.com/apache/gravitino/issues/5031). +* Add credential vending support in Gravitino [#4398](https://github.com/apache/gravitino/issues/4398). +* Support audit log in Gravitino [#4887](https://github.com/apache/gravitino/issues/4887). +* Shrink the package size of Gravitino [#4513](https://github.com/apache/gravitino/issues/4513). + +### Iceberg REST catalog server + +* Add credential vending for Iceberg REST server. [#4993](https://github.com/apache/gravitino/issues/4993). +* Add event listener for Iceberg REST server [#5204](https://github.com/apache/gravitino/issues/5204). +* Support pre-event for event listener [#5112](https://github.com/apache/gravitino/issues/5112). + +### Catalog related + +* Add OSS support for fileset catalog [#5173](https://github.com/apache/gravitino/issues/5173). +* Add GCS support for fileset catalog [#5074](https://github.com/apache/gravitino/issues/5074). +* Add S3 support for fileset catalog [#3379](https://github.com/apache/gravitino/issues/3379). +* Add pluggable storage support fro fileset catalog [#5019](https://github.com/apache/gravitino/issues/5019). +* Add S3 support for Paimon catalog [#4938](https://github.com/apache/gravitino/issues/4938). +* Add catalog support for Hudi [#4306](https://github.com/apache/gravitino/issues/4306). +* Add catalog support for OceanBase[#4848](https://github.com/apache/gravitino/issues/4848). + +### API and client + +* Add S3 fileset support for Python GVFS client [#5188](https://github.com/apache/gravitino/issues/5188). +* Add GCS fileset support for Python GVFS client [#5139](https://github.com/apache/gravitino/issues/5139). +* Add OSS fileset support for Python GVFS client [#5221](https://github.com/apache/gravitino/issues/5221). +* Supports unified auditing of Fileset metadata and data operations [#4021](https://github.com/apache/gravitino/issues/4021). +* Support OAuth2 in Python GVFS [#3758](https://github.com/apache/gravitino/issues/3758). + +### UI + +* Add UI support for operating fileset [#5167](https://github.com/apache/gravitino/issues/5167). +* Add UI support for operating schema [#5140](https://github.com/apache/gravitino/issues/5140). + +All the resolved issues targeting to the 0.7.0 release can be seen at https://github.com/apache/gravitino/issues?q=is%3Aissue+is%3Aclosed+label%3A0.7.0+. + +## Overall + +Apache Gravitino 0.7.0 is the second ASF release, this version add bunch of new features, we would +like to show appreciation to the Gravitino community for their continued support and valuable +contributions. Thanks to the feedback of our users, we are able to continue to innovate and build, +so thanks to all those reading this! + +To explore Gravitino 0.7.0 release, please check [the documentation](https://gravitino.apache.org/docs/0.7.0-incubating). +Your feedback is invaluable to the community and the project. + +## Credits + +This release acknowledges the hard work and dedication of all contributors who have helped make this release possible. + +[@FANNG1](https://github.com/FANNG1) +[@LauraXia123](https://github.com/LauraXia123) +[@LindaSummer](https://github.com/LindaSummer) +[@LiuQhahah](https://github.com/LiuQhahah) +[@Naresh-kumar-Thodupunoori](https://github.com/Naresh-kumar-Thodupunoori) +[@SeanAverS](https://github.com/SeanAverS) +[@caican00](https://github.com/caican00) +[@coolderli](https://github.com/coolderli) +[@diqiu50](https://github.com/diqiu50) +[@featherchen](https://github.com/featherchen) +[@hanwxx](https://github.com/hanwxx) +[@jerqi](https://github.com/jerqi) +[@jerryshao](https://github.com/jerryshao) +[@jingjia88](https://github.com/jingjia88) +[@justinmclean](https://github.com/justinmclean) +[@koonchen](https://github.com/koonchen) +[@lsyulong](https://github.com/lsyulong) +[@lw-yang](https://github.com/lw-yang) +[@mchades](https://github.com/mchades) +[@noidname01](https://github.com/noidname01) +[@puchengy](https://github.com/puchengy) +[@shaofengshi](https://github.com/shaofengshi) +[@theoryxu](https://github.com/theoryxu) +[@xiaozcy](https://github.com/xiaozcy) +[@xloya](https://github.com/xloya) +[@xunliu](https://github.com/xunliu) +[@yangyuxia](https://github.com/yangyuxia) +[@yaoderek](https://github.com/yaoderek) +[@yuanoOo](https://github.com/yuanoOo) +[@yuqi1129](https://github.com/yuqi1129) + diff --git a/src/pages/downloads.mdx b/src/pages/downloads.mdx index 15e5775bb..b85c1b550 100644 --- a/src/pages/downloads.mdx +++ b/src/pages/downloads.mdx @@ -1,5 +1,18 @@ ## Apache Gravitino Downloads +### 0.7.0-incubating + +[Gravitino source](https://www.apache.org/dyn/closer.lua/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-src.tar.gz) [ASC](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-src.tar.gz.asc) [SHA](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-src.tar.gz.sha512) + +[Gravitino binary](https://www.apache.org/dyn/closer.lua/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-bin.tar.gz) [ASC](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-bin.tar.gz.asc) [SHA](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-0.7.0-incubating-bin.tar.gz.sha512) + +[Gravitino Iceberg REST server](https://www.apache.org/dyn/closer.lua/incubator/gravitino/0.7.0-incubating/gravitino-iceberg-rest-server-0.7.0-incubating-bin.tar.gz) [ASC](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-iceberg-rest-server-0.7.0-incubating-bin.tar.gz.asc) [SHA](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-iceberg-rest-server-0.7.0-incubating-bin.tar.gz.sha512) + +[Gravitino Trino connector](https://www.apache.org/dyn/closer.lua/incubator/gravitino/0.7.0-incubating/gravitino-trino-connector-0.7.0-incubating.tar.gz) [ASC](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-trino-connector-0.7.0-incubating.tar.gz.asc) [SHA](https://downloads.apache.org/incubator/gravitino/0.7.0-incubating/gravitino-trino-connector-0.7.0-incubating.tar.gz.sha512) + +The [KEYS file](https://downloads.apache.org/incubator/gravitino/KEYS) and ASC files can used to verify the release, please follow the intructions on the download page to verify the release before you use it. + + ### Gravitino 0.6.1-incubating [Gravitino source](https://www.apache.org/dyn/closer.lua/incubator/gravitino/0.6.1-incubating/gravitino-0.6.1-incubating-src.tar.gz) [ASC](https://downloads.apache.org/incubator/gravitino/0.6.1-incubating/gravitino-0.6.1-incubating-src.tar.gz.asc) [SHA](https://downloads.apache.org/incubator/gravitino/0.6.1-incubating/gravitino-0.6.1-incubating-src.tar.gz.sha512)