v0.5.1
Gravitino Release 0.5.1
Gravitino 0.5.1 is a stable release based on branch-0.5 that contains bug fixes and introduces some core features for Trino and Spark connectors and security. Additionally, the Python client is ready to use in version 0.5.1.
Core Features
-
Apache Spark Connector Enhancement: Supports multiple Spark (3.3, 3.4, 3.5) and Scala (2.12, 2.13) versions. Introduces several advanced features for Iceberg support like row-level operations, time travel query, procedures, etc.
-
Trino Connector Improvement: Improve the Gravitino trino connector to use Trino's dynamic catalog framework to better support queries in the distributed mode.
-
Security Improvement Supports the condition of privileges, support authentication between Gravitino and Hive, Hadoop catalog.
-
Python client Fixed several critical bugs, enhancing the usability of the Gravitino Python client for real environments.
Gravitino Core
- Supports time sliding window to record metrics #3152
- The semantics of the delete operation for relational storage should be aligned with kv storage #3092
- Provide specific classLoader for Partition operation #2997
- Support preassigned partitions when creating partitioned table #2266
- Remove log4j from client and client java runtime module #3649
- Improve compatibility issues with data type serde #3598
Security
- Privilege should have deny mechanism #3342
- Authentication between the Hadoop catalog and Gravitino. #3337
- Support Kerberos client authentication in gvfs #2711
- Add Hive user authentication end-to-end test #3489
- Add IT to test kerberosized HDFS #3432
- Fix the Hive catalog bug In multiple Kerberized HMS #3295
Catalogs
Iceberg
- Support Iceberg RestCatalog in Gravitino #3149
- Add reserved properties to Table Properties when load an Iceberg table #2962
- Bucket and truncate are missing width params in Gravitino SortOrder #2921
Doris
- Support creating Doris table with partition #3347
- Filter system database when list database for Doris catalogs #3275
MySQL
- jdbc-mysql catalogs failed on mysql 8.x driver #3269
- MysqlTypeConverter cannot handle the datetime data type #1761
Kafka
- Adding a new Apache Kafka catalog works but times out listing topics #3249
- Drop kafka catalog maybe failed after altering #3673
Trino Connector
- Gravitino Trino connector loads the catalog using Trino's dynamic catalog feature #3524
- Support automatically loading catalog to Trino by using CREATE CATALOG command #3601
- Support create Internal connector like hive/iceberg/mysql/pg/memory #3587
- [trino-connector] Support create Gravitino catalog by using Trino CREATE CATALOG command #3526
Spark Connector
- Support multiple Spark and Scala versions #1574
- Remove iceberg runtime from spark connector jar #3396
- Support passing catalog properties to spark connector #3292
- Support iceberg metadataColumns #2587
- Support row-level operations to iceberg Table #2543
- Support Iceberg time travel in SQL queries #3264
- Support Iceberg RestCatalog with HiveCatalog backend in spark-connector #3193
- Support Iceberg Spark Procedure #3186
- Alter table comment though table properties failed #3137
- Spark connector create hive table failed for using hive #3129
Python Client
- Correct type specification of create_fileset function in fileset_catalog.py #3508
- When calling hash() function in name_identifier with python client throw an exception #3521
- Add test coverage tool for client-python #3517
- Add Pylint Rules for client-python naming rules #3205
- Add auto formatter for client-python #3206
- Add Pylint Rules for client-python #3203
- Unspecified pylint version in client-python #3558
UI
- Gravitino icon doesn't show on UI #3413
- UI: the font family of gravitino title text in production mode does not match with dev mode #3386
- With only one metalake UI presents user with a choice #3285
- Gravitino UI when creating a new fileset catalog it seems to be missing location property #3257
Build and Others
- Unable to build Gravitino in Linux environment due to JAVA problem #3336
- ./gradlew test -PskipITs failed in python-client test #3319
- TrinoContainer#checkSyncCatalogFromGravitino will always return false #3237
Limitation and known issues
- The query in Trino connector for 0.5.1 has slower performance than version 0.5.0 #3738
Credits
This release acknowledges the hard work and dedication of all contributors who have helped make this release possible.
@caican00 @ch3yne @danhuawang @diqiu50 @FANNG1 @featherchen @jerqi @kalencaya @LauraXia123 @lw-yang @MukarramHaq @mchades @noidname01 @qqqttt123 @SteNicholas @TEOTEO520 @theoryxu @unknowntpo @xloya @xiaozcy @xunliu @yijhenlin @yuqi1129 @zhoukangcn