Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iceberg write.parquet.compression-codec being set even if file-format is not parquet #20401

Open
oneonestar opened this issue Jan 17, 2024 · 0 comments
Labels
bug Something isn't working iceberg Iceberg connector

Comments

@oneonestar
Copy link
Member

oneonestar commented Jan 17, 2024

In Trino 436 (Iceberg 1.4.3), write.parquet.compression-codec property is also being set even if the file-format is not parquet. This doesn't happen in Trino 428 (Iceberg 1.3.1), not sure about Trino 431 to Trino 435 (Iceberg 1.4.1)
Is this an intended behavior?

trino> CREATE TABLE test.property_test (c1 integer) WITH (format = 'ORC');
CREATE TABLE
trino> SELECT * FROM test."property_test$properties";
               key               | value
---------------------------------+-------
 write.format.default            | ORC
 write.parquet.compression-codec | zstd
(2 rows)

trino> CREATE TABLE test.property_test (c1 integer) WITH (format = 'AVRO');
CREATE TABLE
trino> SELECT * FROM test."property_test$properties";
               key               | value
---------------------------------+-------
 write.format.default            | AVRO
 write.parquet.compression-codec | zstd
(2 rows)

Ref:
apache/iceberg#8593 (comment)
https://github.com/trinodb/trino/pull/19434/files#diff-f5cd96d1ec33fae497df9613afcefff034f01c99adf961e927ccd76e637c7b81R4707

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working iceberg Iceberg connector
Development

No branches or pull requests

2 participants