Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove legacy Hive readers and writers #18241

Merged
merged 12 commits into from
Sep 6, 2023
Merged

Conversation

electrum
Copy link
Member

@electrum electrum commented Jul 12, 2023

Description

This PR will be merged once we have confidence in the new readers and writers.

Release notes

(x) Release notes are required, with the following suggested text:

# Hive
* Remove legacy Hive readers. The `*_native_reader_enabled` session properties and the
  `*.native-reader.enabled` configuration properties have been removed. ({issue}`18241`)
* Remove legacy Hive writers. The `*_native_writer_enabled` session properties and the
  `*.native-writer.enabled` configuration properties have been removed. ({issue}`18241`)
* Remove support for S3 Select. The `s3_select_pushdown_enabled` session property
  and the `hive.s3select*` configuration properties have been removed. ({issue}`18241`)
* Optimized symlink listing is always enabled. The `optimize_symlink_listing` session property
  and `hive.optimize-symlink-listing` configuration property have been removed. ({issue}`18241`)

@cla-bot cla-bot bot added the cla-signed label Jul 12, 2023
@github-actions github-actions bot added tests:hive hudi Hudi connector hive Hive connector labels Jul 12, 2023
@electrum electrum force-pushed the hive-remove branch 2 times, most recently from df7e9d5 to 7de468b Compare July 12, 2023 08:09
@github-actions github-actions bot added the docs label Jul 12, 2023
@electrum electrum force-pushed the hive-remove branch 5 times, most recently from 6e00005 to b3dba79 Compare July 16, 2023 20:40
@electrum electrum force-pushed the hive-remove branch 14 times, most recently from 5bcb6cf to 24cbba5 Compare July 27, 2023 23:29
@github-actions github-actions bot added iceberg Iceberg connector delta-lake Delta Lake connector labels Jul 27, 2023
@electrum electrum force-pushed the hive-remove branch 3 times, most recently from ecd6431 to 490b88e Compare August 22, 2023 16:50
@electrum electrum merged commit bcbfd39 into trinodb:master Sep 6, 2023
82 checks passed
@electrum electrum deleted the hive-remove branch September 6, 2023 06:21
@github-actions github-actions bot added this to the 427 milestone Sep 6, 2023
@awakeljw
Copy link

Ihave two questions, Will the s3 select pushdown feature come back ? Will s3 select pushdown feature support parquet file pushdown?

@electrum
Copy link
Member Author

electrum commented Nov 6, 2023

@awakeljw We have no plans to reimplement S3 Select. For Parquet, the results are returned as CSV or JSON, which is why it was never implemented in Trino. The feature introduces significant complexity in Trino and only benefits textfile formats. Additionally, the pricing model for S3 Select seems to put it at a disadvantage compared to simply increasing compute on the Trino side. Thus, we don't believe it is worth the complexity and maintenance burden.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector docs hive Hive connector hudi Hudi connector iceberg Iceberg connector release-notes
Development

Successfully merging this pull request may close these issues.

4 participants