Release notes for 356

[martint]

Ashhar Hasan

• all checked
• 04-22 23:56:38 b72fbf7 Migrate Accumulo tests to use BaseConnectorTest
• 04-22 23:56:38 c055755 Make exception messages consistent
• 04-22 23:56:38 d4ae79b Fix typo in the assertion
• 04-24 22:27:00 accc294 Fix handling of non-lowercase MongoDB views
• 04-25 07:09:59 15babb5 Migrate Cassandra tests to use BaseConnectorTest
• 04-25 07:09:59 205e79d Test against Cassandra 3.11.10

Dain Sundstrom

• all checked
• 04-21 11:47:16 8f196a3 Add query info url template
• 04-22 13:47:17 666daea Fix link to glossary

David Phillips

• all checked
• 04-19 11:09:55 0bbc673 Remove unsed supportsMetadataDelete in HiveMetadata
• 04-19 11:09:55 a9478d0 Use ACID semantics for whole partition deletes
• 04-19 15:55:05 208c9fd Support Hive UPDATE with original files
• 04-22 11:36:23 0b9e2b7 Added new config property for JDBC extern auth token cache
• 04-22 11:39:00 08273cf Adding JDBC classname to docs
• 04-22 11:49:01 27c8b4c Add glossary to Trino
• 04-22 12:09:48 510f7fb Add minimal TLS/HTTPS section
• 04-22 12:13:14 f69e52b Add documentation for Elasticsearch password authentication
• 04-22 12:39:24 6ac0e86 New writer oriented rework of docs/README

Grzegorz Kokosiński

• all checked
• 04-09 03:03:11 ceb2d1b Add cached token option to jdbc externalAuthentication
• 04-09 03:03:11 fe5335b Extract class MockRedirectHandler
• 04-09 04:24:21 92ee334 Add support for optional Kafka SSL
• 04-12 03:58:41 6e9d28b Allow to configure eventListener in DistributedQueryRunner
• 04-13 03:40:25 fe608f2 Make JdbcMetadata::getSchemaTableName public
• 04-13 12:09:05 b940268 Ignore not existing logs when collecting log files
• 04-14 01:34:38 5435fc9 Extract security config from Kafka config
• 04-14 01:34:38 c0dbffe Remove redundant module binding
• 04-14 01:34:38 c9b64f0 Use security protocol enum from Kafka library
• 04-14 12:03:27 184284c Small cleanup in CachingJdbcClient
• 04-14 12:03:27 e27fdf2 Use same user key when no impersonation is used
• 04-16 11:24:31 cdf5261 Allow to customize JdbcIdentityCacheMapping for CredentialProvider
• 04-16 11:25:29 af9feda Cancel PostgreSQL remote query on thread interruption
• 04-16 11:25:29 c35b46c Add tests for Phoenix plugins
• 04-19 02:10:39 76d585b Add recommendations around language injection
• 04-22 10:05:01 8d4710b Allow JDBC connector to invalidate statistics cache
• 04-22 13:21:58 49c2e5e Handle race condition when files are deleted externally
• 04-26 12:03:43 c647c31 Add external/SSO authentication

Karol Sobczak

• all checked
• 04-09 02:07:29 ca97f61 Add that dynamic filtering is default
• 04-09 05:16:59 27ffdb9 Disable join spill for grouped execution
• 04-09 05:16:59 85f4cef Fix join spill race condition
• 04-09 05:16:59 e8d7d17 Remove unused exception
• 04-09 06:48:55 987b1dc Require that materialized view has owner
• 04-09 12:48:59 38960f5 Make SHOW CREATE checks consistent with how user sees objects
• 04-09 12:48:59 6f5553f Test analyzer name resolution order
• 04-09 12:48:59 9cf7da2 Make materialized view check first
• 04-13 02:51:27 7cea728 Refer to table-scan-node-partitioning-min-bucket-to-task-ratio in doc
• 04-13 03:08:49 ee001f8 Collect build side before local dynamic filters are consumed
• 04-14 07:36:50 6968386 Add a flag for HttpRemoteTask to avoid eager but unnecessary sendUpdate
• 04-15 03:17:38 c7c4550 Extract OrderingScheme#toLocalProperties
• 04-16 01:46:56 21ec7d4 Add operator cache and coercion cache in metadata manager
• 04-16 01:46:56 ca64719 Add large IN predicate test in planner benchmark
• 04-19 01:10:08 f652531 Fix Join optimization rule
• 04-19 06:31:27 231853b Enhance view owner validation
• 04-19 06:32:18 60681db Revert "Enhance view owner validation"
• 04-19 06:32:37 f4dcf75 Enhance view owner validation
• 04-20 01:34:51 c903ac0 Speed up varcharToCharSaturatedFloorCast
• 04-20 01:39:01 1cb9b19 Add Connector#getMaterializedViewProperties
• 04-20 01:39:01 3504fcc Use MaterializedViewPropertyManager in CreateMaterializedViewTask
• 04-20 01:39:01 43ee865 Add getMaterializedViewProperties to hive
• 04-20 01:39:01 db60c21 Add system.metadata.materialized_view_properties
• 04-20 01:39:01 e1262b2 Show properties in CREATE MATERIALIZED VIEW
• 04-21 03:39:50 a79ec43 Add static imports
• 04-21 03:39:50 c8e99e3 Add query planning time limit
• 04-22 03:26:43 297c5c6 update the documentation
• 04-22 03:37:44 7c7b3c9 Handle boolean reference in DomainTranslator
• 04-22 06:24:06 ccf48a9 Add checkCanDropMaterializedView access control
• 04-22 06:24:06 e571449 Add checkCanCreateMaterializedView access control
• 04-23 13:56:50 f907ff2 Add varcharToVarcharSaturatedFloorCast
• 04-23 14:00:51 a0a15e7 Add dropMaterializedView to TestingMetadata
• 04-26 04:47:36 531bbd0 Fix population of schema for iceberg getMaterializedView
• 04-26 04:58:35 860a003 Update ConnectorMaterializedViewDefinition
• 04-26 04:58:35 95afc07 Decouple view serde from ConnectorMaterializedViewDefinition
• 04-26 05:21:50 396534d Initialize planner beans lazily
• 04-26 05:21:50 441a74b Expose export/unexport method
• 04-26 05:21:50 91acb92 Make PlanOptimizers extendable
• 04-26 05:21:50 bfa3816 Add IterativeOptimizer#getRules

Martin Traverso

• all checked
• 04-09 12:15:36 b357388 Add Trino client protocol documentation
• 04-19 15:59:58 6476c13 Fix ActiveCount metric name in JMX documentation
• 04-19 16:27:02 52a84f0 Factor out join planning
• 04-19 16:27:02 b1af688 Encapsulate subquery analysis
• 04-20 17:30:57 48cb3b2 Add Soundex function
• 04-21 08:42:20 69926f8 Return varchar(4) instead of varchar for soundex
• 04-21 08:42:20 7dd7a82 Use static Soundex instance
• 04-23 10:39:29 0d443e4 Correct single-char typo
• 04-23 10:40:37 4b9c9d0 Remove incorrect test
• 04-25 09:36:22 ee0f9aa Add link to function in list page
• 04-26 10:51:26 96b39d4 Clean up WindowPartition class
• 04-26 13:28:58 5550d92 Add bitwise example
• 04-26 15:53:17 e41d9af Support row pattern matching via MATCH_RECOGNIZE clause
• 04-26 15:53:17 f3cda33 Rename '?' to QUESTION_MARK in grammar
• 04-26 15:54:04 e72b452 Document MATCH_RECOGNIZE

Piotr Findeisen

• all checked
• 04-09 03:08:36 ed98f24 Add missing functions by name
• 04-09 03:47:59 b39168f Add method to list all objects matching a given prefix
• 04-09 04:01:36 1f3955a Add a doc comment and error message in ParquetPageSource
• 04-09 04:01:36 5f3381f Replace use of isPresent with map and orElseGet in ParquetPageSource
• 04-09 08:19:01 8bac015 Do case insensitive comparison between dereferenced fields and internal ORC field names
• 04-09 15:20:27 a842287 Fix logical merge conflict
• 04-11 13:28:38 b2888b4 Add debug listing to troubleshoot flaky build
• 04-12 02:30:56 4f8221e Fix SNAPSHOT ISOLATION detection in SQL Server
• 04-12 08:45:36 0dbb2b5 Extend Parquet reader system to add an optional row-index column
• 04-12 08:45:36 5619084 Merge loops in ParquetPageSourceFactory
• 04-12 08:45:36 bd66e4a Remove unnecessary parenthesis
• 04-13 00:09:46 692d0fd Add kill switch for SQL Server transaction isolation
• 04-13 00:16:35 41a36d4 Remove bogus option for unsupported-type-handling from docs
• 04-13 02:48:45 05a21c5 Use default writer version in a test
• 04-13 02:48:45 4407c99 Reorder OrcWriterOptions with* methods
• 04-13 02:48:45 7098233 Move useLegacyVersion to OrcWriterOptions
• 04-13 02:48:45 cc4db41 Add OrcWriterOptions builder
• 04-13 02:48:45 e033a95 Use Trino writer identification when writing ORC
• 04-13 03:04:41 3ab3ca9 Format JdbcSortItem in EXPLAIN output
• 04-15 06:49:42 d28bc9f Upgrade Coral to 1.0.42
• 04-15 07:55:20 bfe9d98 Adjust pattern for matching sortItems in explain plan
• 04-16 03:49:42 fea0e2a Disable Phoenix tests due to GHA setup issue
• 04-16 03:55:47 ed91910 Update SPI docs
• 04-16 04:41:39 bc89dee Mark TestHiveViews.testRichSqlSyntax as @flaky
• 04-16 04:48:27 31b4ab0 Add SQL server type mapping tests
• 04-16 04:48:27 5288998 Make varbinaryWriteFunction public
• 04-16 04:48:27 c40fc7c Rename (dummy) TestSqlServerTypeMapping
• 04-16 04:48:27 f78b318 Rename and refactor SQL server type mapping tests
• 04-16 14:36:19 4cf9a95 Supports TIMESTAMP(9) in trino-base-jdbc
• 04-16 14:48:50 714e70e Fractional number double test case
• 04-19 13:21:32 2455684 Simplify condition
• 04-20 00:15:27 b9e0279 Convert oracle char unicode tests to SqlDataTypeTest
• 04-20 00:15:27 c6111d2 Fix oracle incorrect char padding
• 04-20 13:13:46 e70726a Remove obsolete Oracle testing DataTypes
• 04-21 04:02:12 62b7756 Support dates<1583 for SQL Server connector
• 04-21 04:02:12 6773b9c Add testSqlServerDateUnsupported for > 4 digit years
• 04-21 04:02:12 df3c9bd Remove duplicate test case (covered in testTrinoLongChar)
• 04-21 05:42:44 d56b58c Raise minimum PostgreSQL version to 9.6
• 04-22 05:21:05 38b5e81 Enable Phoenix tests
• 04-23 08:27:05 a0dd45b Allow running HiveQueryRunner from IDE without any setup
• 04-23 13:33:29 61e7733 Skip pushFilterIntoTableScan in AddExchanges when no Flter above TableScan
• 04-23 23:44:08 caff4a9 Disable predicate pushdown in AddExchanges when toggled off
• 04-26 00:00:05 49f2bb7 Rename tests to reveal difference
• 04-26 00:00:05 5ad6240 Group related test assertions together
• 04-26 00:00:05 6c97172 Improve formatting of an exception message
• 04-26 00:02:17 1f2854f Annotate sql parameter with @language
• 04-26 08:09:20 5b20173 Make TupleDomain.toString(session) consistent with toString
• 04-26 08:09:20 63d8614 Make SortedRangeSet.toString(session) consistent with toString
• 04-26 08:09:20 93e393c Fix expected Domain type
• 04-26 08:09:20 acb22fe Always include constraint in ThriftIndexHandle
• 04-26 08:09:20 e0e553c Make TupleDomain and Domain toString more useful
• 04-26 13:45:03 83f711a Extensible hive metadata
• 04-26 13:45:03 8b8fc26 Remove redundant suppression

Yuya Ebihara

• all checked
• 04-18 06:03:33 212e1bb Fasten listing tables in BigQuery connector
• 04-19 08:01:24 269e9de Add support for CREATE and DROP SCHEMA in BigQuery
• 04-19 18:31:22 c682a1d Fix NPE when reading empty timestamp in Cassandra

Łukasz Osipiuk

• all checked
• 04-13 23:38:52 01cf2e8 Rename test methods
• 04-13 23:38:52 98fbb58 Push PARTIAL TopN into table scan
• 04-14 01:08:19 5c8ae0a Use .trino as hidden files prefix in file hive metastore
• 04-15 15:10:40 7cc154c Validate ORC ACID version base on data file metadata as a fallback
• 04-15 15:10:40 920be76 Add clarifying comment
• 04-16 03:57:59 270ef1c Change test query to operate on BIGINT values
• 04-16 05:13:15 f304906 Disable UPDATE for Hive insert-only tables
• 04-16 05:13:15 f8825cc Test row DELETE vs non-transaction/insert-only Hive tables
• 04-19 04:47:21 ebe0db7 Update testcontainers to 1.15.3
• 04-20 02:48:02 5ed22be Allow setting connection proxy for ADL connection
• 04-20 02:48:02 8942fdd Update hadoop-apache dependency
• 04-21 01:10:16 6617007 Use proper symbols for computing join stats for pushdown
• 04-26 01:08:48 75ea43d Annotate test as @flaky

[kokosing]

JDBC
---
* Add `externalAuthenticationTokenCache=MEMORY` property so Oauth2 credentials are cached in memory. Thanks to this user can avoid unnecessary authentication flow. ({issue}`7309`)

#7309

[kokosing]

Kafka
---
* Add support for SSL security protocol. ({issue}`6929`

#6929

[sopel39]

General:
* Fix join spill deadlock. ({issue}`7455`)

#7455

[sopel39]

SPI:
* Require that ``ConnectorMaterializedViewDefinition`` provides a view owner. ({issue}`7489`)

#7489

[findepi]

Hive
* Fix incorrect results when non-lowercase field dereference is pushed into the ORC reader,
  if ORC file schema has nested fields with a different case. ({issue}`7350`)

#7350

[findepi]

SQL Server connector
* Fix query failure when snapshot isolation (`ALLOW_SNAPSHOT_ISOLATION`) is disabled in target SQL Server database, but
  but `READ_COMMITTED_SNAPSHOT` is still enabled. ({issue}`7548`)

#7548 #7555

[losipiuk]

General
* improve performance of `ORDER BY … LIMIT` queries over a `LEFT JOIN`. ({issue}`7028`)

#7028 #7563

[sopel39]

General
* Reduce coordinator network overhead when scheduling queries. ({issue}`7351`)

#7351

[kokosing]

JDBC connectors
* Improve metadata caching hit rate by sharing cache entries between different session users. ({issue}`7039`)

#7039

[losipiuk]

Hive
* Allow reading from ORC ACID transactional tables when `_orc_acid_version` metadata file is missing. ({issue}`7579`)

#7579

[sopel39]

General
* Improve planning time for queries with large `IN` predicates. ({issue}`7556`)

#7556

[kokosing]

PostgreSQL
* Cancel PostgreSQL remote query on when Trino query is cancelled. ({issue}`7306`)

#7306

[ebyhr]

BigQuery
* Improve performance for listing tables when case insensitive matching (`bigquery.case-insensitive-name-matching`) is enabled. ({issue}`7628`)

#7628

[sopel39]

General
* Fix invalid query result for queries with full join which has scalar (left or right) input ({issue}`7629`)

#7629

[ebyhr]

BigQuery
* Add support for `CREATE SCHEMA` and `DROP SCHEMA` statements. ({issue}`7543`)

#7543

[ebyhr]

Cassandra
* Fix `NullPointerException` when reading an empty timestamp value. ({issue}`7433`)

#7433 #7544

[sopel39]

SPI
* Add `Connector#getMaterializedViewPropertyManager` for specifying materialized view properties ({issue}`7615`)

#7615

[sopel39]

General
* Introduce `information_schema.materialized_view_properties` table for listing available materialized view properties. ({issue}`7615`)

#7615

[losipiuk]

Hive
* Allow specifying Azure ADL connection proxy via `hive.azure.adl-proxy-host` config property. ({issue}`7509`)

#7509

[sopel39]

General
* Introduce planning time limit which can be specified using `query.max-planning-time` config property ({issue}`7213`)
* Improve cancellation of queries during planning phase ({issue}`7213`)

#7213

[sopel39]

General
* Improve performance of queries with predicates on boolean columns ({issue}`7263`)

#7263

[sopel39]

General
* Add dedicated access control for creating and dropping materialized views. ({issue}`7645`)

#7645

[sopel39]

SPI
* Add `ConnectorAccessControl#checkCanCreateMaterializedView` and
  `ConnectorAccessControl#checkCanDropMaterializedView` for authorizing
  creation and removal of materialized views. ({issue}`7645`)

#7645

[sopel39]

General
* Improve performance of queries that contain join on `VARCHAR` keys of different length. ({issue}`7644`)

#7644

[hashhar]

MongoDB
* Fix handling of non-lowercase MongoDB views. ({issue}`7491`)

#7491, #7546

[sopel39]

Iceberg:
* Fix querying materialized views that were created using session catalog and schema. ({issue}`7711`)

#7711

[sopel39]

SPI:
* Allow for materialized view to return storage table in different catalog and schema. ({issue}`7638`)

#7638

[martint]

* Add support for {doc}`/sql/match-recognize`. ({issue}`6111`)

#6111

[martint]

* Add {func}`soundex` function. ({issue}`4022`)

#4022

[phd3]

* Iceberg Connector:
- Show Iceberg tables created by other engines in ``SHOW TABLES`` output. ({issue}`1592`)

#1592, #7642

[sopel39]

General
* Improve performance of queries with `late_materialization` enabled. ({issue}`7695`)

#7695

[findepi]

Hive, Iceberg
* Improve performance when reading dictionary-encoded Parquet files. ({issue}`7754`)

#7754

[findepi]

SQL Server
* Fix reading `date` values before 1583-10-14. ({issue}`7634`)

#7634

[findepi]

PostgreSQL
* Discontinue support for PostgreSQL 9.5.

[dain]

General
* Add configuration for the query info sent to clients.  This allows clients to be directed to a different location for information about their query.  This is set using the `query.info-url-template` property .  ({issue}`7678`)

#7678

[sopel39]

SPI
* Add `ConnectorAccessControl#checkCanRefreshMaterializedView` for authorizing
  refresh of materialized views. ({issue}`7707`)

File based access control
* Materialized views require `UPDATE` privilege to be refreshed. ({issue}`7707`)

General
* Add dedicated access control for refreshing materialized views.
  Insert privilege on storage table is no longer required. ({issue}`7707`)

#7707

[ebyhr]

MySQL
* Exclude an internal `sys` schema from schema listings. ({issue}`6337`)

#6337

[martint]

* Fix failure when quantified comparison expressions contain scalar subqueries. ({issue}`7792`)

#7792

[phd3]

# Iceberg Connector
* Improve query planning through table metadata caching. ({issue}`7336`) 

#7336, #7367

[kokosing]

* Fix bug when providing multiple scope values for `http-server.authentication.oauth2.scopes`. ({issue}`7706`)

#7706

[martint]

• Fix bug when providing multiple scope values for http-server.authentication.oauth2.scopes. ({issue}7706)

@kokosing, what's the bug? Is it an authentication failure? Incorrect authentication?

[kokosing]

what's the bug? Is it an authentication failure? Incorrect authentication?

It was causing authentication failure due the misconfiguration as scopes in OAuth2 are separate with space. We used to use , and so it was considered as single scope. See:

com.github.scribejava.core.model.OAuth2AccessTokenErrorResponse: {"error":"invalid_scope","error_description":"The requested scope is invalid, unknown, or malformed. The OAuth 2.0 Client is not allowed to request scope 'openid,offline'."}
	at com.github.scribejava.core.extractors.OAuth2AccessTokenJsonExtractor.generateError(OAuth2AccessTokenJsonExtractor.java:72)
	at com.github.scribejava.core.extractors.OAuth2AccessTokenJsonExtractor.extract(OAuth2AccessTokenJsonExtractor.java:40)
	at com.github.scribejava.core.extractors.OAuth2AccessTokenJsonExtractor.extract(OAuth2AccessTokenJsonExtractor.java:18)
	at com.github.scribejava.core.oauth.OAuth20Service.sendAccessTokenRequestSync(OAuth20Service.java:53)
	at com.github.scribejava.core.oauth.OAuth20Service.getAccessTokenClientCredentialsGrant(OAuth20Service.java:288)
	at io.trino.server.security.oauth2.TestDynamicCallbackOAuth2Service.testMultipleScopes(TestDynamicCallbackOAuth2Service.java:90)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:104)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:645)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:851)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1177)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:129)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:112)
	at org.testng.TestRunner.privateRun(TestRunner.java:756)
	at org.testng.TestRunner.run(TestRunner.java:610)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:387)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:382)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
	at org.testng.SuiteRunner.run(SuiteRunner.java:289)