Upgrade to support dbt core v141 #214

damian3031 · 2023-01-26T18:53:01Z

Closes #213

github-actions · 2023-01-26T18:53:22Z

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

hovaesco

New tests introduced in 1.4.0 are missing.

.changes/unreleased/Breaking Changes-20230126-200731.yaml

damian3031 · 2023-01-30T08:26:27Z

New tests introduced in 1.4.0 are missing.

@hovaesco
All new tests from dbt-core 1.4.0 added

findinpath · 2023-01-30T09:27:56Z

dbt/adapters/trino/column.py

@@ -3,7 +3,7 @@
 from typing import ClassVar, Dict

 from dbt.adapters.base.column import Column
-from dbt.exceptions import RuntimeException
+from dbt.exceptions import DbtRuntimeError


My IDE says (while checking only this commit):

Cannot find reference to DbtRuntimeError in exceptions.py

that is a breaking change in dbt 1.4.0 dbt-labs/dbt-core#6460

findinpath · 2023-01-30T09:37:06Z

dbt/adapters/trino/relation.py

@@ -13,7 +13,7 @@ class TrinoQuotePolicy(Policy):

 @dataclass(frozen=True, eq=False, repr=False)
 class TrinoRelation(BaseRelation):
-    quote_policy: TrinoQuotePolicy = TrinoQuotePolicy()
+    quote_policy: TrinoQuotePolicy = field(default_factory=lambda: TrinoQuotePolicy())


Is this change related to the upgrade?

yes, because of this change

findinpath · 2023-01-30T09:53:55Z

dbt/include/trino/macros/materializations/incremental.sql

@@ -86,9 +86,9 @@
    {% endif %}

    {#-- Get the incremental_strategy, the macro to use for the strategy, and build the sql --#}
-    {% set incremental_predicates = config.get('incremental_predicates', none) %}
+    {% set incremental_predicates = config.get('predicates', none) or config.get('incremental_predicates', none) %}


Note that this change is rather characteristic for dbt-core 1.4.1 and not dbt-core 1.4.0

See dbt-labs/dbt-core@e8da84f

Actually this PR is about to adjust to 1.4.1

findinpath · 2023-01-30T09:56:27Z

README.md

+| jwt_token                      | JWT token for authentication                                                                                 | Optional (required if `method` is `jwt`)                                                                | `none` or `abc123`               |
+| client_certificate             | Path to client certificate to be used for certificate based authentication                                   | Optional (required if `method` is `certificate`)                                                        | `/tmp/tls.crt`                   |
+| client_private_key             | Path to client private key to be used for certificate based authentication                                   | Optional (required if `method` is `certificate`)                                                        | `/tmp/tls.key`                   |
+| http_headers                   | HTTP Headers to send alongside requests to Trino, specified as a yaml dictionary of (header, value) pairs.   | Optional                                                                                                | `X-Trino-Client-Info: dbt-trino` |


Pls add a separate commit for the spacing changes in the README.md

Extracted to separate commit

findinpath · 2023-01-30T09:57:38Z

README.md

-| threads                        | How many threads dbt should use                                                                              | Optional (default is `1`)                                                                               | `8`                                      |
-| prepared_statements_enabled    | Enable usage of Trino prepared statements (used in `dbt seed` commands)                                      | Optional (default is `true`)                                                                            | `true` or `false`                        |
-| retries                        | Configure how many times a database operation is retried when connection issues arise                        | Optional (default is `3`)                                                                               | `10`                                     |
+| Option                         | Description                                                                                                  | Required?                                                                                               | Example                          |


btw, you could extract this change to a separate PR which is much easier to grasp and to merge

findinpath · 2023-01-30T09:58:02Z

.changes/unreleased/Under the Hood-20230130-092157.yaml

@@ -0,0 +1,7 @@
+kind: Under the Hood
+body: Adjustment to dbt-core 1.4.0


Why not 1.4.1 ?

Actually this PR is about to adjust to 1.4.1, I changed this here

hovaesco · 2023-01-30T10:58:21Z

tests/functional/adapter/materialization/test_incremental_predicates.py

+            "models": {
+                "+incremental_predicates": ["dbt_internal_dest.id != 2"],
+                "+incremental_strategy": "merge",
+                "+on_table_exists": "drop",


on_table_exists probably not needed here.

…0. Add tests for incremental_predicates.

hovaesco · 2023-01-30T14:25:06Z

setup.py

@@ -86,7 +86,7 @@ def _get_dbt_core_version():
    },
    install_requires=[
        "dbt-core~={}".format(dbt_version),
-        "trino==0.319.0",
+        "trino==0.321.0",


Add to changie.

hovaesco · 2023-01-30T14:25:23Z

dbt/adapters/trino/connections.py

@@ -429,6 +435,7 @@ def open(cls, connection):
            isolation_level=IsolationLevel.AUTOCOMMIT,
            source="dbt-trino",
            verify=credentials.cert,
+            timezone=credentials.timezone,


Add to changie.

findinpath · 2023-01-31T08:27:30Z

.changes/unreleased/Features-20230130-160859.yaml

@@ -0,0 +1,7 @@
+kind: Features
+body: Added new connection property `timezone`


@damian3031 could you please document timezone in README.md ?

Related PR trinodb/trino-python-client#252

damian3031 added 2 commits January 26, 2023 19:31

Adjust to dbt-core v1.4.0 exceptions refactor

3edc045

Adjust to dbt-core v1.4.0 relation.py::BaseRelation refactor

852af3e

damian3031 requested review from findinpath and hovaesco January 26, 2023 18:53

damian3031 linked an issue Jan 26, 2023 that may be closed by this pull request

upgrade to support dbt-core v1.4.0 #213

Closed

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch from 0092ca3 to 1b3e09f Compare January 26, 2023 19:08

hovaesco reviewed Jan 26, 2023

View reviewed changes

.changes/unreleased/Breaking Changes-20230126-200731.yaml Outdated Show resolved Hide resolved

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch 5 times, most recently from f2ca6c2 to 1789204 Compare January 30, 2023 08:24

findinpath reviewed Jan 30, 2023

View reviewed changes

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch 2 times, most recently from b610008 to a93be8a Compare January 30, 2023 10:46

damian3031 changed the title ~~Upgrade to support dbt core v140~~ Upgrade to support dbt core v141 Jan 30, 2023

hovaesco reviewed Jan 30, 2023

View reviewed changes

damian3031 added 3 commits January 30, 2023 13:05

Adjust to addition of incremental_predicates config in dbt-core v1.4.…

3ab2d2c

…0. Add tests for incremental_predicates.

Add Query Comment Tests

d399d59

Add TestChangeRelationTypes

a7aa0c3

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch from a93be8a to 8290c5a Compare January 30, 2023 12:06

hovaesco approved these changes Jan 30, 2023

View reviewed changes

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch from 6147594 to e6517fb Compare January 30, 2023 15:10

damian3031 added 4 commits January 30, 2023 16:26

Bump and adjust to trino-python-client 0.321.0

f23ed40

README spacing changes

ab98fbd

Bump Trino and SEP versions in tests containers

884e84e

Update pre-commit hooks versions

1ae3aac

damian3031 added 2 commits January 30, 2023 16:26

Bump dbt-tests-adapter to version 1.4.1

8db6034

Bump dbt-trino to version 1.4.0

5ee545d

damian3031 force-pushed the upgrade-to-support-dbt-core-v140 branch from e6517fb to 5ee545d Compare January 30, 2023 15:26

hovaesco approved these changes Jan 30, 2023

View reviewed changes

hovaesco merged commit 18fe2cd into master Jan 30, 2023

hovaesco deleted the upgrade-to-support-dbt-core-v140 branch January 30, 2023 15:36

findinpath reviewed Jan 31, 2023

View reviewed changes

damian3031 mentioned this pull request Jan 31, 2023

Upgrade trino-python-client to 0.321.0 #208

Closed

hovaesco mentioned this pull request Feb 3, 2023

upgrade to support dbt-core v1.4.0 #223

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade to support dbt core v141 #214

Upgrade to support dbt core v141 #214

damian3031 commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

hovaesco left a comment

damian3031 commented Jan 30, 2023

findinpath Jan 30, 2023

damian3031 Jan 30, 2023

findinpath Jan 30, 2023

damian3031 Jan 30, 2023

findinpath Jan 30, 2023

damian3031 Jan 30, 2023

findinpath Jan 30, 2023

damian3031 Jan 30, 2023 •

edited

Loading

findinpath Jan 30, 2023

findinpath Jan 30, 2023

damian3031 Jan 30, 2023

hovaesco Jan 30, 2023

damian3031 Jan 30, 2023

hovaesco Jan 30, 2023

damian3031 Jan 30, 2023

hovaesco Jan 30, 2023

damian3031 Jan 30, 2023

findinpath Jan 31, 2023

findinpath Jan 31, 2023

		@@ -0,0 +1,7 @@
		kind: Under the Hood
		body: Adjustment to dbt-core 1.4.0

		@@ -0,0 +1,7 @@
		kind: Features
		body: Added new connection property `timezone`

Upgrade to support dbt core v141 #214

Upgrade to support dbt core v141 #214

Conversation

damian3031 commented Jan 26, 2023

github-actions bot commented Jan 26, 2023

hovaesco left a comment

Choose a reason for hiding this comment

damian3031 commented Jan 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damian3031 Jan 30, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

damian3031 Jan 30, 2023 •

edited

Loading