From 4607be410dbbf74eed78baefe6e6d36ca7e05b0b Mon Sep 17 00:00:00 2001 From: leahwicz <60146280+leahwicz@users.noreply.github.com> Date: Fri, 17 Sep 2021 08:36:31 -0400 Subject: [PATCH] Rebasing `develop` for latest 0.21.0 RC release (#216) * Show more detailed feedback when pyodbc import fails (#192) * Use exception chaining to get more detailed feedback when pyodbc is not installed * Remove pyodbc referenced before assignment * Set back try except * Add flake ignore * Add error message to RunTimeException Error chaining does not show the message in `dbt debug`. Therefore we explicitly add the message to the dbt.exceptions.RunTimeException * Update change log Add to change log that we print the import error when pyodbc can not be imported * Fix parenthesis in change log * Update changelog [skip ci] * Update changelog [skip ci] * Update changelog [skip ci] * Add support for ODBC Server Side Parameters (#201) * Add support for ODBC Server Side Parameters * Update CHANGELOG * Feature/able to retry all connections (#194) * Code changes * README changes * Improve error message default * Changelog * Changelog corrections * Restore accidental deletion * Update dbt/adapters/spark/connections.py Co-authored-by: Jeremy Cohen * Add myself to Contributors Co-authored-by: Jeremy Cohen * fixed get_columns_in_relation for open source delta table (#207) * fixed get_columns_in_relation for open source delta table * fixed E501 linting error and added change log * fix issue parsing structs (#204) * fix issue parsing structs * include contributor in changelog * better error explanation Co-authored-by: Jeremy Cohen * Add adapter unique_field (#211) * Add adapter unique_field * Fix flake8. Add changelog entry * [Snyk] Fix for 2 vulnerabilities (#214) * fix: requirements.txt to reduce vulnerabilities The following vulnerabilities are fixed by pinning transitive dependencies: - https://snyk.io/vuln/SNYK-PYTHON-SQLPARSE-1584201 - https://snyk.io/vuln/SNYK-PYTHON-THRIFT-474615 * Removing Thrift conflict with versions over 12 Co-authored-by: leahwicz <60146280+leahwicz@users.noreply.github.com> * 0.21.0b1 Release * Bumping version to 0.21.0b1 * Bumping to 0.21.0b2 * Bumping version to 0.21.0b2 Co-authored-by: Cor Co-authored-by: Jeremy Cohen Co-authored-by: Jethro Nederhof Co-authored-by: gregingenii <80900458+gregingenii@users.noreply.github.com> Co-authored-by: Jeremy Cohen Co-authored-by: Hariharan Banukumar Co-authored-by: Sergio Co-authored-by: Snyk bot Co-authored-by: Ian Knox Co-authored-by: Leah Antkiewicz --- CHANGELOG.md | 3 +++ dbt/adapters/spark/connections.py | 4 ++++ requirements.txt | 3 ++- 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c6f8c272f..7405e9e0e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,9 @@ - Enhanced get_columns_in_relation method to handle a bug in open source deltalake which doesnt return schema details in `show table extended in databasename like '*'` query output. This impacts dbt snapshots if file format is open source deltalake ([#207](https://github.com/dbt-labs/dbt-spark/pull/207)) - Parse properly columns when there are struct fields to avoid considering inner fields: Issue ([#202](https://github.com/dbt-labs/dbt-spark/issues/202)) +### Under the hood +- Add `unique_field` to better understand adapter adoption in anonymous usage tracking ([#211](https://github.com/dbt-labs/dbt-spark/pull/211)) + ### Contributors - [@harryharanb](https://github.com/harryharanb) ([#207](https://github.com/dbt-labs/dbt-spark/pull/207)) - [@SCouto](https://github.com/Scouto) ([#204](https://github.com/dbt-labs/dbt-spark/pull/204)) diff --git a/dbt/adapters/spark/connections.py b/dbt/adapters/spark/connections.py index 7bf6d159a..41413f355 100644 --- a/dbt/adapters/spark/connections.py +++ b/dbt/adapters/spark/connections.py @@ -135,6 +135,10 @@ def __post_init__(self): def type(self): return 'spark' + @property + def unique_field(self): + return self.host + def _connection_keys(self): return ('host', 'port', 'cluster', 'endpoint', 'schema', 'organization') diff --git a/requirements.txt b/requirements.txt index 2541acfc5..cbe57414b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,4 +2,5 @@ dbt-core==0.21.0b2 PyHive[hive]>=0.6.0,<0.7.0 pyodbc>=4.0.30 sqlparams>=3.0.0 -thrift>=0.11.0,<0.12.0 +thrift>=0.13.0 +sqlparse>=0.4.2 # not directly required, pinned by Snyk to avoid a vulnerability