Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t-sql hacks that shouldn't break mainline behavior #310

Merged
merged 10 commits into from
Jan 11, 2021
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
- Fix `equality` test when used with ephemeral models + explicit column set ([#321](https://github.com/fishtown-analytics/dbt-utils/pull/321))
- Fix `get_query_results_as_dict` integration test with consistent ordering ([#322](https://github.com/fishtown-analytics/dbt-utils/pull/322))
- All macros are now properly dispatched, making it possible for non-core adapters to implement a shim package for dbt-utils ([#312](https://github.com/fishtown-analytics/dbt-utils/pull/312)) Thanks [@chaerinlee1](https://github.com/chaerinlee1) and [@swanderz](https://github.com/swanderz)
- Small, non-breaking changes to accomdate TSQL (can't group by column number references, no real TRUE/FALSE values, aggregation CTEs need named columns) ([#310](https://github.com/fishtown-analytics/dbt-utils/pull/310)) Thanks [@swanderz](https://github.com/swanderz)

# dbt-utils v0.6.3

Expand Down
3 changes: 3 additions & 0 deletions integration_tests/dbt_project.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ clean-targets: # directories to be removed by `dbt clean`
- "target"
- "dbt_modules"

vars:
dbt_utils_dispatch_list: ['dbt_utils_integration_tests']

seeds:

+quote_columns: false
Expand Down
7 changes: 7 additions & 0 deletions integration_tests/macros/limit_zero.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{% macro limit_zero() %}
{{ return(adapter.dispatch('limit_zero', dbt_utils._get_utils_namespaces())()) }}
{% endmacro %}

{% macro default__limit_zero() %}
{{ return('limit 0') }}
{% endmacro %}
Original file line number Diff line number Diff line change
Expand Up @@ -77,5 +77,5 @@ Instead, we'll manually check that the values of these dictionaries are equivale
{% endif %}

{{ log(ns.err_msg, info=True) }}
select 1 {% if ns.pass %} limit 0 {% endif %}
select 1 as col_name {% if ns.pass %} {{ limit_zero() }} {% endif %}
{% endif %}
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{% if dbt_utils.pretty_log_format() is string %}
{# Return 0 rows for the test to pass #}
select 1 limit 0
select 1 as col_name {{ limit_zero() }}
{% else %}
{# Return >0 rows for the test to fail #}
select 1
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{% if dbt_utils.pretty_time() is string %}
{# Return 0 rows for the test to pass #}
select 1 limit 0
select 1 as col_name {{ limit_zero() }}
{% else %}
{# Return >0 rows for the test to fail #}
select 1
Expand Down
5 changes: 3 additions & 2 deletions macros/schema_tests/at_least_one.sql
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,9 @@
select count(*)
from (
select

count({{ column_name }})
{# In TSQL, subquery aggregate columns need aliases #}
{# thus: a filler col name, 'filler_column' #}
count({{ column_name }}) as filler_column

from {{ model }}

Expand Down
8 changes: 6 additions & 2 deletions macros/schema_tests/cardinality_equality.sql
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
{% macro test_cardinality_equality(model, to, field) %}

{{ return(adapter.dispatch('test_cardinality_equality', packages = dbt_utils._get_utils_namespaces())(model, to, field, **kwargs)) }}

{% endmacro %}

{% macro default__test_cardinality_equality(model, to, field) %}

{# T-SQL doesn't let you use numbers as aliases for columns #}
{# Thus, no "GROUP BY 1" #}

{% set column_name = kwargs.get('column_name', kwargs.get('from')) %}


Expand All @@ -13,15 +17,15 @@ select
{{ column_name }},
count(*) as num_rows
from {{ model }}
group by 1
group by {{ column_name }}
),

table_b as (
select
{{ field }},
count(*) as num_rows
from {{ to }}
group by 1
group by {{ column_name }}
),

except_a as (
Expand Down
6 changes: 4 additions & 2 deletions macros/schema_tests/expression_is_true.sql
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
{% macro test_expression_is_true(model, condition='true') %}
{% macro test_expression_is_true(model, condition='1=1') %}
{# T-SQL has no boolean data type so we use 1=1 which returns TRUE #}
{# ref https://stackoverflow.com/a/7170753/3842610 #}
{{ return(adapter.dispatch('test_expression_is_true', packages = dbt_utils._get_utils_namespaces())(model, condition, **kwargs)) }}
{% endmacro %}

{% macro default__test_expression_is_true(model, condition='true') %}
{% macro default__test_expression_is_true(model, condition) %}

{% set expression = kwargs.get('expression', kwargs.get('arg')) %}

Expand Down
4 changes: 3 additions & 1 deletion macros/schema_tests/not_constant.sql
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ select count(*)
from (

select
count(distinct {{ column_name }})
{# In TSQL, subquery aggregate columns need aliases #}
{# thus: a filler col name, 'filler_column' #}
count(distinct {{ column_name }}) as filler_column

from {{ model }}

Expand Down
6 changes: 4 additions & 2 deletions macros/schema_tests/relationships_where.sql
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@
{% macro default__test_relationships_where(model, to, field) %}

{% set column_name = kwargs.get('column_name', kwargs.get('from')) %}
{% set from_condition = kwargs.get('from_condition', "true") %}
{% set to_condition = kwargs.get('to_condition', "true") %}
{# T-SQL has no boolean data type so we use 1=1 which returns TRUE #}
{# ref https://stackoverflow.com/a/7170753/3842610 #}
{% set from_condition = kwargs.get('from_condition', "1=1") %}
{% set to_condition = kwargs.get('to_condition', "1=1") %}

with left_table as (

Expand Down