dbt utils v1.0.0-rc1 (#733)

* add safe_divide documentation * add safe_divide macro * add integration test for safe_divide macro * Merge changes from main into utils v1 (#699) * Correct link from README to the CONTRIBUTING guide. (#687) * fix typo (#688) Co-authored-by: Alex Malins <[email protected]> * Change `escape_single_quotes` Reference in Pivot Macro (#692) * Update pivot.sql * Changelog Updates Co-authored-by: Liam O'Boyle <[email protected]> Co-authored-by: Alex Malins <[email protected]> Co-authored-by: Alex Malins <[email protected]> Co-authored-by: zachoj10 <[email protected]> * Use backwards comaptible versions of timestamp macro * moved macro and documentation to new SQL generator section * add tests with expressions * fix syntax errors (#705) * fix syntax errors * remove whitespace in seed file * Restore dbt. prefix for all migrated cross-db macros (#701) * added prefix dbt. on cross db macros * Also prefix for new macro * Adding changelog change * Squashed commit of the following: commit 5eba82b Author: Deanna Minnick <[email protected]> Date: Wed Oct 12 10:30:42 2022 -0400 remove whitespace in seed file commit 7a2a5e3 Author: Deanna Minnick <[email protected]> Date: Wed Oct 12 10:22:07 2022 -0400 fix syntax errors Co-authored-by: Joel Labes <[email protected]> * Remove obsolete condition argument from expression_is_true (#700) * Remove obsolete condition argument from expression_is_true * Improve docs * Improve docs * Update star.sql to allow for non-quote wrapped column names (#706) * Update star.sql * Update star.sql * feat: add testing to star macro column encased in quotes functionality * chore: update schema.yml * Update star.sql * chore: update star.sql and schema.yml * chore: update star.sql to trim blank space * Update README.md * Update README.md adds example usage of star macro's quote_identifiers argument Co-authored-by: crlough <[email protected]> * Switch to dbt.escape_single_quotes * Change deprecation resolution advice * Wrap xdb warnings in if execute block * Slugify for snowflake (#707) * Merge main into utils-v1 (#726) * Feature/safe divide (#697) * add safe_divide documentation * add safe_divide macro * add integration test for safe_divide macro * moved macro and documentation to new SQL generator section Co-authored-by: Grace Goheen <[email protected]> * Revert "Feature/safe divide (#697)" (#702) This reverts commit f368cec. * Quick nitpicks (#718) I was doing some studying on these and spotted some stuff. One verb conjugation and a consistency in macro description Co-authored-by: deanna-minnick <[email protected]> Co-authored-by: Grace Goheen <[email protected]> Co-authored-by: ian-fahey-dbt <[email protected]> * Feat: add macro get_query_results_as_single_value (#696) * feat: add query_results_as_single_value.sql macro * chore: update the macro definition Current error to work through: "failed to find conversion function from unknown to text" * chore: update test * chore: final edits * chore: remove extra model reference * chore: update return() to handle BigQuery * chore: README.md, macro updates * feat: factoring in first review changes * chore: updates to testing * chore: updates tests * chore: update test for bigquery * chore: update cast for bigquery * Use example with a single record in readme * Add default value when no record found * test when no results are found * Rename test file * Add test definitions * Fix incorrect ref * And another one * Update test_get_query_results_as_single_value.sql * cast strings as strings * Put arg in right place * Update test_get_query_results_as_single_value.sql * switch to limit zero for BQ * Update test_get_query_results_as_single_value.sql * quote column name in arg * snowflake wont let you safe cast something to itself * warning to future readers [skip ci] * Add singular test to check for multi row/multi column setup * forgot to save comment [skip ci] * Rename to get_single_value Co-authored-by: crlough-gitkraken <[email protected]> Co-authored-by: Joel Labes <[email protected]> * Remove rc1 requirement for utils v1 * Recency truncate date option (#731) * WIP changing recency test * Add tests * cast to timestamp for bq * forgot the curlies * avoid lateral column aliasing * ts not dt * cast source as timestamp * don't cast inside test * cast as date instead of truncate * Update recency.sql * log bq events * store pg artifacts * int tests dir * Correctly store artifacts * try casting to date or datetime * order of operations more like order of ooperations * dt -> ts * Do I really have to cast this? * Revert "Do I really have to cast this?" This reverts commit 21e2c0d. * Output a warning when star finds no columns, not '*' (#732) * Change star() behaviour when no columns returned * Code review: return a * in compile mode * README changes * Delete xdb_deprecation_warning.sql * Update README.md * Remove from ToC * Update toc * Fix surrogate key variable example Co-authored-by: Deanna Minnick <[email protected]> Co-authored-by: Liam O'Boyle <[email protected]> Co-authored-by: Alex Malins <[email protected]> Co-authored-by: Alex Malins <[email protected]> Co-authored-by: zachoj10 <[email protected]> Co-authored-by: Grace Goheen <[email protected]> Co-authored-by: deanna-minnick <[email protected]> Co-authored-by: Simon Quvang <[email protected]> Co-authored-by: miles <[email protected]> Co-authored-by: Connor <[email protected]> Co-authored-by: crlough <[email protected]> Co-authored-by: fivetran-catfritz <[email protected]> Co-authored-by: ian-fahey-dbt <[email protected]> Co-authored-by: crlough-gitkraken <[email protected]>
dbt-labs · Dec 1, 2022 · eb12922 · eb12922
1 parent 6d76641
commit eb12922
Show file tree

Hide file tree

Showing 42 changed files with 554 additions and 386 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -23,7 +23,9 @@ jobs:
           name: "Run OG Tests - Postgres"
           command: ./run_test.sh postgres
       - store_artifacts:
-          path: ./logs
+          path: integration_tests/logs
+      - store_artifacts:
+          path: integration_tests/target
 
   integration-redshift:
     docker:
@@ -35,7 +37,9 @@ jobs:
           name: "Run OG Tests - Redshift"
           command: ./run_test.sh redshift
       - store_artifacts:
-          path: ./logs
+          path: integration_tests/logs
+      - store_artifacts:
+          path: integration_tests/target
 
   integration-snowflake:
     docker:
@@ -47,8 +51,10 @@ jobs:
           name: "Run OG Tests - Snowflake"
           command: ./run_test.sh snowflake
       - store_artifacts:
-          path: ./logs          
-
+          path: integration_tests/logs          
+      - store_artifacts:
+          path: integration_tests/target
+
   integration-bigquery:
     environment:
       BIGQUERY_SERVICE_KEY_PATH: "/home/circleci/bigquery-service-key.json"
@@ -64,7 +70,9 @@ jobs:
           name: "Run OG Tests - BigQuery"
           command: ./run_test.sh bigquery
       - store_artifacts:
-          path: ./logs
+          path: integration_tests/logs
+      - store_artifacts:
+          path: integration_tests/target
 
 workflows:
   version: 2

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -11,30 +11,66 @@
 # Unreleased
 
 ## New features
+- Updated the `slugify` macro to prepend "_" to column names beginning with a number since most databases do not allow names to begin with numbers. 
 - Implemented an optional `group_by_columns` argument across many of the generic testing macros to test for properties that only pertain to group-level or are can be more rigorously conducted at the group level. Property available in `recency`, `at_least_one`, `equal_row_count`, `fewer_rows_than`, `not_constant`, `not_null_proportion`, and `sequential` tests [#633](https://github.com/dbt-labs/dbt-utils/pull/633)
 - New feature to omit the `source_column_name` column on the `union_relations` macro ([#331](https://github.com/dbt-labs/dbt-utils/issues/331), [#624](https://github.com/dbt-labs/dbt-utils/pull/624))
+- New macro `get_single_value` ([#696](https://github.com/dbt-labs/dbt-utils/pull/696))
 - New feature to select fewer columns in `expression_is_true` ([#683](https://github.com/dbt-labs/dbt-utils/issues/683), [#686](https://github.com/dbt-labs/dbt-utils/pull/686))
 - Add `not_empty_string` generic test that asserts column values are not an empty string. ([#632](https://github.com/dbt-labs/dbt-utils/issues/632), [#634](https://github.com/dbt-labs/dbt-utils/pull/634))
 
 ## Under the hood
-- Remove deprecated table argument from unpivot ([#671](https://github.com/dbt-labs/dbt-utils/pull/671))
+- Remove deprecated table argument from `unpivot` ([#671](https://github.com/dbt-labs/dbt-utils/pull/671))
 - Delete the deprecated identifier macro ([#672](https://github.com/dbt-labs/dbt-utils/pull/672))
 - Handle deprecations in deduplicate macro ([#673](https://github.com/dbt-labs/dbt-utils/pull/673))
-- Fully remove varargs usage in surrogate_key and safe_add ([#674](https://github.com/dbt-labs/dbt-utils/pull/674))
+- Fully remove varargs usage in `surrogate_key` and `safe_add` ([#674](https://github.com/dbt-labs/dbt-utils/pull/674))
+- Remove obsolete condition argument from `expression_is_true` ([#699](https://github.com/dbt-labs/dbt-utils/pull/699))
+
+## Migration instructions
+- If your project uses the `expression_is_true` macro, replace `condition` argument with `where`. 
+
+Before:
+```yaml
+version: 2
+
+models:
+  - name: model_name
+    tests:
+      - dbt_utils.expression_is_true:
+          expression: "col_a + col_b = total"
+          condition: "created_at > '2018-12-31'"
+```
+After:
+```yaml
+version: 2
+
+models:
+  - name: model_name
+    tests:
+      - dbt_utils.expression_is_true:
+          expression: "col_a + col_b = total"
+          config:
+            where: "created_at > '2018-12-31'"
+```
 
 ## Fixes
+- Add star macro option to not encase column names in quotes. ([#706](https://github.com/dbt-labs/dbt-utils/pull/706))
+- Explicitly stating the namespace for cross-db macros so that the dispatch logic works correctly by restoring the dbt. prefix for all migrated cross-db macros ([#701](https://github.com/dbt-labs/dbt-utils/pull/701))
 - Better handling of whitespaces in the star macro ([#651](https://github.com/dbt-labs/dbt-utils/pull/651))
 - Fix to correct behavior in `mutually_exclusive_ranges` test in certain situations when `zero_length_range_allowed: true` and multiple ranges in a partition have the same value for `lower_bound_column`. ([[#659](https://github.com/dbt-labs/dbt-utils/issues/659)], [#660](https://github.com/dbt-labs/dbt-utils/pull/660))
 - Fix to utilize dbt Core version of `escape_single_quotes` instead of version from dbt Utils ([[#689](https://github.com/dbt-labs/dbt-utils/issues/689)], [#692](https://github.com/dbt-labs/dbt-utils/pull/692))
 
 ## Contributors:
+- [@CR-Lough] (https://github.com/CR-Lough) (#706) (#696)
+- [@fivetran-catfritz](https://github.com/fivetran-catfritz)
+- [@crowemi](https://github.com/crowemi)
+- [@SimonQuvang](https://github.com/SimonQuvang) (#701)
 - [@christineberger](https://github.com/christineberger) (#624)
 - [@epapineau](https://github.com/epapineau) (#634)
 - [@courentin](https://github.com/courentin) (#651)
 - [@sfc-gh-ancoleman](https://github.com/sfc-gh-ancoleman) (#660)
 - [@zachoj10](https://github.com/zachoj10) (#692)
 - [@miles170](https://github.com/miles170)
-- [@emilyriederer](https://github.com/emilyriederer) 
+- [@emilyriederer](https://github.com/emilyriederer)
 
 # dbt-utils v0.8.6