Add optional condition to not_null_proportion test

The dbt core `not_null` test allows a `where` parameter and the dbt
utils `expression_is_true` allows a `condition` parameter, both of which
allow the test to operate on a subset of rows.

This adds a similar functoinality to the `not_null_proportion` test,
sticking with the `condition` convention from `expression_is_true.

CHANGELOG.md

@@ -12,6 +12,7 @@
 
 ## New features
 - New feature to omit the `source_column_name` column on the `union_relations` macro ([#331](https://github.com/dbt-labs/dbt-utils/issues/331), [#624](https://github.com/dbt-labs/dbt-utils/pull/624))
+- New feature to allow a `condition` option to `not_null_proportion` tests ([#691](https://github.com/dbt-labs/dbt-utils/pull/691))
 
 ## Fixes
 - Better handling of whitespaces in the star macro ([#651](https://github.com/dbt-labs/dbt-utils/pull/651))

README.md

@@ -317,6 +317,23 @@ models:
               at_least: 0.95
 ```
 
+The macro accepts an optional argument `condition` that allows for asserting the proportion is not null on a subset of all records.
+
+**Usage:**
+
+```yaml
+version: 2
+
+models:
+  - name: my_model
+    columns:
+      - name: id
+        tests:
+          - dbt_utils.not_null_proportion:
+              at_least: 0.95
+              condition: "created_at > '2018-12-31'"
+```
+
 #### not_accepted_values ([source](macros/generic_tests/not_accepted_values.sql))
 
 Asserts that there are no rows that match the given values.

integration_tests/models/generic_tests/schema.yml

@@ -146,6 +146,10 @@ seeds:
         tests:
           - dbt_utils.not_null_proportion:
               at_least: 0.9
+          - dbt_utils.not_null_proportion:
+              at_least: 1
+              at_most: 1
+              condition: "point_9 IS NOT NULL"
 
 models:
   - name: test_recency

macros/generic_tests/not_null_proportion.sql

@@ -7,11 +7,13 @@
 {% set column_name = kwargs.get('column_name', kwargs.get('arg')) %}
 {% set at_least = kwargs.get('at_least', kwargs.get('arg')) %}
 {% set at_most = kwargs.get('at_most', kwargs.get('arg', 1)) %}
+{% set condition = kwargs.get('condition', kwargs.get('arg', '1=1')) %}
 
 with validation as (
   select
     sum(case when {{ column_name }} is null then 0 else 1 end) / cast(count(*) as numeric) as not_null_proportion
   from {{ model }}
+  where {{ condition }}
 ),
 validation_errors as (
   select