implement type_boolean macro

[jpmmcneill]

resolves (partially) #5739

Description

dbt-core component required for #5739.

Other PRs

• dbt-redshift: add type boolean dbt-redshift#190
• dbt-bigquery: add type boolean dbt-bigquery#313
• dbt-snowflake: Jpmmcneill/snowflake type boolean dbt-snowflake#268
• dbt-spark: Jpmmcneill/spark type boolean dbt-spark#471

Checklist

• I have read the contributing guide and understand what's expected of me
• I have signed the CLA
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• I have opened an issue to add/update docs, or docs changes are not required/relevant for this PR
  • Update list of cross-database macros docs.getdbt.com#2049
• I have run changie new to create a changelog entry

[jpmmcneill]

every other comment had this many dashes 🙃

[dbeatty10]

@jpmmcneill this is looking good to me! 🤩

Here's the next steps for this PR:

1. Run changie new per these instructions and commit and push the resulting file.
2. Open up pull requests for dbt-redshift, dbt-snowflake, and dbt-spark that add your new test to their test suites.
   • Unfortunately, new tests aren't inherited and executed automatically -- they need to be added explicitly.

I'll add some further instructions on your PR for dbt-bigquery on how to point at this PR branch for the purposes of automated CI tests in GitHub. (We'll want the adapters to be using your PR branch rather than the main branch.)

Once all of the PRs are passing the tests in GitHub Actions, we will merge dbt-core first. Then we will run CI once more for each adapter (pointing back at main) before merging them.

[dbeatty10]

Probably want to cover both boolean options at the very least.

Suggested change

	seeds__expected_csv = """boolean_col
	True
	""".lstrip()
	models__actual_sql = """
	select cast('True' as {{ type_boolean() }}) as boolean_col
	"""
	seeds__expected_csv = """boolean_col
	True
	False
	""".lstrip()
	models__actual_sql = """
	select cast('True' as {{ type_boolean() }}) as boolean_col
	union all
	select cast('False' as {{ type_boolean() }}) as boolean_col
	"""

Even better

But test cases covering the truth tables for conjunction, disjunction, and negation would be even better. That way, we are verifying that everything is acting like booleans.

Something like the following untested code:

seeds__boolean_permutations_csv = """
x,y
False,False
True,False
False,True
True,True
""".lstrip()

seeds__expected_csv = """
x,y,conjunction,disjunction,negation_x
False,False,False,False,True
True,False,False,True,False
False,True,False,True,True
True,True,True,True,False
""".lstrip()

models__actual_sql = """
select
    x,
    y,
    x and y as conjunction,
    x or y as disjunction,
    not x as negation_x
from {{ ref("boolean_permutations" }}

Best? 🤷

Even though BOOLEAN is a data type described in the SQL standard, some databases don't have it (looking at you, SQL Server!).

If we are feeling extra magnanimous, we could change all the True/False values in the seeds to be 1/0 instead.

I'm hoping it wouldn't be necessary, but we could update the models__actual_sql definition so that x/y values are replaced with the following instead:

• cast(x as {{ type_boolean() }})
• cast(y as {{ type_boolean() }})

[dbeatty10]

Update:

The "Even better" example I gave above didn't actually test the type_boolean() macro at all! 😅

Something like this should fix that situation:

models__actual_sql = """
select
    cast(x as {{ type_boolean() }}) as x_bool,
    cast(y as {{ type_boolean() }}) as y_bool,
    cast(x as {{ type_boolean() }}) and cast(y as {{ type_boolean() }}) as conjunction,
    cast(x as {{ type_boolean() }}) or cast(y as {{ type_boolean() }}) as disjunction,
    not cast(x as {{ type_boolean() }}) as negation_x
from {{ ref("boolean_permutations" }}

[jpmmcneill]

Cheers @dbeatty10. I'll take a second pass at this later today :)

[dbeatty10]

@jpmmcneill I just realized that your simple pytest for type_boolean() was exactly in line with the rest of the type_x macros.

So I'm approving as-is.

After this PR is merged, then the next steps will be to restore the original dev-requirements.txt files in each of your adapter PRs.

[jpmmcneill]

@jpmmcneill I just realized that your simple pytest for type_boolean() was exactly in line with the rest of the type_x macros.

So I'm approving as-is.

After this PR is merged, then the next steps will be to restore the original dev-requirements.txt files in each of your adapter PRs.

Hey @dbeatty10 - thanks. I personally completely agree with the sentiment around the current level of testing :).

Do you agree with me that an issue that basically scopes "improve the current test coverage for data types" would be welcome?

[dbeatty10]

Do you agree with me that an issue that basically scopes "improve the current test coverage for data types" would be welcome?

Formally logging where expectations differs from reality is an important mechanism for us. A new issue describing the current level of testing and how that compares to what your expectations were as a contributor (here) and dbt package maintainer (here) would be great!

From there, someone from dbt Labs (maybe me!) will triage the issue submission and give feedback, determine urgency, compare to current roadmap and capacity, etc.

[jpmmcneill]

Brill, will do. Thanks @dbeatty10 🐐