Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Add option to remove the source_column_name on the union_relations macro #624

Merged
merged 6 commits into from
Jul 26, 2022

Conversation

christineberger
Copy link
Contributor

@christineberger christineberger commented Jul 21, 2022

This is a:

  • documentation update
  • bug fix with no breaking changes
  • new functionality
  • a breaking change

All pull requests from community contributors should target the main branch (default).

Description & motivation

This covers issue #331, which allows passing in none in order to omit a source_column_name column when unioning.

Checklist

  • I have verified that these changes work locally on the following warehouses (Note: it's okay if you do not have access to all warehouses, this helps us understand what has been covered)
    • BigQuery
    • Postgres
    • Redshift
    • Snowflake - This PR only adds a Jinja snippet and does not modify any SQL.
  • I followed guidelines to ensure that my changes will work on "non-core" adapters by: N/A
    • dispatching any new macro(s) so non-core adapters can also use them (e.g. the star() source)
    • using the limit_zero() macro in place of the literal string: limit 0
    • using dbt_utils.type_* macros instead of explicit datatypes (e.g. dbt_utils.type_timestamp() instead of TIMESTAMP
  • I have updated the README.md (if applicable)
  • I have added tests & descriptions to my models (and macros if applicable)
  • I have added an entry to CHANGELOG.md

Copy link
Contributor

@joellabes joellabes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it 😻 thank you for doing this! For integration tests, you can follow the same pattern as the existing union relations tests: update: this is good advice for testing in general, but not in this case! see below

  • make a model that calls union_relations but passes None as the column name, like this one
  • Make a seed file for the expected output, like this one
  • Add an equality test like this

So here's a fun thing I found while building this list! We actually are already dropping the column in our testing. The commit message implies that it's because the tables change depending on the environment which makes sense!

Perhaps what you do instead is run a test that validates the columns that you expect are there, but don't worry about actually testing the union itself. You could do that either by making an empty seed file and comparing that to a {{ limit_zero() }}'d version of the table, or by using dbt-expectations' expect_table_columns_to_match_ordered_list

README.md Outdated Show resolved Hide resolved
Co-authored-by: Joel Labes <[email protected]>
@christineberger
Copy link
Contributor Author

@joellabes I committed your suggestion and also added integration tests! Requested re-review. 😊

Copy link
Contributor

@joellabes joellabes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

outstanding! thank you ⭐ i'll commit the typo fix on your behalf and then get this merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants