Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Selections with unions fail if one of the sets is empty #10991

Closed
2 tasks done
mroy-seedbox opened this issue Nov 12, 2024 · 2 comments
Closed
2 tasks done

[Bug] Selections with unions fail if one of the sets is empty #10991

mroy-seedbox opened this issue Nov 12, 2024 · 2 comments
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core

Comments

@mroy-seedbox
Copy link

mroy-seedbox commented Nov 12, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

tl;dr: Unions should be able to handle empty sets. There is no programming language which crashes when doing the union of empty arrays/list/sets/etc.

When doing unions in an evolving project, models will come and go (and thus some selections/sets may be empty). However, dbt seems unable to handle this, and returns an error instead.

For example: dbt ls -s something,tag:one something,tag:two ...

If the last tag:one or tag:two model is removed from something, then the entire selection will fail, and as a result the entire job/pipeline will fail also.

Error message:

Compilation Error
  The selection criterion 'something,tag:one|two' does not match any enabled nodes

Expected Behavior

Empty sets are simply ignored.

Or maybe introduce a new parameter in the CLI, something like --ignore-empty-sets, or --ignore-empty-selection. It should be a fairly simple change: if the parameter is set, do not throw this error. It would be the most explicit option, while also preserving backward-compatibility.

Alternatives Considered

Steps To Reproduce

Do a union with an empty set. See above for more details.

Relevant log output

No response

Environment

- OS: Any
- Python: Any
- dbt: 1.8.8 (but happens in any version)

Which database adapter are you using with dbt?

Any.

Additional Context

No response

@mroy-seedbox
Copy link
Author

#10992 would offer a clean solution to for this issue.

@mroy-seedbox
Copy link
Author

Actually, there is already a solution for this!

We can specify NoNodesForSelectionCriteria in the warn_error_options flag/config (see documentation here).

Closing this issue! 🎉

@dbeatty10 dbeatty10 added wontfix Not a bug or out of scope for dbt-core and removed triage labels Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working wontfix Not a bug or out of scope for dbt-core
Projects
None yet
Development

No branches or pull requests

2 participants