Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node selection mental model #514

Open
jtcohen6 opened this issue Jan 18, 2021 · 2 comments
Open

Node selection mental model #514

jtcohen6 opened this issue Jan 18, 2021 · 2 comments
Assignees
Labels
priority: medium Fix or enhancement to existing information that’s creating some requests from customers size: small This change will take 1 to 2 days to address

Comments

@jtcohen6
Copy link
Collaborator

It feels like there's endless appetite to provide explanation and example for when resource selection feels counterintuitive. I think we have a lot to gain from laying out exactly the way that node selection does work:

  1. dbt selects any resource that is matched by one or more of the --models/--select criteria, in the order of selection methods (e.g. tag:), then graph operators (e.g. +), then set operators (,). The selected resources may be models, sources, seeds, snapshots, tests. IFF the command is dbt test, dbt also selects any test that directly depends on a selected resource.
  2. dbt de-selects any resource that is matched by one or more of the --exclude criteria. IFF the command is dbt test, dbt also de-selects any test that directly depends on an excluded resource.
  3. dbt now has a list of still-selected resources of varying types. As a final step, it tosses away any resource that does not match the resource type of the current invocation. (Only seeds are kept for dbt seed, only models for dbt run, only tests for dbt test, and so on.)

Things like test selection and the source: selection method feel tricky because they intersect with this process is ways that can feel quite different from dbt run -m my_model. Rather than write more about each edge case, I think we need to build the mental model, so that they seem less like "exceptions" to rote-memorize and more like tricky cases that are still consistent with the overall schematic.

(Note that step 2 works a bit different for YAML selectors because the exclude: key subtly differs from --exclude, in order to power more complex set logic. That's briefly documented here.)

@runleonarun
Copy link
Collaborator

@joellabes and @jtcohen6 is this request still valid?

@joellabes
Copy link
Contributor

Yes please! Happy to help work with someone as a technical resource

@runleonarun runleonarun added the size: medium This change will take up to a week to address label May 7, 2023
@runleonarun runleonarun added priority: medium Fix or enhancement to existing information that’s creating some requests from customers size: small This change will take 1 to 2 days to address and removed size: medium This change will take up to a week to address labels Jun 14, 2023
nghi-ly pushed a commit that referenced this issue May 2, 2024
REPO SYNC - Public to Private
@mirnawong1 mirnawong1 self-assigned this Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: medium Fix or enhancement to existing information that’s creating some requests from customers size: small This change will take 1 to 2 days to address
Projects
None yet
Development

No branches or pull requests

4 participants