Custom tags

[cmcarthur]

Currently all data tests are tagged with data, and all schema tests are tagged with schema. The test selector supports selecting tests by tag -- that's how schema-only and data-only test runs work in development.

YAML has some notion of tags for schema tests. For data tests (and possibly models?) we'll need a way to configure tags with Jinja.

Finally we'll need a way to select tags via the CLI.

Some thought should be given to how tags behave -- a good naive approach given a tag t is to include any node that has tag t, even if it has other tags, and to exclude any node without tag t. So given tags data,schema, run all tests. This doesn't let you exclude tags or do boolean logic on which tags to run.

[jthandy]

/cc @atjones0011

[atjones0011]

This would be incredibly useful for me and my team. Here's a transcript of what I posted in the DBT Slack group on this topic:

We have 50+ models that are organized by the area of business that they support. We are currently working on splitting up our dbt run into multiple runs that are organized by how frequently the data is needed. With this setup, some models would run multiple times per hour and some would run once per day. We would be able to accomplish this goal by reorganizing our sql files, but we don’t want to lose our current organization.

Proposal:
Support for a new parameter in the project YAML named tags which will have a value that is a list of strings. When performing dbt run, have a parameter --tags which allows us to run any models that have the tag provided.

[cmcarthur]

@atjones0011 can you elaborate a little bit on this part?

Support for a new parameter in the project YAML named tags which will have a value that is a list of strings.

My brain is interpreting that as applying tags to models from the dbt_project.yml file, like so:

models:
  project_name:
    accounting:
      tags: accounting
    product:
      tags: product
    common:
      tags: accounting, product

Is that what you had in mind?

[atjones0011]

That's exactly what I was thinking. The tags I have in mind for my particular use case are frequent, daily, weekly.

[cmcarthur]

Oh, that is cool. Thanks @atjones0011 !

[drewbanin]

We should do this soon. The primary blocker is #550, which handles the selection of tags. Once that's implemented, it should be exceedingly straightforward to implement this feature.

I'll add: we'll probably want to move schema and data out of tags and into some other attribute for tests. They don't really belong there, and I think they have the potential to complicate some things down the line. Eg: if a user adds a tag called data to a model