Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create check to ensure that every model is tested #160

Closed
Tracked by #156
azaslavsky opened this issue Jul 10, 2023 · 1 comment
Closed
Tracked by #156

Create check to ensure that every model is tested #160

azaslavsky opened this issue Jul 10, 2023 · 1 comment

Comments

@azaslavsky
Copy link

azaslavsky commented Jul 10, 2023 •
edited
Loading

It would be nice to add some static checking for the following properties, to ensure that every model that ever gets added to Sentry interacts in a predictable, well-tested manner with the backup script:

  1. Every class that inherits from BaseModel is required to set the __include_in_export__ field with a boolean value.
  2. Every class that is ``include_in_export = Truemust be explicitly imported into/tests/sentry/backup/test_models.py`.

A good candidate for performing these checks is flake8, if it can express these constraints. The end result would be a guarantee that new model that gets created, or existing model that is newly marked as included in the export, will have at least one test.
@azaslavsky azaslavsky mentioned this issue Jul 10, 2023
Closed
@azaslavsky azaslavsky mentioned this issue Jul 11, 2023
Closed
azaslavsky added a commit to getsentry/sentry that referenced this issue Jul 11, 2023
@azaslavsky
test(backup): Ensure expected models are in output
343e98c 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
@azaslavsky azaslavsky mentioned this issue Jul 11, 2023
Merged
azaslavsky added a commit to getsentry/sentry that referenced this issue Jul 14, 2023
@azaslavsky
test(backup): Ensure expected models are in output
05c821e 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
azaslavsky added a commit to getsentry/sentry that referenced this issue Jul 14, 2023
@azaslavsky
test(backup): Ensure expected models are in output
14136eb 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
azaslavsky added a commit to getsentry/sentry that referenced this issue Jul 14, 2023
@azaslavsky
test(backup): Ensure expected models are in output
c3bacde 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
azaslavsky added a commit to getsentry/sentry that referenced this issue Jul 14, 2023
@azaslavsky
test(backup): Ensure expected models are in output (#52690)
d4a0ce2 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
armenzg pushed a commit to getsentry/sentry that referenced this issue Jul 17, 2023
@azaslavsky @armenzg
test(backup): Ensure expected models are in output (#52690)
ef18514 
We add a new decorator, `@targets_models`, for the
`.../backup/test_models.py` test suite. The goal is two-fold: for each
individual test, the decorator provides a concise way to express which
models must be included in the output, lest we end up with a test that
passes the equality check, but only because it excluded our target
model(s) altogether. The second goal is to make the set of models being
exercised in the `ModelBackupTests` class easily visible to static
analysis tools like flake8, so that we may later create a check ensuring
that all `__include_in_export__ = True` marked models in this repository
are included in this test suite.

Issue: getsentry/team-ospo#156
Issue: getsentry/team-ospo#160
@azaslavsky azaslavsky changed the title Create flake8 check to ensure that every model is tested Create check to ensure that every model is tested Jul 26, 2023
@azaslavsky
Copy link
Author

This has been completed with the addition of @targets(...) decorators.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@azaslavsky