Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(backup): Fix query logic for dangling model exports #57737

Conversation

azaslavsky
Copy link
Contributor

There are a small number of models that have no unambiguous direct connection to their relocation scope's root model - these are called "dangling" models. The key factor that defines them, and makes them difficult to handle, is that we cannot use our "query already exported foreign keys" filtering methodology to select only the models relevant to our export targets, because these models have no foreign keys that connect them back to the root of that target. For example, TimeSeriesSnapshot has no foreign keys at all, see: https://tinyurl.com/27z4x6tk.

In cases like the one above, we ended up exporting ALL of the TimeSeriesSnapshots in the database - clearly a very bad outcome when we only want to export those related to a specific org! A better approach is to define custom filtering logic for these models, thereby enabling them to use "adjacent" models in the model graph to select only models that we care about for a given export. In the example above, we query all Incidents filtered down by our previous exports to get a sneak-peek at the set of IncidentSnapshots (a set that is currently empty due to going in reverse dependency order), then use that information to work backwards to grab the TimeSeriesSnapshots we need.

The upshot is that this commit introduces a generic method for constructing filtered queries for a specific model, the overridable query_for_relocation_export.

Issue: getsentry/team-ospo#203

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 6, 2023
@azaslavsky azaslavsky force-pushed the azaslavsky/backup/refine_dangling branch from 288e0fd to 11fcf1d Compare October 7, 2023 00:12
@azaslavsky azaslavsky force-pushed the azaslavsky/backup/fix_dangling_exports branch from a79ef11 to 0ee6c61 Compare October 7, 2023 00:12
Base automatically changed from azaslavsky/backup/refine_dangling to azaslavsky/backup/unique_coverage October 10, 2023 21:44
@azaslavsky azaslavsky marked this pull request as ready for review October 11, 2023 00:26
@azaslavsky azaslavsky requested review from a team October 11, 2023 00:26
@azaslavsky azaslavsky merged this pull request into azaslavsky/backup/unique_coverage Oct 11, 2023
44 checks passed
@azaslavsky azaslavsky deleted the azaslavsky/backup/fix_dangling_exports branch October 11, 2023 00:27
@github-actions github-actions bot locked and limited conversation to collaborators Oct 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants