Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROX-13692: create final snapshot for tenant db #994

Merged
merged 12 commits into from
May 4, 2023

Conversation

johannes94
Copy link
Contributor

@johannes94 johannes94 commented Apr 27, 2023

Description

This PR changes the central tenant DB deletion for RDS so that it always creates a final snapshot this is required in order to restore the tenant in case it has been deleted accidentally by a customer, an engineer or an error in the code.

Checklist (Definition of Done)

  • Unit and integration tests added
  • Added test description under Test manual
  • Documentation added if necessary (i.e. changes to dev setup, test execution, ...)
  • CI and all relevant tests are passing
  • Add the ticket number to the PR title if available, i.e. ROX-12345: ...
  • Discussed security and business related topics privately. Will move any security and business related topics that arise to private communication channel.
  • Add secret to app-interface Vault or Secrets Manager if necessary

Test manual

Adding test case and running rds tests locally and in CI.

# To run tests locally run:
make db/teardown db/setup db/migrate
make ocm/setup OCM_OFFLINE_TOKEN=<ocm-offline-token> OCM_ENV=development
make verify lint binary test test/integration

@johannes94 johannes94 temporarily deployed to development April 27, 2023 13:26 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 27, 2023 13:26 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 06:56 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 06:57 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 06:58 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 06:58 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 08:24 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 08:24 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 08:47 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 08:47 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 09:18 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 09:18 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development April 28, 2023 09:18 — with GitHub Actions Inactive
@johannes94
Copy link
Contributor Author

/retest

Copy link
Contributor

@kovayur kovayur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any retention rules for the final snapshot?

@johannes94
Copy link
Contributor Author

Do we have any retention rules for the final snapshot?

@kovayur This is an excellent point. I thought it would use the same retention period as automated Snapshots. But turns out "Final Snapshots" are "Manual Snapshots" in AWS terms which don't have a retention mechanism like "System Snapshots". So this means if we do final snapshots, we have to make sure to clean them up at some point.

What do you think about creating a scheduled GitHub Workflow to do this?

@johannes94 johannes94 temporarily deployed to development May 3, 2023 13:02 — with GitHub Actions Inactive
@johannes94 johannes94 force-pushed the jmalsam/ROX-13692-db-final-snapshot branch from aef98bb to 68ea635 Compare May 3, 2023 13:40
@johannes94 johannes94 temporarily deployed to development May 3, 2023 13:40 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 3, 2023 13:40 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 3, 2023 13:40 — with GitHub Actions Inactive
@johannes94 johannes94 force-pushed the jmalsam/ROX-13692-db-final-snapshot branch from 68ea635 to bdcf920 Compare May 3, 2023 17:20
@johannes94 johannes94 temporarily deployed to development May 3, 2023 17:20 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 3, 2023 17:20 — with GitHub Actions Inactive
@johannes94 johannes94 temporarily deployed to development May 3, 2023 17:20 — with GitHub Actions Inactive
@johannes94 johannes94 merged commit 06e11f6 into main May 4, 2023
@johannes94 johannes94 deleted the jmalsam/ROX-13692-db-final-snapshot branch May 4, 2023 05:35
johannes94 added a commit that referenced this pull request May 4, 2023
johannes94 added a commit that referenced this pull request May 4, 2023
@johannes94 johannes94 restored the jmalsam/ROX-13692-db-final-snapshot branch May 4, 2023 09:54
johannes94 added a commit that referenced this pull request May 4, 2023
johannes94 added a commit that referenced this pull request May 9, 2023
* Revert "Revert "ROX-13692: create final snapshot for tenant db (#994)" (#1009)"

This reverts commit e5e9b24.

* do not repeat delete requests if cluster in backing-up status

* ignore final snapshot errors in reconcilation
@johannes94 johannes94 deleted the jmalsam/ROX-13692-db-final-snapshot branch October 23, 2023 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants