Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: added status table cleanup logic before backup with code refactor #2542

Merged
merged 9 commits into from
Oct 21, 2022

Conversation

saurav-malani
Copy link
Contributor

@saurav-malani saurav-malani commented Oct 10, 2022

Description

  1. Refactored backup DS logic to reduce code complexity & improve readability.
  2. added logic to clean-up backup status table to delete status corresponding to attempt older than last 2.

Notion Ticket

https://www.notion.so/rudderstacks/backup-refactor-Delete-stale-job-statuses-before-backing-up-946496e6688749b790122d20e6c93c7d

@codecov
Copy link

codecov bot commented Oct 10, 2022

Codecov Report

Base: 44.00% // Head: 43.99% // Decreases project coverage by -0.00% ⚠️

Coverage data is based on head (1309e30) compared to base (09dba21).
Patch coverage: 74.95% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2542      +/-   ##
==========================================
- Coverage   44.00%   43.99%   -0.01%     
==========================================
  Files         187      188       +1     
  Lines       39086    39203     +117     
==========================================
+ Hits        17198    17248      +50     
- Misses      20807    20850      +43     
- Partials     1081     1105      +24     
Impacted Files Coverage Δ
jobsdb/backup.go 74.90% <74.90%> (ø)
jobsdb/jobsdb.go 65.44% <100.00%> (-2.63%) ⬇️
warehouse/schema.go 47.82% <0.00%> (-1.16%) ⬇️
enterprise/reporting/reporting.go 9.77% <0.00%> (+1.43%) ⬆️
enterprise/reporting/setup.go 52.38% <0.00%> (+14.28%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@saurav-malani saurav-malani changed the title Chore.backup stale jobs cleanup Chore: backup stale job status cleanup before backup & minor refactor. Oct 10, 2022
@saurav-malani saurav-malani changed the title Chore: backup stale job status cleanup before backup & minor refactor. chore: backup stale job status cleanup before backup & minor refactor. Oct 10, 2022
@saurav-malani saurav-malani marked this pull request as ready for review October 11, 2022 11:06
@saurav-malani saurav-malani marked this pull request as draft October 11, 2022 14:04
@saurav-malani saurav-malani changed the title chore: backup stale job status cleanup before backup & minor refactor. chore: added status table cleanup logic before backup with code refactor Oct 13, 2022
@saurav-malani saurav-malani marked this pull request as ready for review October 13, 2022 10:34
@atzoum atzoum force-pushed the chore.backupStaleJobsCleanup branch from 230ef87 to 07280a7 Compare October 17, 2022 07:00
Copy link
Contributor

@atzoum atzoum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @saurav-malani!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants