Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Epic] Improving Reap Performance of History Lookup Tables #4870

Closed
tamirms opened this issue May 4, 2023 · 0 comments · Fixed by #5405
Closed

[Epic] Improving Reap Performance of History Lookup Tables #4870

tamirms opened this issue May 4, 2023 · 0 comments · Fixed by #5405
Labels
cdp-horizon-scrum horizon
Milestone
platform sprint 50

Comments

@tamirms
Copy link
Contributor

tamirms commented May 4, 2023

This is an epic tracking the overall implementation of the reaping solution proposed in Improving Reap Performance of History Lookup Tables

  1. Simplify history.CreateAccounts() to avoid select statement #4864
  2. Estimate how long it would take to run ReapLookupTables on all rows #4865
  3. Add migration to add latest_referenced_ledger to history lookup tables #4866
  4. Update the latest_referenced_ledger column during ingestion #4867
  5. Reap lookup tables in DeleteUnretainedHistory #4868
  6. Test performance of new reaping solution #4869
@tamirms tamirms moved this from Backlog to Next Sprint Proposal in Platform Scrum May 16, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Jun 13, 2023
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum Sep 12, 2023
@urvisavla urvisavla moved this from Next Sprint Proposal to Current Sprint in Platform Scrum Oct 10, 2023
@urvisavla urvisavla moved this from Current Sprint to Next Sprint Proposal in Platform Scrum Nov 2, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Nov 2, 2023
@sreuland sreuland added this to the platform sprint 49 milestone Jul 2, 2024
@sreuland sreuland moved this from Backlog to To Do in Platform Scrum Jul 2, 2024
@sreuland sreuland mentioned this issue Jul 3, 2024
Closed
8 tasks
@tamirms tamirms moved this from To Do to In Progress in Platform Scrum Jul 30, 2024
@tamirms tamirms moved this from In Progress to Needs Review in Platform Scrum Aug 27, 2024
@tamirms tamirms mentioned this issue Aug 28, 2024
Merged
7 tasks
@github-project-automation github-project-automation bot moved this from Needs Review to Done in Platform Scrum Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cdp-horizon-scrum horizon
Projects
Platform Scrum
Status: Done
Milestone
platform sprint 50
Development

Successfully merging a pull request may close this issue.

services/horizon/internal/ingest: reap lookup tables without blocking ingestion tamirms/go
4 participants
@Shaptic @tamirms @mollykarcher @sreuland