Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-architect lookup table (yet again!) #863

Open
bpblanken opened this issue Aug 5, 2024 · 0 comments
Open

Re-architect lookup table (yet again!) #863

bpblanken opened this issue Aug 5, 2024 · 0 comments

Comments

@bpblanken
Copy link
Collaborator

bpblanken commented Aug 5, 2024

Re-architect the lookup table:

  • One large table with project_guids kept as is, and project_stats migrated to an array of structs (rather than an array of arrays), where each struct is the sum of the ref_samples, het_samples, hom_samples for a project.
  • A lookup table for every project, with a family_guids global and a project_stats array of structs, each index representing a family.
  • Pipeline logic to update both the project lookup table and the large lookup table whenever a project is loaded.
  • Airflow logic to make sure that the lookup table but not the project lookup tables are copied to ssd.
  • Seqr logic to read the new lookup table format.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant