Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[gitpod-db] Don't consider garbage-collected prebuilds as potential bases for incremental prebuilds #9121

Merged
merged 1 commit into from
Apr 5, 2022

Conversation

jankeromnes
Copy link
Contributor

@jankeromnes jankeromnes commented Apr 5, 2022

Description

Don't consider garbage-collected prebuilds as potential bases for incremental prebuilds.

Related Issue(s)

Fixes high cardinality on frequently-run SQL queries.

For example, for the Gitpod repository,

  • The current query returns 16K+ entries
  • The fixed query returns 335 entries

For the official GitLab repository,

  • The current query returns 81K+ entries
  • The fixed query returns 6 entries (😳)

How to test

  1. Run the query in a copy of the production database (e.g. eu-failover) for a few highly-active repositories

Release Notes

[gitpod-db] Don't consider garbage-collected prebuilds as potential bases for incremental prebuilds

Documentation

@@ -754,6 +754,7 @@ export abstract class AbstractTypeORMWorkspaceDBImpl implements WorkspaceDB {
query = query.orderBy("pws.creationTime", "DESC");
query = query.innerJoinAndMapOne("pws.workspace", DBWorkspace, "ws", "pws.buildWorkspaceId = ws.id");
query = query.andWhere("ws.deleted = false");
query = query.andWhere("ws.contentDeletedTime = ''");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will try this in prod (offline) to get a feeling for cardinality...

Copy link
Contributor Author

@jankeromnes jankeromnes Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently trying this in prod-failover with the official GitLab repo (with and without the fix). Will add findings to the PR description.

EDIT: See examples in PR description

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reduced cardinality from 16919 to 337. 💪

@jankeromnes jankeromnes marked this pull request as ready for review April 5, 2022 08:30
@jankeromnes jankeromnes requested a review from a team April 5, 2022 08:30
@github-actions github-actions bot added the team: webapp Issue belongs to the WebApp team label Apr 5, 2022
@geropl
Copy link
Member

geropl commented Apr 5, 2022

@jankeromnes Thx for tackling this! One small nit: Could you add a comment to this line about the reasoning behind cardinality? That would be golden!

@jankeromnes jankeromnes force-pushed the jx/lower-cardinality branch from 87f8858 to d7939b2 Compare April 5, 2022 09:13
@jankeromnes
Copy link
Contributor Author

jankeromnes commented Apr 5, 2022

Thanks @geropl!

One small nit: Could you add a comment to this line about the reasoning behind cardinality? That would be golden!

Sure! Done

Copy link
Member

@geropl geropl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx!

@roboquat roboquat merged commit 81a79af into main Apr 5, 2022
@roboquat roboquat deleted the jx/lower-cardinality branch April 5, 2022 13:28
@roboquat roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployed: webapp Meta team change is running in production deployed Change is completely running in production release-note size/XS team: webapp Issue belongs to the WebApp team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants