Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[db] Improve performance of DBUserStorageResource.update(...) #3151

Merged
merged 2 commits into from
Feb 15, 2021
Merged

[db] Improve performance of DBUserStorageResource.update(...) #3151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9b4e22f
[db] UserStorageResource: make userId char(36) instead of varchar(255)
geropl Feb 8, 2021
c05795b
[db] Use INSERT INTO ... ON DUPLICATE KEY UPDATE for user storage
geropl Feb 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions components/gitpod-db/src/typeorm/migration/1612781029090-UserStorageUserIdChar36.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import {MigrationInterface, QueryRunner} from "typeorm";

export class UserStorageUserIdChar1612781029090 implements MigrationInterface {

public async up(queryRunner: QueryRunner): Promise<any> {
await queryRunner.query("ALTER TABLE d_b_user_storage_resource MODIFY userId char(36);");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it a problem?
changing userId varchar(255) to fixed char(36) sound like there is some risk.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it a problem?

Yes, it is for the lookup performance. The DB cannot assume that the 219 characters that are always blank. Ideally they all end up in one leaf of the Red-Black-tree but one never know. Also, it blows up the memory used up by the index.

changing userId varchar(255) to fixed char(36) sound like there is some risk.

There is no risk to break something as we never used anything else than UUIDs (36 characters), and we're maintaining that limit with the current id change.
During deployment however this might be a problem as it might take some time because it requires re-creating indexes.

}

public async down(queryRunner: QueryRunner): Promise<any> {
}

}
22 changes: 9 additions & 13 deletions components/gitpod-db/src/typeorm/user-storage-resources-db-impl.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ import { injectable, inject } from "inversify";
import { TypeORM } from "./typeorm";
import { UserStorageResourcesDB } from "../user-storage-resources-db";
import { DBUserStorageResource } from "./entity/db-user-storage-resource";
import { log } from '@gitpod/gitpod-protocol/lib/util/logging';

@injectable()
export class TypeORMUserStorageResourcesDBImpl implements UserStorageResourcesDB {
Expand All @@ -31,19 +30,16 @@ export class TypeORMUserStorageResourcesDBImpl implements UserStorageResourcesDB
}

async update(userId: string, uri: string, content: string): Promise<void> {
// docs: https://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
const repo = await this.getUserStorageResourceRepo();
let resource = await this.getResource(userId, uri);
if (resource) {
log.info({ userId }, 'updating resource', { uri });
await repo.update(resource, { content });
} else {
log.info({ userId }, 'saving resource', { uri });
resource = new DBUserStorageResource();
resource.userId = userId;
resource.uri = uri;
resource.content = content;
await repo.save(resource);
}
await repo.query(`
INSERT INTO d_b_user_storage_resource
(userId, uri, content)
VALUES
(?, ?, ?)
ON DUPLICATE KEY UPDATE
content = VALUES(content);
`, [ userId, uri, content ]);
}

async deleteAllForUser(userId: string): Promise<void> {
Expand Down