Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perform one-time migration of Test, Stage, Prod database for file modification time #116

Closed
kburke opened this issue Jan 12, 2023 · 2 comments
Assignees
Labels
P Pitt dev team

Comments

@kburke
Copy link

kburke commented Jan 12, 2023

Execute the SQL commands below to migrate existing databases for uuid-api #115, rather than dump and reload each database.

-- Add the new column in the DDL modifications for https://github.com/hubmapconsortium/uuid-api/issues/115
ALTER TABLE `hm_uuid`.`files`
ADD `LAST_MODIFIED` timestamp NOT NULL DEFAULT now() ON UPDATE now()
;

-- Change column to align with the DDL modifications for https://github.com/hubmapconsortium/uuid-api/issues/115
ALTER TABLE `hm_uuid`.`uuids` 
ALTER  `TIME_GENERATED` SET DEFAULT (now())
;

-- Revise column values set during ALTER statement above to a reasonable value pulled from UUID.TIME_GENERATED.
-- 2,989,892 rows in 98 seconds on Dev
-- 2,989,805 rows in 98 seconds on Test
-- 2,991,145 rows in 98 seconds on Prod
UPDATE `hm_uuid`.`files`
SET
`LAST_MODIFIED` = (SELECT `TIME_GENERATED` FROM `hm_uuid`.`uuids` where `uuids`.`UUID`=`files`.`UUID`)
;

-- verify times on two tables are the same after UPDATE
SELECT u.TIME_GENERATED, f.LAST_MODIFIED
FROM `hm_uuid`.`uuids` AS u
	INNER JOIN `hm_uuid`.`files` AS f ON u.`UUID`=f.`UUID`
WHERE u.`TIME_GENERATED` <> f.`LAST_MODIFIED`
;

-- verify no other outliers
SELECT *
FROM `hm_uuid`.`files` AS f
WHERE f.`LAST_MODIFIED` IS NULL OR f.`LAST_MODIFIED` > now() OR f.`LAST_MODIFIED` < (SELECT min(time_generated) FROM hm_uuid.uuids);
@kburke kburke added the P Pitt dev team label Jan 12, 2023
@kburke kburke self-assigned this Jan 12, 2023
@kburke kburke changed the title Perform one-time migration of Dev, Test, Stage, Prod database for file modification time Perform one-time migration of Test, Stage, Prod database for file modification time Jan 12, 2023
@kburke
Copy link
Author

kburke commented Jan 17, 2023

UPDATE command results.
-- 2,989,892 rows in 98 seconds on Dev
-- 2,989,805 rows in 98 seconds on Test

Stage still not running it's own uuid-api.

Will backup PROD and take offline for update around lunchtime.

@kburke
Copy link
Author

kburke commented Jan 17, 2023

UPDATE command results while in maintenance mode, announced on #hive-developers Slack channel:
-- 2,991,145 rows in 98 seconds on Prod

@shirey shirey closed this as completed Jan 24, 2023
@shirey shirey added this to Pitt HIVE Jun 7, 2024
@shirey shirey moved this to Done in Pitt HIVE Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P Pitt dev team
Projects
None yet
Development

No branches or pull requests

2 participants