Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assure UTC time stored in MySQL by using an aware object #119

Merged
merged 6 commits into from
Jan 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ cd docker
./docker-development.sh [check|config|build|start|stop|down]
```

## Docker build for deployment on TEST/STAGE/PROD
## Docker build for deployment on PROD

```
cd hubmap-docker
Expand All @@ -57,13 +57,6 @@ cd hubmap-docker

## Development process

### To release via TEST infrastructure
- Make new feature or bug fix branches from `main` branch (the default branch)
- Make PRs to `main`
- As a codeowner, Zhou (github username `yuanzhou`) is automatically assigned as a reviewer to each PR. When all other reviewers have approved, he will approve as well, merge to TEST infrastructure, and redeploy the TEST instance.
- Developer or someone on the team who is familiar with the change will test/qa the change
- When any current changes in the `main` have been approved after test/qa on TEST, Zhou will release to PROD using the same docker image that has been tested on TEST infrastructure.

### To work on features in the development environment before ready for testing and releasing
- Make new feature branches off the `main` branch
- Make PRs to `dev-integrate`
Expand All @@ -73,4 +66,4 @@ cd hubmap-docker

### Updating API Documentation

The documentation for the API calls is hosted on SmartAPI. Modifying the `uuid-api-spec.yaml` file and commititng the changes to github should update the API shown on SmartAPI. SmartAPI allows users to register API documents. The documentation is associated with this github account: [email protected].
The documentation for the API calls is hosted on SmartAPI. Modifying the `uuid-api-spec.yaml` file and committing the changes to GitHub should update the API shown on SmartAPI. SmartAPI allows users to register API documents. The documentation is associated with this GitHub account: [email protected].
8 changes: 8 additions & 0 deletions sql/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,11 @@ Click "Next". Examine the generated SQL script to assure it is independent of t

Click "Cancel" to abandon the rest of the "forward engineer" process.

## Create as database in which DDL script will execute
A database should be created on the server to receive the DDL commands generated by "forward engineer."

Ways of creating the database vary by platform (e.g. MySQL Workbench, dBeaver, mysql AKA the command line client.) The essential command to run is:
````
CREATE DATABASE `hm_kbkbkb_uuid`
````
Note the backquotes of this command. Also decide whether you want to start with a customized name, and how you will manage that if the database will eventually become Dev, Stage, or Prod.
Binary file modified sql/uuid-api.mwb
Binary file not shown.
Binary file modified sql/uuid-api.pdf
Binary file not shown.
3 changes: 2 additions & 1 deletion sql/uuid-api.sql
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,N
CREATE TABLE IF NOT EXISTS `uuids` (
`UUID` CHAR(32) NOT NULL,
`ENTITY_TYPE` VARCHAR(20) NOT NULL,
`TIME_GENERATED` TIMESTAMP NOT NULL,
`TIME_GENERATED` TIMESTAMP NOT NULL DEFAULT now(),
`USER_ID` VARCHAR(50) NOT NULL,
`USER_EMAIL` VARCHAR(50) NULL DEFAULT NULL,
PRIMARY KEY (`UUID`))
Expand Down Expand Up @@ -77,6 +77,7 @@ CREATE TABLE IF NOT EXISTS `files` (
`PATH` MEDIUMTEXT NULL DEFAULT NULL,
`CHECKSUM` VARCHAR(50) NULL DEFAULT NULL,
`SIZE` BIGINT NULL DEFAULT NULL,
`LAST_MODIFIED` TIMESTAMP NOT NULL DEFAULT now() ON UPDATE now(),
PRIMARY KEY (`UUID`),
CONSTRAINT `files_ibfk_1`
FOREIGN KEY (`UUID`)
Expand Down
7 changes: 5 additions & 2 deletions src/uuid_worker.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from datetime import datetime, timezone
import logging

import mysql.connector.errors
Expand Down Expand Up @@ -611,7 +612,9 @@ def newUUIDs(self, parentIDs, entityType, userId, userEmail, nIds, organ_code=No
# if entityType == 'DONOR':
gen_base_ids = entityType in ID_ENTITY_TYPES
returnIds = []
now = time.strftime('%Y-%m-%d %H:%M:%S')
# See admonition at https://docs.python.org/3/library/datetime.html#datetime.datetime.utcnow to
# retrieve a non-naive UTC time.
awareUTCTimeNow = datetime.now(timezone.utc)
store_file_info = False
if entityType == 'FILE':
store_file_info = True
Expand Down Expand Up @@ -673,7 +676,7 @@ def newUUIDs(self, parentIDs, entityType, userId, userEmail, nIds, organ_code=No
# insRow = (insUuid, ins_app_base_id, entityType, now, userId, userEmail)

# Only set the attributes tuple of committing attributes to the table besides the UUID
insertVals.append((insUuid, entityType, now, userId, userEmail))
insertVals.append((insUuid, entityType, awareUTCTimeNow, userId, userEmail))
if insert_attributes_for_uuid:
insertAttribVals.append((insUuid, ins_app_base_id, submission_ids[n] if gen_submission_ids else None))

Expand Down