Upload micro service design
The upload micro service is one of the core services that enables both HSLynk and its customers to be upload documents. Following are few design considerations for the upload micro service.
Each upload belongs to a upload type. This is a pre-defined type set up by HSLynk and this type determines how a document will be stored or managed. Each upload type determines the mimetypes (extensions of the files) that are allowed and the total size that is allowed for each upload and the no of files can be part of a each upload.
- Only PDF / DOCs are allowed
- Maximum of 3 files will be allowed in each upload.
- Maximum size of each file cannot exceed 20 MB
- Total size of all the files cannot exceed 50 MB.
- Default storage destination is S3.
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique Identifier for the upload |
| UPLOAD_TYPE_CODE | CODE for the upload type , CONSENT_UPLOAD |
| DESCRIPTION | Description about the upload Type |
| MAX_SIZE | Maximum size that is allowed for the upload type in MB |
| DESTINATION_TYPE | Value could be S3 |
| DESTINATION | The root location for the storage (S3 Bucket name |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique Identifier for the upload |
| UPLOAD_TYPE_ID | Upload Type Id |
| MIME_TYPE | The mime type that is allowed for the upload type |
| MAX_SIZE | Maximum size that is allowed for this mime type , for this upload type |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
Each upload can contain multiple files (as per the pre-defined configuration for upload type). Upload micro service should track the number of files that are uploaded for each upload along with the status for each and individual upload. The following design enables to track the status of uploads . Also, an upload can happen for any micro service . For instance, upload can be performed by global-api (for consent uploads) or from reporting micro service to upload the report, after its generated. So, the source of the upload should be maintained by the upload micro service.
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique Identifier for the upload |
| UPLOAD_SOURCE | Name of the micro service that performed the upload |
| UPLOAD_TYPE_ID | Type of the upload |
| NO_OF_FILES | The number of files that are uploaded with this request |
| TOTAL_SIZE | Total size of the files that are uploaded with this request. |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique identifier for the upload line entity |
| MIME_TYPE | Mime type of the file that is uploaded |
| FILE_SIZE | Size of the file |
| FILE_ORIGINAL_NAME | Original name of the file |
| FILE_NAME | Name of the file , if we need to maintain a specific naming convention |
| UPLOAD_LOCATION | Path where the file is stored |
| UPLOAD_HEADER_ID | Upload header associated with this upload item |
| UPLOAD_STATUS | Status of the upload - success or failure |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
HSLynk exposes APIs for uploading various documents and to capture the metadata associated with the upload. Each customer can be provided granted access only for few upload types. For instance, few clients may not be allowed to perform CLIENT_FILE_UPLOADs but just CONSENT_UPLOAD. This can be determined based on the contract between HSLynk and the customer.
Following DB design describes how the uploads are managed at project group level and how the meta data will be stored for several uploads.
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique identifier for the upload line entity |
| PROJECT_GROUP_ID | Project group |
| UPLOAD_TYPE_ID | Upload Type Id |
| DESTINATION_TYPE | This overrides the default destination type defined in UPLOAD_TYPE configuration |
| DESTINATION | This overrides the default destination defined in UPLOAD_TYPE configuration |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
| FIELD | DESCRIPTION |
|---|---|
| UUID | Unique identifier for the upload line entity |
| PROJECT_GROUP_ID | Project group |
| UPLOAD_TYPE_ID | Upload Type Id |
| UPLOAD_HEADER_ID | This determines how the details of the files is fetched |
| UPLOAD_METADATA | Meta data information for the upload - used for retrieval / search purposes |
| UPLOAD_KEY | key search identifier for the upload (for instance clientId) |
| UPLOAD_VALUE | value of the key search identifier for the upload ex: UUID 68803de3-b54f-438d-b7da-4f482409f671 |
| DATE_CREATED | Audit fields |
| DATE_UPDATED | Audit fields |
| CREATED_BY | Audit fields |
| UPDATED_BY | Audit fields |
Once the documents are uploaded we can set up rules to notify the concerned teams. For instance, once a report is generated and uploaded, the targeted users can be sent an e-mail with the link to the report.