Feat/bulk/all #2020
Merged
Feat/bulk/all #2020
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
gweiying
force-pushed
the
feat/bulk/all
branch
2 times, most recently
from
e168edb to
d798d54
Compare
gweiying
force-pushed
the
feat/bulk/all
branch
2 times, most recently
from
c7336d8 to
e733293
Compare
* chore: create new CreateTypeButton for CreateLinkForm * chore: swap isFile state boolean to createType enum * chore: add CreateType BULK button * feat: add BulkUpload form * chore: fix button disabling check * chore: add download sample csv functionality * fix: bugs * chore: add user actions for bulk create * fix: misspelling and children nesting * chore: remove isFile from LinkFormStyles * chore: try to fix the css
* chore: refactor FileInputField component * fix: remove unused styles
gweiying
force-pushed
the
feat/bulk/all
branch
from
e733293 to
601171e
Compare
* chore: add e2e tests for bulk creation * chore: clean up unused code Co-authored-by: halfwhole <[email protected]> * chore: add small wait time to stop test flakiness Co-authored-by: halfwhole <[email protected]>
…2007) * Feat/link audit to include tags (#1950) * feat: added tags table and many to many rel to urls * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: merge link-audit-to-include-tags with develop * feat: link-audit-to-include-tags * feat: search for tags in lower cap (#1973) * feat: search for tags in lower cap * feat: search for tags in lower cap * feat(tagging): add tags to link audit history (#1974) * chore(datadog): refactor custom metric names (#1965) * feat(tagging): add frontend tagging on create new link form (#1919) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add dropdown for tags on user page search bar (#1934) * feat(tagging): add dropdown for tags on user page search bar * feat(tagging): add link icon on search dropdown * feat(tagging): refactor link icon and add tag icon on search dropdown * feat(tagging): revert link icon to original size * build(deps): bump winston from 3.3.3 to 3.8.1 (#1889) * build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890) * feat(tagging): add frontend API integrations on create new link form (#1971) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add frontend API integrations for create new link form * chore(tagging): update tag validation tests for underscores * chore(tagging): refactor tags to use redux instead of useState * Feat/add tag to link audit frontend (#1975) * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat(tagging): add editing of link tags (#1976) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add frontend API integrations for create new link form * chore(tagging): update tag validation tests for underscores * chore(tagging): refactor tags to use redux instead of useState * feat(tagging): add editing of link tags * feat(tagging): use manual save button for editing link tags * fix: return successful response when no URLs are found (#1979) * fix(tagging): reset tags state after link creation (#1982) * feat(tagging): add tags to links on user page (#1972) * feat(tagging): add tags to links on user page * feat(tagging): apply search when clicking on tags * chore(tagging): remove duplicate identifier for tags * fix: revise findUrlsForUser to return urls in correct order (#1981) * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: fix urlMapper to correctly check for empty tagStrings before parsing (#1989) * chore: remove disallowed file extensions (#1985) * Fix/link tagging for file upload (#1986) * fix: add tags when creating new link from file * chore: update sentry message * fix: wrap backend errors in json messages (#1991) * fix(tagging): serialize single tags for file uploads (#1990) * fix(tagging): serialize single tags for file uploads * chore(tagging): add comment for preprocessing file tags * feat: add job and job item models * feat: add jobmanagementservice * chore: fix conflicts * chore: rename JobStatusEnum to JobItemStatusEnum * chore: remove JobItemStatusEnum - Ready * chore: replaced isSuccess to getJobStatus in JobManagementService Co-authored-by: thanhdatle <[email protected]> Co-authored-by: halfwhole <[email protected]>
* chore: add icons * feat: add StatusBarComponent * chore: fix alignment * chore: add actions and reducers * chore: fix JSX linting error with eslint * chore: attach actions and reducers to StatusBar component * chore: display QR code inprogress after bulk upload (#2041) * fix: combine class styles * chore: add docs inline for disabling eslint no-undef
* chore: refactor existing DownloadButton to separate component from logic * chore: add CloseButton as a reusable component * chore: add DownloadBulkButton to StatusBar * chore: styling * chore: rename icon
* feat: add lambda deployment and sample code * feat: add qrcode and zip lambda code * feat: add createCsv, uploadToS3 and sendsqsMessage logic * feat: update handler logic * feat: stream files to s3 bulk-qrcode-generation * chore: add assets * chore: add sns publish method * chore: refactor code * chore: temp remove unnecessary ci build commadns * chore: fix qrcode url * chore: import services instead of functions * Revert "chore: temp remove unnecessary ci build commadns" This reverts commit f4474a19e25a82aa0205fdc5bdb7c7ef8562824a. * chore: set memorySize, ephemeralStorageSize, timeout on serverless * fix: clean up dir on err * chore: use archiver directory false * chore: remove streamToS3 from exports * chore: set domain in env variables Co-authored-by: Alexis <[email protected]>
* chore: remove JobItemType from JobItem * chore: remove JobItemStatusEnum Ready * chore: refactor Job module to group files and remove mappers Co-authored-by: Jim Engay <[email protected]>
thanhdatle
reviewed
Nov 22, 2022
| getJobStatus: (jobId: number) => Promise<JobItemStatusEnum> = async ( | ||
| jobId, | ||
| ) => { | ||
| const jobItems = await this.findJobItemsByJobId(jobId) |
There was a problem hiding this comment.
this can just be a single SQL query like: look for job items with jobId=jobId AND status != 'SUCCESS'
but we are adding 1 more column to Job table, this is not relevant anymore
* feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: fix env variable documentation Co-authored-by: Jim Engay <[email protected]>
* feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: typo in docs Co-authored-by: halfwhole <[email protected]> * chore: refactor job status computation logic Co-authored-by: halfwhole <[email protected]> * chore: fix lint errors after merging suggestions * chore: attach only jobId to req body in updateJobItem * chore: change enum from Failed to Failure * chore: add unique constraint on jobItemId Co-authored-by: Jim Engay <[email protected]> Co-authored-by: halfwhole <[email protected]>
* feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: typo in docs Co-authored-by: halfwhole <[email protected]> * chore: refactor job status computation logic Co-authored-by: halfwhole <[email protected]> * chore: fix lint errors after merging suggestions * chore: attach only jobId to req body in updateJobItem * chore: change enum from Failed to Failure * fix: add env variable to feature flag job creation * chore: add repository methods to retrieve jobs * chore: add service methods for long polling and retrieving job information * chore: add endpoints for users to retrieve their latest job and updated job status * chore: add tests * fix: add env variables to docker-compose * chore: add env variable for bulk download bucket * chore: fix tests * chore: fix typos in test * chore: fix tests * chore: change job status endpoint to get * chore: rename jobItemIds to jobItemUrls * chore: update error statuses * chore: fix lint errors Co-authored-by: Jim Engay <[email protected]> Co-authored-by: halfwhole <[email protected]>
* feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: add env variable to feature flag job creation * chore: add repository methods to retrieve jobs * chore: add service methods for long polling and retrieving job information * chore: add endpoints for users to retrieve their latest job and updated job status * chore: add tests * fix: add env variables to docker-compose * chore: add env variable for bulk download bucket * fix: env variable naming in docker-compose * chore: fix tests * chore: fix tests * fix: status bar should appear even if no urls * chore: pass download urls to Download QR code button * chore: add callback variable to actions and reducers * chore: remove redundant bulkQRCodesStarted action * chore: add client-side logic to retrieve user job * chore: add component logic for status bar * chore: fix end to end tests * fix: uncomment sqs.sendService function * chore: update api call on client for job update Co-authored-by: Jim Engay <[email protected]>
* chore: make sender domain optional * chore: add email sending for job completion * chore: fix local mailDev logging for non-OTP emails * chore: fix types * chore: fix tests * Fix/bulk/all errors (#2102) * chore: add dogstatsd variables * fix: add promise to papaparse and refactor bulk validation * chore: fix bulk upload error styling and handling * chore: fix tests * chore: fix types * chore: add enums to shared folder * chore: add email sending logic into email service
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
False alarm, development secret |
gweiying
force-pushed
the
feat/bulk/all
branch
from
c3560e8 to
4039da8
Compare
* chore: fix copy * chore: update copy Co-authored-by: halfwhole <[email protected]> Co-authored-by: halfwhole <[email protected]>
* chore: refactor guard to use api key access * chore: remove unused env variables and files * chore: add tests * chore: remove admin default email
halfwhole
approved these changes
Dec 5, 2022
| |BULK_QR_CODE_BATCH_SIZE|No|Maximum batch size of QR codes to generate in a single Lambda run. Defaults to 1000| | ||
| |BULK_QR_CODE_BUCKET_URL|No|Link to download QR codes from| | ||
| |ACTIVATE_BULK_QR_CODE_GENERATION|No|Whether to start Lambda for bulk QR code generation or not. Defaults to false| | ||
| |REPLICA_URI|Yes|The postgres connection string, e.g. `postgres://postgres:postgres@postgres:5432/postgres`| |
There was a problem hiding this comment.
REPLICA_URI here appears to be a duplicate
src/server/api/index.ts
Outdated
| const { userId } = req.body | ||
| const isAdmin = await apiKeyAuthService.isAdmin(userId) | ||
| if (!isAdmin) { | ||
| res.unauthorized() |
There was a problem hiding this comment.
nit: an error message for this could be helpful!
src/client/app/util/download.ts
Outdated
| } | ||
|
|
||
| export const downloadSampleBulkCsv = () => { | ||
| const headers = 'Original links to be shortened' |
There was a problem hiding this comment.
nit: could this be taken from src/shared/constants.ts?
src/server/services/email.ts
Outdated
|
|
||
| <p>Download QR codes for your links (PNG): ${downloadLinks.map( | ||
| (downloadLink) => | ||
| `<a href="${downloadLink}/generated_png.zip" target="_blank">here </a>`, |
There was a problem hiding this comment.
nit: would be nice if the names generated.csv or generated_[png/svg].zip could be shared across FE/BE/serverless, but if not possible it's fine too!
There was a problem hiding this comment.
Not including serverless, but for FE and BE for now
gweiying
added a commit
that referenced
this pull request
Dec 6, 2022
* feat: bulk frontend modal (#1958) * chore: create new CreateTypeButton for CreateLinkForm * chore: swap isFile state boolean to createType enum * chore: add CreateType BULK button * feat: add BulkUpload form * chore: fix button disabling check * chore: add download sample csv functionality * fix: bugs * chore: add user actions for bulk create * fix: misspelling and children nesting * chore: remove isFile from LinkFormStyles * chore: try to fix the css * chore: refactor FileInputField component (#1960) * chore: refactor FileInputField component * fix: remove unused styles * feat: connect bulk csv upload to backend * fix: tag serialization and deserialization on FormData * fix: renamed button * chore: add e2e tests for bulk creation (#2022) * chore: add e2e tests for bulk creation * chore: clean up unused code Co-authored-by: halfwhole <[email protected]> * chore: add small wait time to stop test flakiness Co-authored-by: halfwhole <[email protected]> * feat(bulk-backend): Job and JobItem DB model, repository and service (#2007) * Feat/link audit to include tags (#1950) * feat: added tags table and many to many rel to urls * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create tag API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: create url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: update url API to include tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: search-link-with-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: link-audit-to-include-tags * feat: merge link-audit-to-include-tags with develop * feat: link-audit-to-include-tags * feat: search for tags in lower cap (#1973) * feat: search for tags in lower cap * feat: search for tags in lower cap * feat(tagging): add tags to link audit history (#1974) * chore(datadog): refactor custom metric names (#1965) * feat(tagging): add frontend tagging on create new link form (#1919) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add dropdown for tags on user page search bar (#1934) * feat(tagging): add dropdown for tags on user page search bar * feat(tagging): add link icon on search dropdown * feat(tagging): refactor link icon and add tag icon on search dropdown * feat(tagging): revert link icon to original size * build(deps): bump winston from 3.3.3 to 3.8.1 (#1889) * build(deps): bump express-fileupload from 1.2.1 to 1.4.0 (#1890) * feat(tagging): add frontend API integrations on create new link form (#1971) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add frontend API integrations for create new link form * chore(tagging): update tag validation tests for underscores * chore(tagging): refactor tags to use redux instead of useState * Feat/add tag to link audit frontend (#1975) * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat: added TagList component by using mui/Chip to show tags in Link Audit * feat(tagging): add editing of link tags (#1976) * feat(tagging): add frontend tagging on create new link form * chore: refactor autocomplete and address comments * chore: refactor text field height and add disable submit checks * chore: refactor max tag length in helper error message * chore(tagging): refactor max number of tags per link * chore(tagging): update tag validation tests for underscores * feat(tagging): add frontend API integrations for create new link form * chore(tagging): update tag validation tests for underscores * chore(tagging): refactor tags to use redux instead of useState * feat(tagging): add editing of link tags * feat(tagging): use manual save button for editing link tags * fix: return successful response when no URLs are found (#1979) * fix(tagging): reset tags state after link creation (#1982) * feat(tagging): add tags to links on user page (#1972) * feat(tagging): add tags to links on user page * feat(tagging): apply search when clicking on tags * chore(tagging): remove duplicate identifier for tags * fix: revise findUrlsForUser to return urls in correct order (#1981) * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: revise findUrlsForUser to return urls in correct order * fix: fix urlMapper to correctly check for empty tagStrings before parsing (#1989) * chore: remove disallowed file extensions (#1985) * Fix/link tagging for file upload (#1986) * fix: add tags when creating new link from file * chore: update sentry message * fix: wrap backend errors in json messages (#1991) * fix(tagging): serialize single tags for file uploads (#1990) * fix(tagging): serialize single tags for file uploads * chore(tagging): add comment for preprocessing file tags * feat: add job and job item models * feat: add jobmanagementservice * chore: fix conflicts * chore: rename JobStatusEnum to JobItemStatusEnum * chore: remove JobItemStatusEnum - Ready * chore: replaced isSuccess to getJobStatus in JobManagementService Co-authored-by: thanhdatle <[email protected]> Co-authored-by: halfwhole <[email protected]> * chore: add error message for invalid date string and csv header * Feat/bulk/frontend/status bar (#2040) * chore: add icons * feat: add StatusBarComponent * chore: fix alignment * chore: add actions and reducers * chore: fix JSX linting error with eslint * chore: attach actions and reducers to StatusBar component * chore: display QR code inprogress after bulk upload (#2041) * fix: combine class styles * chore: add docs inline for disabling eslint no-undef * Feat/bulk/frontend/status bar button (#2047) * chore: refactor existing DownloadButton to separate component from logic * chore: add CloseButton as a reusable component * chore: add DownloadBulkButton to StatusBar * chore: styling * chore: rename icon * Feat/bulk/all lambda (#2033) * feat: add lambda deployment and sample code * feat: add qrcode and zip lambda code * feat: add createCsv, uploadToS3 and sendsqsMessage logic * feat: update handler logic * feat: stream files to s3 bulk-qrcode-generation * chore: add assets * chore: add sns publish method * chore: refactor code * chore: temp remove unnecessary ci build commadns * chore: fix qrcode url * chore: import services instead of functions * Revert "chore: temp remove unnecessary ci build commadns" This reverts commit f4474a19e25a82aa0205fdc5bdb7c7ef8562824a. * chore: set memorySize, ephemeralStorageSize, timeout on serverless * fix: clean up dir on err * chore: use archiver directory false * chore: remove streamToS3 from exports * chore: set domain in env variables Co-authored-by: Alexis <[email protected]> * Feat/bulk/all cleanup unused (#2082) * chore: remove JobItemType from JobItem * chore: remove JobItemStatusEnum Ready * chore: refactor Job module to group files and remove mappers Co-authored-by: Jim Engay <[email protected]> * Feat/bulk/all create job (#2083) * feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: fix env variable documentation Co-authored-by: Jim Engay <[email protected]> * Feat/bulk/all update job (#2085) * feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: typo in docs Co-authored-by: halfwhole <[email protected]> * chore: refactor job status computation logic Co-authored-by: halfwhole <[email protected]> * chore: fix lint errors after merging suggestions * chore: attach only jobId to req body in updateJobItem * chore: change enum from Failed to Failure * chore: add unique constraint on jobItemId Co-authored-by: Jim Engay <[email protected]> Co-authored-by: halfwhole <[email protected]> * Feat/bulk/all get job server (#2092) * feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: typo in docs Co-authored-by: halfwhole <[email protected]> * chore: refactor job status computation logic Co-authored-by: halfwhole <[email protected]> * chore: fix lint errors after merging suggestions * chore: attach only jobId to req body in updateJobItem * chore: change enum from Failed to Failure * fix: add env variable to feature flag job creation * chore: add repository methods to retrieve jobs * chore: add service methods for long polling and retrieving job information * chore: add endpoints for users to retrieve their latest job and updated job status * chore: add tests * fix: add env variables to docker-compose * chore: add env variable for bulk download bucket * chore: fix tests * chore: fix typos in test * chore: fix tests * chore: change job status endpoint to get * chore: rename jobItemIds to jobItemUrls * chore: update error statuses * chore: fix lint errors Co-authored-by: Jim Engay <[email protected]> Co-authored-by: halfwhole <[email protected]> * Feat/bulk/all get job client (#2093) * feat: add sqs client and service * feat: create job and job items and sendSqsmessage in bulk controller * chore: move qrCode job creation into JobController * chore: generalize sqs message sending * chore: add sqs timeout * chore: fix missing region * chore: create qrBatchSize and sqsRegion env variable * chore: fix error logging and type * chore: fix bulk tests * chore: fix lambda params * chore: add JobItemId to job_items model for easy update * chore: refactor logic to update jobItemStatus * feat: expose endpoint to update jobItem * chore: use env var for lambda hash value * chore: use http post request to send completion callback * chore: fix validation * chore: add tests * chore: fix error catching for lambda * chore: add status to job table * chore: add logic to update job after job item callback * chore: fix tests * chore: fix tests * fix: call next to pass control * fix: add env variable to feature flag job creation * chore: add repository methods to retrieve jobs * chore: add service methods for long polling and retrieving job information * chore: add endpoints for users to retrieve their latest job and updated job status * chore: add tests * fix: add env variables to docker-compose * chore: add env variable for bulk download bucket * fix: env variable naming in docker-compose * chore: fix tests * chore: fix tests * fix: status bar should appear even if no urls * chore: pass download urls to Download QR code button * chore: add callback variable to actions and reducers * chore: remove redundant bulkQRCodesStarted action * chore: add client-side logic to retrieve user job * chore: add component logic for status bar * chore: fix end to end tests * fix: uncomment sqs.sendService function * chore: update api call on client for job update Co-authored-by: Jim Engay <[email protected]> * Feat/bulk/all email send (#2099) * chore: make sender domain optional * chore: add email sending for job completion * chore: fix local mailDev logging for non-OTP emails * chore: fix types * chore: fix tests * Fix/bulk/all errors (#2102) * chore: add dogstatsd variables * fix: add promise to papaparse and refactor bulk validation * chore: fix bulk upload error styling and handling * chore: fix tests * chore: fix types * chore: add enums to shared folder * chore: add email sending logic into email service * chore: fix copy (#2103) * chore: fix copy * chore: update copy Co-authored-by: halfwhole <[email protected]> Co-authored-by: halfwhole <[email protected]> * Feat/bulk/all api guard (#2104) * chore: refactor guard to use api key access * chore: remove unused env variables and files * chore: add tests * chore: remove admin default email * chore: fix email copy (#2107) * chore: get header from shared folder * chore: create shared types for bulk download file names * chore: remove duplicate REPLICA_URI * chore: add unauthorized error message Co-authored-by: Jim Engay <[email protected]> Co-authored-by: halfwhole <[email protected]> Co-authored-by: Jim Vincent Andes Engay <[email protected]> Co-authored-by: thanhdatle <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR collates all the changes done for the bulk functionality