refactor: use sequelize-typescript #374

prestonlimlianjie · 2022-03-03T11:57:40Z

This PR adds sequelize-typescript to allow us to

type our database models so that these types can be used elsewhere in the codebase, and
express our model files in a declarative manner for better clarity

Dependency changes

sequelize-typescript
reflect-metadata - peer dep of sequelize-typescript
ts-node
@swc/core - third party transpiler that we pass to ts-node

Dev-dependency changes

@types/node - peer dep of sequelize-typescript
@types/validator - peer dep of sequelize-typescript
ts-node-dev - equivalent of ts-node but with hot reloading

As described in https://www.npmjs.com/package/sequelize-typescript

Also specified in sequelize-typescript docs

we forc ets node to require tsconfig/paths to allow for path aliasing. this is because typescript requires that the runtime (node/ts-node) does the path aliasing, rather than ts. hence, requiring tsconfig/paths allows us to use path alias at runtime. additionally, we add a transpileOnly option. This prevents ts-node from typechecking and transpiles to js only, which increases its transpilation speed. another option, swc, was also added. this option uses the rust swc compiler for ts, which is significantly faster than the other compilers. this should speed up transpilation speed.

sequelize (the class) is not a default export and has been changed accordingly. also, logger has been changed from morgan to winston (the one in our package) as morgan is less powerful than winston

…in server

seaerchin · 2022-03-10T03:14:01Z

we have a dependency cycle in our db models, with each model transitively requiring the other 2 - is this intended and if not, should we fix it?

seaerchin · 2022-03-11T05:57:07Z

i think we need to rollback the changes to config.js - sequelize only accepts .js or .json extensions. this isn't a problem when initializing the database, as we run this through sequelize-typescript but it does mean that when we run migrations, sequelize-cli will fail (can't recognise the .ts extension).

if we run npx sequelize db:migrate,

it fails beacuse it can't find ./database/config.js which is specified in .sequelizerc (file is in .ts)
if we change the extension to .ts, it fails due to the point mentioned above
we have to rollback to .js then in our .ts file that consumes the config, cast to the correct type (SequelizeOptions). json isn't a good choice here because we depend on env vars for the db config

…sues

prestonlimlianjie · 2022-03-14T06:59:56Z

i think we need to rollback the changes to config.js - sequelize only accepts .js or .json extensions. this isn't a problem when initializing the database, as we run this through sequelize-typescript but it does mean that when we run migrations, sequelize-cli will fail (can't recognise the .ts extension).

if we run npx sequelize db:migrate,

it fails beacuse it can't find ./database/config.js which is specified in .sequelizerc (file is in .ts)

if we change the extension to .ts, it fails due to the point mentioned above

we have to rollback to .js then in our .ts file that consumes the config, cast to the correct type (SequelizeOptions). json isn't a good choice here because we depend on env vars for the db config

~~@seaerchin~~
~~I've proposed a way to resolve this issue with these commits:~~
~~- 55ccb9b~~
~~- 93c37a2~~

~~The commits~~
~~- Revert the change in .sequelizerc back to using the ts config file, i.e. database/config.ts~~
~~- Add an npm run build script to compile the entire repo (including database/config.ts) into the build/ folder~~
~~- Update .sequelizerc to use the compiled config file, i.e. build/database/config.js~~
~~- Add an npm run database:migrate script that calls npm run build and then sequelize db:migrate~~

Your suggestion is the cleanest - let's go with that

prestonlimlianjie · 2022-03-14T08:18:35Z

we have a dependency cycle in our db models, with each model transitively requiring the other 2 - is this intended and if not, should we fix it?

Hm I'm not quite sure how to fix it while keeping the declarative way of describing model associations within the model file, and it hasn't really been a problem in the past.

I suspect sequelize-typescript must be doing some sort of behind-the-scenes magic (perhaps by doing a lazy-load on associations specifically); their examples seem to recommend a dependency cycle too.

…cript

seaerchin

some comments regarding model definition/validators! also, we should file an issue and handle bootstrapping in a separate file, which will allow us to split up each individual loading function rather than lumping them together in server.js. wdyt about this?

bin/www

database/config.js

seaerchin · 2022-03-16T10:31:52Z

database/models/User.ts

+    unique: true,
+    type: DataType.TEXT,
+    validate: {
+      isEmail: true,


i think this is already validated in newroutes/users (here) so it might be helpful to just drop a comment here. (they're the exact same validation underneath as sequelize also uses validators.js, see here)

Will fix in a separate issue/PR.

seaerchin · 2022-03-16T10:36:57Z

database/models/User.ts

+  @Column({
+    allowNull: true,
+    type: DataType.STRING(255),
+  })


we should either 1. have the same validator as here or 2. remove validation entirely (we allow everything)

i think we should just use the validator as per the first point and actually stricten the check in here/the routes to sg only by specifying the options.

Will fix in a separate issue/PR.

seaerchin · 2022-03-16T10:39:08Z

database/models/User.ts

+
+  @Column({
+    type: DataType.DATE,
+    defaultValue: DataType.NOW,


i think this might be a little dangerous. we are explicitly setting the value when users login and if we allow creation of users by admins, might lead to an erroneous logged in time.

should we just remove the default value?

Let's follow up on this thread before returning to this discussion

Defer decision till later. File an issue

* refactor(server): separated out sequelize from server.js * chore(services/identity): update to use esm export chore(tokenstore): use esm export * refactor(userservice): convert to ts refactor(userservice): update to ts * refactor(sitesservice): use ts * refactor(identity): migrate index to ts * refactor(server): use new export from identity * chore(totpgenerator): relaxed typing * refactor(identity): add comments and rename function * fix(databse/index): add typecast * fix(database/config): updated exports * fix(totpgenerator): made expiry required * test(identity): add unit tests for user/sites service (#378) * refactor(server): separated out sequelize from server.js * build(package): removed sequelize cli * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * refactor(usersservice): refactor to take in instance of sequelize for ease of testing * refactor(database): defer sequelization initialization to caller this allows us to pass in base models at run time rather than statically deciding at compile time. this is useful when we want to test services that require a db but we wish to mock the db * test(usersservice): add test * refactor(sitesservice): change types; add non-null assertion (TO BE REMOVED LATER) * refactor(sitesservice): propagate null value upwards * fix(auth): set accessToken only if siteAccessToken exists * test(sitesservice): add tests * build(package): installed missing ts-node-dev dependency * chore(database): allow additional dbconfig for testing * chore(sequelizerc): removed extra .js * fix(usersservice): removed extra dot in default whitelist * feat(identity): add types to user router (#380) * build(types): installed express and jest typings * feat(types): add more types * refactor(req): changed setting userId on req to res.locals * build(package): installed type-fest * fix(usersservice): update typing for updateUserByGitHubId * refactor(users): shift to ts * refactor(users): fixed typing

…cript

@database

* build(src): add tsconfig and deletes jsconfig * build(package): install ts in dev deps * feat(eslint): adds config for ts * feat(eslint): adds config for ts * refactor(authservice): add types to auth service * refactor(mailclient): add types * chore(package-lock): remove trailing comma * build(package): installed ts import resolver * chore(eslint): update eslint to recognise imports properly * refactor(smsclient): add types for smsclient * refactor(tokenstore): add ts * refactor(totpgenerator): add ts * chore(eslint/tsconfig): update to recognise @database paths * chore(services/identity): changed to new imports for TS * build(tsconfig): changed module to cjs and changed to import statements * build(package): removed sequelize cli * feat(identity): unit tests for services (#369) * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * test(authservice): add spec * refactor(mailclient): changed mailclient so that initialization fails if api key is empty * test(mailclient): add tests and axios mock * refactor(constants): add constants for tests * refactor(smsclient): change api key retrieval to be done at constructor to allow for tesing * test(smsclient): add test * test(tokenstore): added tests * build(paths): add mocks to path * chore(tokenstore): update naming for clarity * chore(totpgenerator): changed access values * test(totpgenerator): add tests * refactor(mailclient): changed mailclient initialization for better readability * refactor: use sequelize-typescript (#374) * build(deps): add sequelize-typescript and required deps As described in https://www.npmjs.com/package/sequelize-typescript * chore(tsconfig): add required flags Also specified in sequelize-typescript docs * refactor: convert config file to ts * refactor: convert db model files into ts * refactor: use sequelize-ts in server.js * build(package): installed ts-node-dev * build(tsconfig): updates tsconfig with ts-node options we forc ets node to require tsconfig/paths to allow for path aliasing. this is because typescript requires that the runtime (node/ts-node) does the path aliasing, rather than ts. hence, requiring tsconfig/paths allows us to use path alias at runtime. additionally, we add a transpileOnly option. This prevents ts-node from typechecking and transpiles to js only, which increases its transpilation speed. another option, swc, was also added. this option uses the rust swc compiler for ts, which is significantly faster than the other compilers. this should speed up transpilation speed. * fix(server): fixed import of logger and sequelize sequelize (the class) is not a default export and has been changed accordingly. also, logger has been changed from morgan to winston (the one in our package) as morgan is less powerful than winston * chore(loaders/www): removed extra loader in www as we now perform it in server * chore(logger): fixed exports * fix(pcakage): update build scripts to use ts-node for start * build(package): install swc transpiler * revert(database/config): rollback to .js to avoid migration having issues * feat(identity): add types to last 2 services (#376) * refactor(server): separated out sequelize from server.js * chore(services/identity): update to use esm export chore(tokenstore): use esm export * refactor(userservice): convert to ts refactor(userservice): update to ts * refactor(sitesservice): use ts * refactor(identity): migrate index to ts * refactor(server): use new export from identity * chore(totpgenerator): relaxed typing * refactor(identity): add comments and rename function * fix(databse/index): add typecast * fix(database/config): updated exports * fix(totpgenerator): made expiry required * test(identity): add unit tests for user/sites service (#378) * refactor(server): separated out sequelize from server.js * build(package): removed sequelize cli * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * refactor(usersservice): refactor to take in instance of sequelize for ease of testing * refactor(database): defer sequelization initialization to caller this allows us to pass in base models at run time rather than statically deciding at compile time. this is useful when we want to test services that require a db but we wish to mock the db * test(usersservice): add test * refactor(sitesservice): change types; add non-null assertion (TO BE REMOVED LATER) * refactor(sitesservice): propagate null value upwards * fix(auth): set accessToken only if siteAccessToken exists * test(sitesservice): add tests * build(package): installed missing ts-node-dev dependency * chore(database): allow additional dbconfig for testing * chore(sequelizerc): removed extra .js * fix(usersservice): removed extra dot in default whitelist * feat(identity): add types to user router (#380) * build(types): installed express and jest typings * feat(types): add more types * refactor(req): changed setting userId on req to res.locals * build(package): installed type-fest * fix(usersservice): update typing for updateUserByGitHubId * refactor(users): shift to ts * refactor(users): fixed typing Co-authored-by: seaerchin <[email protected]> Co-authored-by: seaerchin <[email protected]> Co-authored-by: Preston Lim <[email protected]>

@database

* feat: add sequelize and pg libraries * feat: add sequelize migrations configuration * feat: add sequelize models for identity * feat: add database migrations to create tables * feat: init database connection before starting server * refactor: capitalize enum values and change to admin or user * fix: use correct key in db config for username * feat: add local dev postgres through docker compose * refactor: remove repo_name column * feat: add unique index on email * feat: add SiteService * feat: add AuthService * feat: add middleware to inject site access token * refactor: change variable casing of imported services * feat: add scaffold for AuthService otp methods * feat: add UserService * feat: extend auth middleware to check for isomer user id * feat: add routes for send and verify otp * feat: implement actual totp methods * feat: implement check for whether otp can be sent * feat: add validator for email format * feat: implement mail client using postman transactional api * feat: implement TokenStore * refactor: initialize services from module index * fix: use correct expiry value * feat: add github_id column to user model * feat: allow email to be null * feat: create user with github id * feat: apply verifyJwt for otp routes * feat: add methods for update and find by github id * feat: whitelist only by domain * feat: return email in whoami response * feat: add verifyOtp endpoint * refactor: rename to email otp methods * refactor: move otp routes to /user * refactor: move identity services to module * refactor: move email verification methods to user service * feat: add verification for contact number * feat: use bad request error * chore: upgrade sequelize * refactor: use router pattern for users routes * fix: undefined reference to usersService * chore: add sms cred name to example env file * feat: track last logged in date time * feat: update err message for non-govt emails * build(pcakage-lock): update build deps this commit is from running npm i - there should be no breakage and should be safe to rollback. * feat(identity): add types (#367) * build(src): add tsconfig and deletes jsconfig * build(package): install ts in dev deps * feat(eslint): adds config for ts * feat(eslint): adds config for ts * refactor(authservice): add types to auth service * refactor(mailclient): add types * chore(package-lock): remove trailing comma * build(package): installed ts import resolver * chore(eslint): update eslint to recognise imports properly * refactor(smsclient): add types for smsclient * refactor(tokenstore): add ts * refactor(totpgenerator): add ts * chore(eslint/tsconfig): update to recognise @database paths * chore(services/identity): changed to new imports for TS * build(tsconfig): changed module to cjs and changed to import statements * build(package): removed sequelize cli * feat(identity): unit tests for services (#369) * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * test(authservice): add spec * refactor(mailclient): changed mailclient so that initialization fails if api key is empty * test(mailclient): add tests and axios mock * refactor(constants): add constants for tests * refactor(smsclient): change api key retrieval to be done at constructor to allow for tesing * test(smsclient): add test * test(tokenstore): added tests * build(paths): add mocks to path * chore(tokenstore): update naming for clarity * chore(totpgenerator): changed access values * test(totpgenerator): add tests * refactor(mailclient): changed mailclient initialization for better readability * refactor: use sequelize-typescript (#374) * build(deps): add sequelize-typescript and required deps As described in https://www.npmjs.com/package/sequelize-typescript * chore(tsconfig): add required flags Also specified in sequelize-typescript docs * refactor: convert config file to ts * refactor: convert db model files into ts * refactor: use sequelize-ts in server.js * build(package): installed ts-node-dev * build(tsconfig): updates tsconfig with ts-node options we forc ets node to require tsconfig/paths to allow for path aliasing. this is because typescript requires that the runtime (node/ts-node) does the path aliasing, rather than ts. hence, requiring tsconfig/paths allows us to use path alias at runtime. additionally, we add a transpileOnly option. This prevents ts-node from typechecking and transpiles to js only, which increases its transpilation speed. another option, swc, was also added. this option uses the rust swc compiler for ts, which is significantly faster than the other compilers. this should speed up transpilation speed. * fix(server): fixed import of logger and sequelize sequelize (the class) is not a default export and has been changed accordingly. also, logger has been changed from morgan to winston (the one in our package) as morgan is less powerful than winston * chore(loaders/www): removed extra loader in www as we now perform it in server * chore(logger): fixed exports * fix(pcakage): update build scripts to use ts-node for start * build(package): install swc transpiler * revert(database/config): rollback to .js to avoid migration having issues * feat(identity): add types to last 2 services (#376) * refactor(server): separated out sequelize from server.js * chore(services/identity): update to use esm export chore(tokenstore): use esm export * refactor(userservice): convert to ts refactor(userservice): update to ts * refactor(sitesservice): use ts * refactor(identity): migrate index to ts * refactor(server): use new export from identity * chore(totpgenerator): relaxed typing * refactor(identity): add comments and rename function * fix(databse/index): add typecast * fix(database/config): updated exports * fix(totpgenerator): made expiry required * test(identity): add unit tests for user/sites service (#378) * refactor(server): separated out sequelize from server.js * build(package): removed sequelize cli * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * refactor(usersservice): refactor to take in instance of sequelize for ease of testing * refactor(database): defer sequelization initialization to caller this allows us to pass in base models at run time rather than statically deciding at compile time. this is useful when we want to test services that require a db but we wish to mock the db * test(usersservice): add test * refactor(sitesservice): change types; add non-null assertion (TO BE REMOVED LATER) * refactor(sitesservice): propagate null value upwards * fix(auth): set accessToken only if siteAccessToken exists * test(sitesservice): add tests * build(package): installed missing ts-node-dev dependency * chore(database): allow additional dbconfig for testing * chore(sequelizerc): removed extra .js * fix(usersservice): removed extra dot in default whitelist * feat(identity): add types to user router (#380) * build(types): installed express and jest typings * feat(types): add more types * refactor(req): changed setting userId on req to res.locals * build(package): installed type-fest * fix(usersservice): update typing for updateUserByGitHubId * refactor(users): shift to ts * refactor(users): fixed typing Co-authored-by: seaerchin <[email protected]> Co-authored-by: seaerchin <[email protected]> Co-authored-by: Preston Lim <[email protected]> * test(identity): integration tests for user routes (#382) * build(package): installed sqlite3 and types * chore(users): removed extra await * build(build deps): installed umzug (query interface for sequelize) * chore(package): update test command to use test env * feat(docker-compose): update to have another test db * test(jest.config.js): setup global setup/teardown files for integraiton * chore(config): add path aliases * build(tests): add setup/teardown scripts for integration * chore(tests): add logs for setup/teardown * test(users): add integration tests!!!!!! * test(usersservice): update test to fit new api * chore(.env.test): add test env file * fix(services/identity/index): fixed typing of mockMailer * chore(constants): shift test constants to tests folder * chore(constants): add new bearertokenheader * chore(testss): miinor test fixes * fix(users.spec): remove explicit mocking for otplib in integration testing * chore(users): remove extra console log * chore(users.spec): shift into integration * Refactor/move identity to v2 endpoints (#368) * Fix/retrieve last updated time (#354) * Fix: use pushed_at instead of updated_at * Fix: getLastUpdated endpoint * Fix: prepend unique prefix for release script (#370) * 0.1.0 * Refactor/auth (#328) * Feat: Add Auth service * Feat: add auth router * Feat: add auth endpoints * Feat: convert auth middleware to use dependency injection * feat: add tests for new auth router and service * Fix: add updated auth router endpoints to auth middleware * Refactor: verifyJwt into separate helper method * Nit: add exports for constants * Nit: add helper function for secure and comments for time * Style: early return * Style: test styling * Nit: adjust comments for importing error * Nit: adjust unknown route comment * Nit: return early * Nit: link issue * Refactor: move isSecure to utils * Style: early return * Nit: remove irrelevant comment * Refactor/separate auth initialisation (#377) * Chore: remove unused index router * Refactor: initialise middleware outside of server.js * Refactor: move unknown route check to server.js * Refactor: import auth middleware directly in routers * Chore: remove noverify method * Refactor/sites (#341) * Feat: add sitesService * Feat: add sites router * Feat: add site endpoints * Tests: add tests * Fix: use pushed_at instead of updated_at * Style: add spacing to tests * Style: object destructuring * Style: return type and map * Fix: return last updated time instead of string representing time and update tests * Feat: throw 404 error if no staging url found * Style: destructuring response * Feat: add access check in github service * Fix: tests * Fix: test comment * Rebase: use auth verify middleware in sites router * Fix: test for config priority * Fix: use originalUrl instead of url (#388) * fix: order of ping (#392) * 0.2.0 * chore(build): add mergify workflow (#385) * chore(add mergify): add mergify workflow * chore(mergify): remove extra build step * Fix: use existing githubservice instead of axiosinstance to check for user access * Refactor: move new logic from auth route into authService * refactor: shift auth middleware methods to authMiddlewareService * Fix: await method * Rebase: inject site token middleware into routers This commit also adds a path prefix for each router when using the middleware - I realised that we were previously hitting verifyJwt multiple times because a request would sequentially hit all routers until it reached the correct one - given that there was no check on the verifyJwt use in each router, it was called repeatedly * Fix: add auth middleware to users router * Fix: auth route and service tests * Feat: use dependency injection for middleware and identity services * Refactor: move v2 endpoints into subrouters * Refactor: use subrouters for v1 endpoints * Fix: tests * Fix: move identity constants to fixtures * Fix: test folder name * Nit: variable and function name * Nit: add comments * Chore: store repeated params in variable * Nit: test name * Nit: spacing * Nit: separate condition for readability * Chore: remove unnecessary error throwing * Chore: add comment on error transformation * Chore: add log statements for errors * Fix: set userId in res.locals instead of in req * Nit: add log statement for error * Nit: add proper error message for not logged in error * Fix: rebase errors * Chore: update comment * Chore: update error message * Fix: router path * Chore: update comment * Chore: update error messages * Fix: log error instead of providing information in error Co-authored-by: seaerchin <[email protected]> * Fix: merge errors * Fix: modify sequelizerc file to use config.js (#406) * feat(identity): infrastructure changes (#405) * chore(www.js): rename www -> www.js for ts to transpile it * build(package): changed start commands and added a build script the start command is retrieved from: https://stackoverflow.com/questions/61342753/paths-from-tsconfig-json-doesnt-work-after-tsc this is required because ts doesn't destructure the path aliases into the actual imports. * build(tsocnfig): updated tsconfig to emit this won't affect npm run dev as we are using ts-node and it will transpile on the fly * ci(ci): add step to build * chore(gitignore): ignore build drivre * fix(tsconfig): remove swc :( swc transpiles experimental flags (metadata/decorators) wrongly; this results in sequelize-typescript being transpiled wrongly * chore(authmiddlewareservice): remove logging of cookies * Fix: set reply_to param for OTP email (#408) * Fix: set reply_to param for OTP email * Chore: use support@isomer instead * Chore: update email to noreply * Fix: ping endpoint order (#407) Co-authored-by: Lam Kee Wei <[email protected]> Co-authored-by: Preston Lim <[email protected]> Co-authored-by: Alexander Lee <[email protected]>

@database

* feat: add sequelize and pg libraries * feat: add sequelize migrations configuration * feat: add sequelize models for identity * feat: add database migrations to create tables * feat: init database connection before starting server * refactor: capitalize enum values and change to admin or user * fix: use correct key in db config for username * feat: add local dev postgres through docker compose * refactor: remove repo_name column * feat: add unique index on email * feat: add SiteService * feat: add AuthService * feat: add middleware to inject site access token * refactor: change variable casing of imported services * feat: add scaffold for AuthService otp methods * feat: add UserService * feat: extend auth middleware to check for isomer user id * feat: add routes for send and verify otp * feat: implement actual totp methods * feat: implement check for whether otp can be sent * feat: add validator for email format * feat: implement mail client using postman transactional api * feat: implement TokenStore * refactor: initialize services from module index * fix: use correct expiry value * feat: add github_id column to user model * feat: allow email to be null * feat: create user with github id * feat: apply verifyJwt for otp routes * feat: add methods for update and find by github id * feat: whitelist only by domain * feat: return email in whoami response * feat: add verifyOtp endpoint * refactor: rename to email otp methods * refactor: move otp routes to /user * refactor: move identity services to module * refactor: move email verification methods to user service * feat: add verification for contact number * feat: use bad request error * chore: upgrade sequelize * refactor: use router pattern for users routes * fix: undefined reference to usersService * chore: add sms cred name to example env file * feat: track last logged in date time * feat: update err message for non-govt emails * build(pcakage-lock): update build deps this commit is from running npm i - there should be no breakage and should be safe to rollback. * feat(identity): add types (#367) * build(src): add tsconfig and deletes jsconfig * build(package): install ts in dev deps * feat(eslint): adds config for ts * feat(eslint): adds config for ts * refactor(authservice): add types to auth service * refactor(mailclient): add types * chore(package-lock): remove trailing comma * build(package): installed ts import resolver * chore(eslint): update eslint to recognise imports properly * refactor(smsclient): add types for smsclient * refactor(tokenstore): add ts * refactor(totpgenerator): add ts * chore(eslint/tsconfig): update to recognise @database paths * chore(services/identity): changed to new imports for TS * build(tsconfig): changed module to cjs and changed to import statements * build(package): removed sequelize cli * feat(identity): unit tests for services (#369) * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * test(authservice): add spec * refactor(mailclient): changed mailclient so that initialization fails if api key is empty * test(mailclient): add tests and axios mock * refactor(constants): add constants for tests * refactor(smsclient): change api key retrieval to be done at constructor to allow for tesing * test(smsclient): add test * test(tokenstore): added tests * build(paths): add mocks to path * chore(tokenstore): update naming for clarity * chore(totpgenerator): changed access values * test(totpgenerator): add tests * refactor(mailclient): changed mailclient initialization for better readability * refactor: use sequelize-typescript (#374) * build(deps): add sequelize-typescript and required deps As described in https://www.npmjs.com/package/sequelize-typescript * chore(tsconfig): add required flags Also specified in sequelize-typescript docs * refactor: convert config file to ts * refactor: convert db model files into ts * refactor: use sequelize-ts in server.js * build(package): installed ts-node-dev * build(tsconfig): updates tsconfig with ts-node options we forc ets node to require tsconfig/paths to allow for path aliasing. this is because typescript requires that the runtime (node/ts-node) does the path aliasing, rather than ts. hence, requiring tsconfig/paths allows us to use path alias at runtime. additionally, we add a transpileOnly option. This prevents ts-node from typechecking and transpiles to js only, which increases its transpilation speed. another option, swc, was also added. this option uses the rust swc compiler for ts, which is significantly faster than the other compilers. this should speed up transpilation speed. * fix(server): fixed import of logger and sequelize sequelize (the class) is not a default export and has been changed accordingly. also, logger has been changed from morgan to winston (the one in our package) as morgan is less powerful than winston * chore(loaders/www): removed extra loader in www as we now perform it in server * chore(logger): fixed exports * fix(pcakage): update build scripts to use ts-node for start * build(package): install swc transpiler * revert(database/config): rollback to .js to avoid migration having issues * feat(identity): add types to last 2 services (#376) * refactor(server): separated out sequelize from server.js * chore(services/identity): update to use esm export chore(tokenstore): use esm export * refactor(userservice): convert to ts refactor(userservice): update to ts * refactor(sitesservice): use ts * refactor(identity): migrate index to ts * refactor(server): use new export from identity * chore(totpgenerator): relaxed typing * refactor(identity): add comments and rename function * fix(databse/index): add typecast * fix(database/config): updated exports * fix(totpgenerator): made expiry required * test(identity): add unit tests for user/sites service (#378) * refactor(server): separated out sequelize from server.js * build(package): removed sequelize cli * build(package/): installed jest-mock-axios * build(package): installed ts-jest and removed jest config in package.json * test(jest.config,.js): added jest config this resolves ts files so that jest can handle them and ts-jest can provide proper typings. the module name mapping is relocated here * refactor(usersservice): refactor to take in instance of sequelize for ease of testing * refactor(database): defer sequelization initialization to caller this allows us to pass in base models at run time rather than statically deciding at compile time. this is useful when we want to test services that require a db but we wish to mock the db * test(usersservice): add test * refactor(sitesservice): change types; add non-null assertion (TO BE REMOVED LATER) * refactor(sitesservice): propagate null value upwards * fix(auth): set accessToken only if siteAccessToken exists * test(sitesservice): add tests * build(package): installed missing ts-node-dev dependency * chore(database): allow additional dbconfig for testing * chore(sequelizerc): removed extra .js * fix(usersservice): removed extra dot in default whitelist * feat(identity): add types to user router (#380) * build(types): installed express and jest typings * feat(types): add more types * refactor(req): changed setting userId on req to res.locals * build(package): installed type-fest * fix(usersservice): update typing for updateUserByGitHubId * refactor(users): shift to ts * refactor(users): fixed typing Co-authored-by: seaerchin <[email protected]> Co-authored-by: seaerchin <[email protected]> Co-authored-by: Preston Lim <[email protected]> * test(identity): integration tests for user routes (#382) * build(package): installed sqlite3 and types * chore(users): removed extra await * build(build deps): installed umzug (query interface for sequelize) * chore(package): update test command to use test env * feat(docker-compose): update to have another test db * test(jest.config.js): setup global setup/teardown files for integraiton * chore(config): add path aliases * build(tests): add setup/teardown scripts for integration * chore(tests): add logs for setup/teardown * test(users): add integration tests!!!!!! * test(usersservice): update test to fit new api * chore(.env.test): add test env file * fix(services/identity/index): fixed typing of mockMailer * chore(constants): shift test constants to tests folder * chore(constants): add new bearertokenheader * chore(testss): miinor test fixes * fix(users.spec): remove explicit mocking for otplib in integration testing * chore(users): remove extra console log * chore(users.spec): shift into integration * Refactor/move identity to v2 endpoints (#368) * Fix/retrieve last updated time (#354) * Fix: use pushed_at instead of updated_at * Fix: getLastUpdated endpoint * Fix: prepend unique prefix for release script (#370) * 0.1.0 * Refactor/auth (#328) * Feat: Add Auth service * Feat: add auth router * Feat: add auth endpoints * Feat: convert auth middleware to use dependency injection * feat: add tests for new auth router and service * Fix: add updated auth router endpoints to auth middleware * Refactor: verifyJwt into separate helper method * Nit: add exports for constants * Nit: add helper function for secure and comments for time * Style: early return * Style: test styling * Nit: adjust comments for importing error * Nit: adjust unknown route comment * Nit: return early * Nit: link issue * Refactor: move isSecure to utils * Style: early return * Nit: remove irrelevant comment * Refactor/separate auth initialisation (#377) * Chore: remove unused index router * Refactor: initialise middleware outside of server.js * Refactor: move unknown route check to server.js * Refactor: import auth middleware directly in routers * Chore: remove noverify method * Refactor/sites (#341) * Feat: add sitesService * Feat: add sites router * Feat: add site endpoints * Tests: add tests * Fix: use pushed_at instead of updated_at * Style: add spacing to tests * Style: object destructuring * Style: return type and map * Fix: return last updated time instead of string representing time and update tests * Feat: throw 404 error if no staging url found * Style: destructuring response * Feat: add access check in github service * Fix: tests * Fix: test comment * Rebase: use auth verify middleware in sites router * Fix: test for config priority * Fix: use originalUrl instead of url (#388) * fix: order of ping (#392) * 0.2.0 * chore(build): add mergify workflow (#385) * chore(add mergify): add mergify workflow * chore(mergify): remove extra build step * Fix: use existing githubservice instead of axiosinstance to check for user access * Refactor: move new logic from auth route into authService * refactor: shift auth middleware methods to authMiddlewareService * Fix: await method * Rebase: inject site token middleware into routers This commit also adds a path prefix for each router when using the middleware - I realised that we were previously hitting verifyJwt multiple times because a request would sequentially hit all routers until it reached the correct one - given that there was no check on the verifyJwt use in each router, it was called repeatedly * Fix: add auth middleware to users router * Fix: auth route and service tests * Feat: use dependency injection for middleware and identity services * Refactor: move v2 endpoints into subrouters * Refactor: use subrouters for v1 endpoints * Fix: tests * Fix: move identity constants to fixtures * Fix: test folder name * Nit: variable and function name * Nit: add comments * Chore: store repeated params in variable * Nit: test name * Nit: spacing * Nit: separate condition for readability * Chore: remove unnecessary error throwing * Chore: add comment on error transformation * Chore: add log statements for errors * Fix: set userId in res.locals instead of in req * Nit: add log statement for error * Nit: add proper error message for not logged in error * Fix: rebase errors * Chore: update comment * Chore: update error message * Fix: router path * Chore: update comment * Chore: update error messages * Fix: log error instead of providing information in error Co-authored-by: seaerchin <[email protected]> * Fix: merge errors * Fix: modify sequelizerc file to use config.js (#406) * feat(identity): infrastructure changes (#405) * chore(www.js): rename www -> www.js for ts to transpile it * build(package): changed start commands and added a build script the start command is retrieved from: https://stackoverflow.com/questions/61342753/paths-from-tsconfig-json-doesnt-work-after-tsc this is required because ts doesn't destructure the path aliases into the actual imports. * build(tsocnfig): updated tsconfig to emit this won't affect npm run dev as we are using ts-node and it will transpile on the fly * ci(ci): add step to build * chore(gitignore): ignore build drivre * fix(tsconfig): remove swc :( swc transpiles experimental flags (metadata/decorators) wrongly; this results in sequelize-typescript being transpiled wrongly * chore(authmiddlewareservice): remove logging of cookies * Fix: set reply_to param for OTP email (#408) * Fix: set reply_to param for OTP email * Chore: use support@isomer instead * Chore: update email to noreply * Fix: ping endpoint order (#407) Co-authored-by: Lam Kee Wei <[email protected]> Co-authored-by: Preston Lim <[email protected]> Co-authored-by: Alexander Lee <[email protected]>

prestonlimlianjie added 5 commits March 3, 2022 15:54

build(deps): add sequelize-typescript and required deps

12093ea

As described in https://www.npmjs.com/package/sequelize-typescript

chore(tsconfig): add required flags

df77e1d

Also specified in sequelize-typescript docs

refactor: convert config file to ts

f2958c7

refactor: convert db model files into ts

c904d01

refactor: use sequelize-ts in server.js

2aec291

prestonlimlianjie changed the base branch from develop to feat/identity/types March 3, 2022 11:57

seaerchin added 5 commits March 8, 2022 12:56

build(package): installed ts-node-dev

0b14c4e

fix(server): fixed import of logger and sequelize

8bc46d6

sequelize (the class) is not a default export and has been changed accordingly. also, logger has been changed from morgan to winston (the one in our package) as morgan is less powerful than winston

chore(loaders/www): removed extra loader in www as we now perform it …

867a716

…in server

chore(logger): fixed exports

f120eaa

seaerchin added 2 commits March 10, 2022 18:02

fix(pcakage): update build scripts to use ts-node for start

091ab2a

build(package): install swc transpiler

67a1f35

revert(database/config): rollback to .js to avoid migration having is…

bfbc967

…sues

prestonlimlianjie force-pushed the feat/identity/sequelize-typescript branch from 3684bc2 to bfbc967 Compare March 16, 2022 07:51

Merge branch 'feat/identity/types' into feat/identity/sequelize-types…

0f548be

…cript

prestonlimlianjie changed the title ~~[WIP] refactor: use sequelize-typescript~~ refactor: use sequelize-typescript Mar 16, 2022

prestonlimlianjie marked this pull request as ready for review March 16, 2022 08:16

prestonlimlianjie requested review from seaerchin, alexanderleegs and rc-davis and removed request for seaerchin March 16, 2022 08:16

seaerchin reviewed Mar 16, 2022

View reviewed changes

seaerchin and others added 2 commits March 17, 2022 17:16

Merge branch 'feat/identity/types' into feat/identity/sequelize-types…

ce6b4a6

…cript

seaerchin merged commit 1744668 into feat/identity/types Mar 17, 2022

seaerchin deleted the feat/identity/sequelize-typescript branch March 17, 2022 09:21

prestonlimlianjie mentioned this pull request Mar 17, 2022

Validate email field only once #396

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor: use sequelize-typescript #374

refactor: use sequelize-typescript #374

prestonlimlianjie commented Mar 3, 2022 •

edited

Loading

seaerchin commented Mar 10, 2022

seaerchin commented Mar 11, 2022

prestonlimlianjie commented Mar 14, 2022 •

edited

Loading

prestonlimlianjie commented Mar 14, 2022

seaerchin left a comment

seaerchin Mar 16, 2022

prestonlimlianjie Mar 17, 2022 •

edited

Loading

seaerchin Mar 16, 2022

prestonlimlianjie Mar 17, 2022

seaerchin Mar 16, 2022

prestonlimlianjie Mar 17, 2022

prestonlimlianjie Mar 17, 2022

refactor: use sequelize-typescript #374

refactor: use sequelize-typescript #374

Conversation

prestonlimlianjie commented Mar 3, 2022 • edited Loading

Dependency changes

Dev-dependency changes

seaerchin commented Mar 10, 2022

seaerchin commented Mar 11, 2022

prestonlimlianjie commented Mar 14, 2022 • edited Loading

prestonlimlianjie commented Mar 14, 2022

seaerchin left a comment

Choose a reason for hiding this comment

seaerchin Mar 16, 2022

Choose a reason for hiding this comment

prestonlimlianjie Mar 17, 2022 • edited Loading

Choose a reason for hiding this comment

seaerchin Mar 16, 2022

Choose a reason for hiding this comment

prestonlimlianjie Mar 17, 2022

Choose a reason for hiding this comment

seaerchin Mar 16, 2022

Choose a reason for hiding this comment

prestonlimlianjie Mar 17, 2022

Choose a reason for hiding this comment

prestonlimlianjie Mar 17, 2022

Choose a reason for hiding this comment

prestonlimlianjie commented Mar 3, 2022 •

edited

Loading

prestonlimlianjie commented Mar 14, 2022 •

edited

Loading

prestonlimlianjie Mar 17, 2022 •

edited

Loading