Although you find no hardcoded secrets anywhere in the project,
there is more that can be done to improve security and secret storage.
At first I set up a secretsmanager in ../infra/datastore/secrets.tf
,
however I found the overhead in my lambda wrapper and cost was not justified at this point.
It did work, but overall unnecessary right now.
(Really I was just curious how it would feel to use it, so I did for a bit and left the store in for later use)
Currently the api supports creating a unit, creating a group with associated rules, and searching the system for units/groups/rules. There where more methods that I looked at providing such as full CRUD, user login, getting user preferences, and creating custom validation rules. In the end I decided to keep this short (yet extendable).
Just an apikey is supported currently. The auth layer could stand to be abstracted further by something like kong or cognito. It is currently a lightweight lambda authorizor (custom) that validates the apikey. Also should be noted that the key must be associated with an apigateway usage plan as well (so that each key is throttled).
As of now, I just need to know:
- is the key valid?
- who is using the system?
- what org do they belong to?
No location data. These units represent physical objects, it would be nice to know where they are (store number/aisle etc). Right now I have no such information in the system. Only the basic properties and whatever extended properties that users declare.
There is a whole world of database management techniques. I pretty much avoided most of them for this first pass. The decision was to have simple scripts that are manually run at to get a schema going instead of an ORM or something like knex.
While I was inventing a dynamic rule system, I kept the database design pretty flat around rules.
A single table rule_map
is used to contain the one-to-many relationship (group-rules).
While this isn't the worst thing ever, it could be improved too by creating a true rule concept at the database level.
alternative: unit-ext could have been a mapping table where each key/value is a row instead of a json blob for all the props.
Totally skipped that for this demo. The overhead for automating terraform changes is particularly complex.
I've done it before and makes a great next step. Deployments are manual via make deploy TARGET=myModule ENV=dev
.
Since there isn't a ton of data, this didn't even cross my mind until I was adding new units in and seeing all the results come back. Something like this eventually with the N result responses.
count: int
per_page: int
page: int
pages: int
has_prev: bool
has_next: bool
This is fine for now, but needs to change pretty quick.
There are some edge cases I know about that I'd like to be covered too. For now they are skipped over since they are very very unlikely to be hit. I decided to spend time on other aspects and just add this to tech debt.
I would prefer that the actual sql be abstracted from the methods.
Somthing like import queries from my-db-abstraction
then mysql.query(queries.getUnit, params)
.
in no particular order
Establish github flow where tagging a release deploys to prod and push to master deploys to nonprod.
Create a build box ubuntu image to run tests and deploy OR docker container that does the same.
Link up a circle file to the make
commands for automatic testing on commit and good build logs.
I want to test even more with a better db mock like sinon
.
I also want to create a local docker database that can be used for quick integration tests before the engineer deploys changes.
This is particularly more important on teams, because people can block eachother too easily if everyone has to deploy to test the actual sql they wrote.
The docker database can additionally be utilized for integration tests with seeded data.
Oh yeah and a test runner, those make coding feel more gamified.
Use something like bunyan
, to be able to set a log level and switch em on or off via configuration.
Just have to close up the remaining edge cases once the improved tests are written. This can be done at the same time.
Maybe each unit can be a part of multiple groups? Can refactor database to use a mapping table instead of just a 1-1 unit-group_id relationship.
Template the oas spec (so that the aws account number / region isn't hardcoded). Find a way for customer facing api spec docs to be generated and not manually updated
(minimal roles, tighter access)
Shrink deployment size by removing test libs on lambda deploys.
Only recopy node_modules
in zip when it has changed.
because hflshdfbla.aws.com is not fun to type, read, or remember.
Save costs and boost performance by 30ms
Try out cognito for abstracted auth. Haven't used it before, but want to give it a shot.
Ideally this system's only bottleneck would be database. It was designed to be totally serverless and completely parallelizable. I'd want to put this to the test.
Can only be done after some kind of SLA or usage pattern is known. After some benchmarks, can establish some autoscaling rules.
Swagger-ui is nice, but an experience that captures a persona is great for selling.
Support for weight approximate match. For example, you must search for identical mass/weight. You can't use pounds or kilograms. It must be exact grams. So I'd want this to be way more friendly before filter by weight/mass is user worthy I think.
Right now all filtering is done as %value%
, maybe you want to do a greater than, less than, strict equal, etc.
It is possible for a method to fail halfway leaving an undesireable half-state. This can be resolved with transactions.
We already have custom validators, extending the engine a bit to support async is possible and would be useful to validate if a rule exists dynamically for example.
This would also help address a minor security breach where creation of new units cannot easily validate if the group is valid for that unit, only if the id is a valid int.
The parameterizedValidators
could use a pretty error name instead of <Function>
Change column names that are reserved keywords -_- (key, desc, group).
Create a terraform module for common api method pattern.