Add automated testing

[carterm]

intercept fetches and replace

Inputs can be static json
outputs with json schema validation

[chachasikes]

@carterm

What are your current thoughts on automated testing?

1. Do you have a list of the main business logic of your service that need tests?
2. What kinds of tests do you think this script needs?
3. Can you make an outline of the kinds of tests that make sense for your service?
4. What are some of the risks of adding tests to this system?

[carterm]

Some quick wins

• JSON Schema validation.

  • Build/Deploy time - Validate all saved project JSON files during build to make sure they match their defined "$schema". Could be done with lint.
  • Runtime - Validate input schema from external sources not covered by project schema.
  • Runtime - Validate output schema

• Dummy (better word needed) environment, complete separate end-to-end test

  • Need Azure instance to run a "automated test" branch that will have a custom endpoints.json. And logic to report a test fail.
  • Need WordPress instance where nothing changes
  • Need target branch where nothing changes
  • Test fails if service makes an actual change commit.

[carterm]

Javascript schema validation examples can be found in this project, although using the $schema ref directly would be better than relying on paths.
https://github.com/cagov/Cron/tree/master/common/SQL/CDT_COVID/Daily-stats-v2/schema