Functional tests

[rockerbacon]

The unit tests that we currently have provide a decent level of quality assurance but it is far from ideal. As made evident by #20, without some amount of functional or integrated testing we will always be subject to catastrophic application failures caused by coding errors.

While anticipating some work for #17, it was also made apparent to me that the kind of application we have can be greatly simplified and made more stable if we reduce the threshold for unit test coverage and supplement the "gaps" with good quality functional tests. The solution initially proposed in that discussion requires a lot less code when dependency inversion is ignored and there's much more value in testing it against more "real world" scenarios.

[nlaranjeira01]

I agree that having functional testing is going to be beneficial to the project. About the unit test coverage threshold, we can decrease it as needed, as long as we keep adding valuable functional testing cases that cover those gaps, as you said.

👍

[Jason1roy]

I completely agree with your point about the importance of functional testing in our project. While having a solid unit test coverage threshold is crucial, it's also essential to recognize that it's not the only aspect of testing that matters. By prioritizing valuable functional testing cases and continuously adding them to cover any gaps, we can ensure comprehensive test coverage while maintaining flexibility in adjusting the unit test coverage threshold as needed. This approach allows to strike the right balance between thoroughness and adaptability in our testing strategy.

[rockerbacon]

Since there's consensus that we should have function tests, it's just a matter of deciding on how to do them. The questions I have are:

1. Are we going to write and run them separate from the unit tests?
   • My suggestion: yes, we have more control if things are kept separate. It may be slightly more costly to maintain the pipeline automations but I believe the benefits far outweigh this additional cost.
2. How are we going to enforce them?
   • My suggestion: The easiest way would be to try and do the functional tests with the standard Go tools we're already using. From there we can calculate the aggregate coverage of both test levels and just try to keep the coverage very high. I don't know of any other way to properly enforce tests other than coverage, any humanized approach would be very frail (IMO).