Integration test suite

[jackgerrits]

We should keep unit tests lean, using mocking etc for things like LLMs we need to ensure there is a larger integration test suite that can be run either as a cron, or triggered based on less frequent events. We can explore using benchmarks as the means for this kind of larger regression testing.

Additionally, this testing should always be run prior to release.