tests are silently failing in CI

[alexwlchan]

Quoting Paul in Slack:

The test stages in Experience fail silently on buildkite. e.g. see the test content stage here https://buildkite.com/wellcomecollection/experience/builds/5721#fd698ae6-77eb-42e6-bb31-e46799205505

[paul-butcher]

My first suspicion is that it is to do with this (and a similar thing in catalogue):
process.exit called with "0" when a server is closed during the last test.

So the tests could all fail, but it will still return a zero exit code as long as that test runs. It may also mean that some tests are not being run (if process.exit(0) happens when there are still outstanding tests)

[paul-butcher]

Yes. I have commented out most of that file in catalogue and forced a failure.

[paul-butcher]

It would appear that the mocking of process.exit is not being applied in the expected scope.

If I call process.exit in the test file, it uses the mock, but when it calls server.close, which calls process.exit it calls the real process.exit.

[paul-butcher]

It has been failing and erroneously passing this way since November, build 4704 - https://buildkite.com/wellcomecollection/experience/builds/4704#831bee00-4923-47f0-885f-890239bd8bce

with this change.

3c6d5b7

[paul-butcher]

That turns out to be when this mocking was introduced.

[paul-butcher]

It also appears that catalogue (at least) is missing some libraries required for testing.

This is probably missed by developers working in their own environments because they are sharing an environment between all the workspaces, but when the Docker images are all built, they are built from the individual package.json files.