use vitest for integration tests #1444

Florent-Bouisset · 2024-05-31T13:22:38Z

We currently use karma, mocha and chai for integration test, however this need to compile our code with "karma-webpack" which is a plugin that is not very much maintain and that simply does not work on MacOS devices.
So it's currently not possible to start integration tests on that device.

The goal of this PR is to change tooling to vitest so the tests can be run on all devices

peaBerberian · 2024-06-03T17:02:19Z

Nice.

The main issue is that vitest seems to have to whole planet as a dependency, symbolically growing our package-lock.json from an already-consequent 20k lines file to a 32k one (and I'm sure the more significant node_modules directory took a nice bump in size also), though this is secondary (principal issues this could be linked to would be security and maintainance-related but it's only for tests anyway) and gains are very nice.

With #1435 and this, we could also just remove dependencies linked to webpack anyway.

package.json

.github/workflows/checks.yml

package.json

src/parsers/texttracks/webvtt/html/__tests__/parse_style_block.test.ts

peaBerberian · 2024-06-03T17:14:39Z

tests/integration/globalSetup.js

+  // Start the server content prior to unit tests
+  const CONTENT_SERVER_PORT = 3000;
+  const testContentServer = TestContentServer(CONTENT_SERVER_PORT);
+};


I don't get the point of that file compared to the ../contents/server.mjs.

If the issue is that we need an exported function without parameters, maybe we should just rely on globals :/

Yes the issue is that Vitest execute the default export function of the file specified in the vitest config : globalSetup: "tests/integration/globalSetup.js"

We cannot pass argument to that function.
I can change contents/server.mjs so that it uses globals for the port number, but I don't understand why it would be better ?

Just having one supplementary file, just to set a default argument and call it is seems too hacky

Maybe TestContentServer could just rely on a default port?

Also at worst it seems a provide API (https://vitest.dev/config/#globalsetup) is available for the server to transmit data (like the port) to tests. Though in reality, I would prefer the server be separated from the testing framework and that it was the caller that provided the setup :/

vitest.config.mjs

To extrapolate on #1444, this PR shows how it would look like to switch from relying on Jest to rely on vitest for our unit tests. The end goal being to simplify our codebase by relying on a single testing framework. --- I only done so for testing files in the `src/compat` directory in this demo. It can be tested right now by calling `npm run test:unit:vitest`. --- Sadly for now, we are still forced to rely on a JSDom-ed Node.js environment for unit tests and a browser environment for integration tests - meaning we have very different configs for both. This is because we want to mock imported files in unit tests - something that is not possible for now in browser environment through vitest (though vitest-dev/vitest#5765 seems to have been merged very recently so maybe we could rely on the browser for both soon), yet we want to replicate as much as a real browser as possible in our integration tests (because we're also testing that media playback on the tested browsers goes as expected).

To extrapolate on #1444, this PR shows how it would look like to switch from relying on Jest to rely on vitest for our unit tests. The end goal being to simplify our codebase by relying on a single testing framework. --- I only done so for testing files in the `src/compat` directory in this demo. It can be tested right now by calling `npm run test:unit:vitest`. --- Sadly for now, we are still forced to rely on a JSDom-ed Node.js environment for unit tests and a browser environment for integration tests - meaning we have very different configs for both. This is because we want to mock imported files in unit tests - something that is not possible for now in browser environment through vitest (though vitest/5765 seems to have been merged very recently so maybe we could rely on the browser for both soon), yet we want to replicate as much as a real browser as possible in our integration tests (because we're also testing that media playback on the tested browsers goes as expected).

… it by vitest

To extrapolate on #1444, this PR shows how it would look like to switch from relying on Jest to rely on vitest for our unit tests. The end goal being to simplify our codebase by relying on a single testing framework. --- I only done so for testing files in the `src/compat` directory in this demo. It can be tested right now by calling `npm run test:unit:vitest`. --- Sadly for now, we are still forced to rely on a JSDom-ed Node.js environment for unit tests and a browser environment for integration tests - meaning we have very different configs for both. This is because we want to mock imported files in unit tests - something that is not possible for now in browser environment through vitest (though vitest/5765 seems to have been merged very recently so maybe we could rely on the browser for both soon), yet we want to replicate as much as a real browser as possible in our integration tests (because we're also testing that media playback on the tested browsers goes as expected).

* create a globalSetup for vitest * rename tests files to .test.js * delete useless files and lint * skip problematic test on firefox * fix * skip test * [WIP] Switch unit tests to use vitest To extrapolate on #1444, this PR shows how it would look like to switch from relying on Jest to rely on vitest for our unit tests. The end goal being to simplify our codebase by relying on a single testing framework. --- I only done so for testing files in the `src/compat` directory in this demo. It can be tested right now by calling `npm run test:unit:vitest`. --- Sadly for now, we are still forced to rely on a JSDom-ed Node.js environment for unit tests and a browser environment for integration tests - meaning we have very different configs for both. This is because we want to mock imported files in unit tests - something that is not possible for now in browser environment through vitest (though vitest/5765 seems to have been merged very recently so maybe we could rely on the browser for both soon), yet we want to replicate as much as a real browser as possible in our integration tests (because we're also testing that media playback on the tested browsers goes as expected). * Add test:unit:watch * tests: Fix and work around remaining unit test issues * wip * Reorder checks in often-failing integration test * Try to '''fix''' memory tests --------- Co-authored-by: Florent <[email protected]>