Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests #17

Closed
Blond11516 opened this issue May 19, 2023 · 4 comments
Closed

Add tests #17

Blond11516 opened this issue May 19, 2023 · 4 comments

Comments

@Blond11516
Copy link
Collaborator

Right now we only have the one E2E test using @vscode/test-electron, which is better than nothing but clearly not enough. We should add tests to have better coverage of the auto-installation process at the very least.

The default vscode extension template uses mocha for tests, which I would like to move away from because:

  1. I have zero experience with it, nor with the mocking and assertion libraries we might eventually need since mocha doesn't come with those functionalities. I don't usually shy away from using new technologies but...
  2. ... the documentation sucks. It's just a single static page with poor discoverability and navigation.

While we could absolutely learn and use mocha, I'd like the tools we depend on to be as easy to setup, learn and use as possible since the vscode extension likely won't have a lot of time dedicated to it any time soon.

However, using another library isn't so easy. We could keep mocha for E2E tests and another library for unit tests, but Mocha sadly defines global functions to define suites and tests, with no way to opt out. This means that when trying to use another one we cannot use globals and editors always detect mocha globals by default, meaning they won't suggest importing functions from other libraries. This makes the developer experience pretty poor unless we use some workaround like another type definition package that removes the globals or something.

From there, setting up vscode electron tests with another library is also an ordeal.

It appears to be possible with Jest (see microsoft/vscode-test#37 (comment) and https://github.com/daddykotex/jest-tests) but it doesn't work with Jest 28 and upwards (latest version as of writing is 29.5.0, 27 having been initially release 2 years ago).

I've also tried to do it with vitest, but custom vitest environments currently require publishing a package with a specific name prefix, which feels both clunky and overkill. There is an open issue to support custom environments from local files, but I've no idea if this feature will be prioritized by the vitest team any time soon. I might try to do a package or contribute support for local custom environments if that's what it takes to get this working.

There are also other testing libraries we could try like Ava or Tape. However, them being less used than even Vitest means there is likely less support for the weird kind of edge cases that vscode tests are.

I'll keep trying things out in the the following days because I'd really like to get some decent tests in before doing too much more work on the extension.

@Blond11516
Copy link
Collaborator Author

I managed to adapt the example I found with Jest to support the latest version. Setup is in #18.

@jsphstls
Copy link

You can now extend your Vitest environment by specifying a path to a valid JS file (supported since 0.34.0).

https://vitest.dev/guide/environment.html#custom-environment

@Blond11516
Copy link
Collaborator Author

@jsphstls Thanks for the heads up! It does look like we should be able to run E2E test with vitest now, though after trying to get it to work for ~2 hours I don't think it's worth it for now 😅 Maybe I'll revisit this when we have more tests and vitest's speed makes a bigger difference

@Blond11516
Copy link
Collaborator Author

I've added unit tests with almost complete coverage in #45. The only thing not covered is the extension entrypoint itself, but the happy path at least is covered by the E2E test, so I'll consider this issue closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants