[ViteJS] Empty coverage for all files not covered by tests

[Sytten]

Versions

• What is this plugin's version? 3.9.12
• What is Cypress version? 9.1.0
• What is your operating system? MacOS
• How do you instrument your application? Instanbul
• Is there .nyc_output folder? Yes, not empty
• Do you have any custom NYC settings? Yes, all
• Do you run Cypress tests in a Docker container? No

Describe the bug
Despite using the all setting of NYC, it seems like only files touched by tests have their line numbers included in the report.
The other files are present but show 0 lines thus are ignored in the final computation of the code coverage.
Using the debug flag, I see them added:

  code-coverage adding empty coverage for file /Users/Sytten/Projects/*****/src/main.ts +0ms
  code-coverage adding empty coverage for file /Users/Sytten/Projects/*****/src/composables/index.ts +0ms
  code-coverage adding empty coverage for file /Users/Sytten/Projects/*****/src/composables/useId.ts +0ms
  code-coverage adding empty coverage for file /Users/Sytten/Projects/*****src/composables/useName.ts +0ms

But I would expect them to get line numbers at some point. But they only get:

TN:
SF:src/main.ts
FNF:0
FNH:0
LF:0
LH:0
BRF:0
BRH:0
end_of_record

I use ViteJS and my plugins looks like:

const path = require("path");
const { startDevServer } = require("@cypress/vite-dev-server");
const codeCoverageTask = require("@cypress/code-coverage/task");
const istanbul = require("vite-plugin-istanbul");

module.exports = (on, config) => {
  on("dev-server:start", (options) =>
    startDevServer({
      options,
      viteConfig: {
        logLevel: "warn",
        configFile: path.resolve(__dirname, "..", "..", "vite.config.js"),
        plugins: [istanbul({})],
      },
    })
  );

  codeCoverageTask(on, config);
  return config;
};

Support contains:

// Import commands.js using ES2015 syntax:
import "./commands";

// Alternatively you can use CommonJS syntax:
// require('./commands')

import "@cypress/code-coverage/support";

The coverage for files under test works fine.

Link to the repo
This is not an open-source project, but happy to provide a reproduction if that can help debug it.

[lmiller1990]

Thanks for the bug report - this should be enough to reproduce and work on this issue.

[noor-tg]

@Sytten did it work ? or the bug is fixed ?
do you use vite dev server for e2e tests or unit tests ?

[Sytten]

It doesnt work but we worked around it by forcing it to load/import all files, I don't exactly remember how.
We do component tests.

[wanbinkimoon]

any update on this?
@Sytten do you mind share your solution would be very helpful here :)

[Sytten]

Basically create a dummy test for your main entrypoint:

import "./main";

describe("Main", () => {
  it("Main loads", () => {
    expect(true).equal(true);
  });
});

That will load all the files of your app (unless you do some dynamic importing stuff) and it will correct your coverage.

[lucaspeterson22]

@lmiller1990 Is there any update on this issue? Thanks for any info you can provide 👍🏻

[lmiller1990]

Hello, we are working with the Vite team to make it easier for people to build on top of Vite (including us) which should make things like code coverage plugins much easier to author.

I don't have an update right now, but I'd expect to have something in the next month or so. @JessicaSachs is working on this primarily, maybe she can chime in if her work will help with code coverage?

I didn't get to try fixing it with your current example, I figured if Vite is making big internal changes, it's better to wait.

[JessicaSachs]

There was an open issue in the Istanbul plugin that was recently fixed for this... can you give this a try @lucaspeterson22 iFaxity/vite-plugin-istanbul#13?

[yann-combarnous]

There was an open issue in the Istanbul plugin that was recently fixed for this... can you give this a try @lucaspeterson22 iFaxity/vite-plugin-istanbul#13?

Unfortunately, ´all: true’ option for nyc - as described in your code coverage doc - is not working with this vite plugin, causing issues for merging parallel tests code coverage results. Any progress with Vite team, by any chance?

[yann-combarnous]

My workaround is to disable chunk-splitting in Vite using:

{
  rollupOptions: {
    output: {
      manualChunks: () => 'bundle',
    }
}

Otherwise with code splitting, Vite only parses the code files loaded by the tests.

[elamurugan-r]

Basically create a dummy test for your main entrypoint:

import "./main";

describe("Main", () => {
  it("Main loads", () => {
    expect(true).equal(true);
  });
});

That will load all the files of your app (unless you do some dynamic importing stuff) and it will correct your coverage.

I tried this in my project. But still i can see 0 lines for the files to which no test cases are available. I can see some coverage lines in all the files only if i mount all the components with dummy configuration. Is there any other work around?

[lmiller1990]

I'm sorry this issue flew under my radar. I have not had a chance to look into Vite code coverage, but this is something I'd really like to do soon. We are using Vite for the front-end at Cypress, so it would be hugely beneficial for us, too.

I'm not sure what the general situation is for Vite code coverage (even outside of Cypress). If anyone has more info, please share it here, so when we do work on this, we will have lots of useful information to go on.

[illegalnumbers]

Any update on this? The workarounds presented (chunking, dummy entry point file) didn't work for me to get non-visited vue files to show up in the code coverage report.

[rbong]

I believe this is the same issue as #552

I have a rough fix working: using Istanbul and Babel to read initial coverage data

[belwalshubham]

after integrate cypress code coverage in nuxt , Though I'm getting options.ssr value as both true & false for different files, I got this error on my console ,
[Vue warn]: Hydration node mismatch:
Client vnode:..
can anyone help me on this ?

[lmiller1990]

^ That happens when the SSR'd DOM doesn't match the front-end.

You'll need to share a minimal reproduction, I can't do much more than guess without one. Thanks!

[belwalshubham]

@lmiller1990 Thanks for the response. Here are my steps.
Nuxt3 Vite support for Cypress coverage instrumentation

I was trying to instrument the project code using the vite-plugin-istanbul npm package.

Inside the package.json, here are my package dependencies.

"@cypress/code-coverage": "^3.10.0",
"vite-plugin-istanbul": "^3.0.4",

Here's my nuxt.config.ts after installing the vite-plugin-Istanbul package:

vite: {
  vue: {
    template: {
      transformAssetUrls: true
    }
  },
  plugins: [
    istanbul({
      exclude: ['node_modules', 'test/', 'coverage/'],
      extension: [ '.js', '.ts', '.vue' ],
      cypress: true
    }),
  ]
},

Here is my cypress/support/e2e.ts file.

import '@cypress/code-coverage/support'

Here is my cypress.config.ts file.

import { defineConfig } from "cypress"
export default defineConfig({
  projectId: "x83arq", // TODO: hide this
  e2e: {
    baseUrl: "http://localhost:3000",
    setupNodeEvents(on, config) {
      require("@cypress/code-coverage/task")(on, config)
      return config
    }
  }
})

After running npm run dev, my UI interface crashed and was not getting complied properly. and an error message appears on the console, as shown in the screenshot below.

However, when I run npx Cypress open, it displays my Cypress code coverage report after the test is completed, but the main issue is that my UI is not functioning and is not being compiled properly.

[lmiller1990]

Can you please upload your entire GH repo, so I can clone and reproduce? It looks like Nuxt has a bunch of different config options when you make a new project. Thanks!

[belwalshubham]

hey @lmiller1990 this above code is part of a private repo, but I have another public repo, here same code was implemented for code coverage in nuxt3 and same error occurs in console, please have a look https://github.com/belwalshubham/twitter-clone-cypress-code-coverage

[belwalshubham]

hello @lmiller1990 Please let me know if you find anything related to this issue on the above repo.

[lmiller1990]

I am looking into code coverage for Vite.

[JessicaSachs]

I have an example working on my work codebase. Do you need a hand? Poke me on Discord.

…

On Tue, Dec 27, 2022 at 8:03 PM Lachlan Miller ***@***.***> wrote: I am looking into code coverage for Vite. — Reply to this email directly, view it on GitHub <#539 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAVL4BHQY474EHKSKY6AGEDWPOGVHANCNFSM5JPRVOSA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[belwalshubham]

@lmiller1990 thanks

[belwalshubham]

hello @lmiller1990 thank you, the problem has been resolved

[lmiller1990]

So it looks like you can get coverage just fine with vite-plugin-istanbul. Look in that repo for more.

In the meantime, I think we need 1) better docs 2) example repo 3) ideally ship this out of the box in some fashion.

For now, I'd recommend we focus on 1) and 2) so people can get started, and look at a more "batteries included" solution down the track.

[ThijSlim]

The same issue is still persistent in vite-plugin-istanbul: iFaxity/vite-plugin-istanbul#28

[ThijSlim]

Basically create a dummy test for your main entrypoint:

import "./main";

describe("Main", () => {
  it("Main loads", () => {
    expect(true).equal(true);
  });
});

That will load all the files of your app (unless you do some dynamic importing stuff) and it will correct your coverage.

I tried this in my project. But still i can see 0 lines for the files to which no test cases are available. I can see some coverage lines in all the files only if i mount all the components with dummy configuration. Is there any other work around?

I continued upon the quoted solution. To make sure that coverage is created for every Typescript file in my application directory I create a glob pattern and then execute it for every file in the glob. This implicitly imports the file. Now my code coverage report is complete.

// @ts-ignore
const allFilesGlobExcludingCypress: Record<string, () => Promise<unknown>> = import.meta.glob([
  './**/*.{tsx,ts}',
  '!./**/*.cy.tsx',
]);

describe('CoverageWorkaround', () => {
  it('should import all files to make coverage report complete', { defaultCommandTimeout: 60000 }, async () => {
    const isCoverageEnabled = Cypress.env('coverage');
    if (!isCoverageEnabled) {
      return;
    }

    for (const file in allFilesGlobExcludingCypress) {
      await allFilesGlobExcludingCypress[file]();
    }
  });
});