Not using --recursive causing problems?

[jescalan]

Hey there,

Thanks for making such a fantastic project! Just a quick question -- I'm setting up a existing large project that formerly compiled code in order to get coverage so that it works with coffee-coverage. However, I'm finding that the report generated shows little to no actual coverage at all, while the compiled report was quite high and the project is tested thoroughly. I have followed instructions exactly, and it's not erroring out, just giving an incorrect report. The one thing I am not including is the --recursive flag, as it's not needed to run my test suite. I tried including this, but then mocha tried to run some of the fixtures included in the tests folder and errored.

Is the --recursive flag necessary to generate correct coverage? If not, what further information could I provide to make this incorrect coverage issue easier to shed light on? And if so, how do people typically deal with fixtures being run as tests with the recursive flag?

Thanks again!

[jwalton]

--recursive shouldn't be needed for mocha (unless you actually have tests that need it, obviously.) I wrote a really, really high level overview of how coffee-coverage works over here if that's at all helpful.

The obvious thing to check off the top of my head is; Do your unit tests run straight from your .coffee files, or are they loading your compiled .js files? If you're running from compiled code, and trying to use run-time instrumentation, the coffee-coverage never has a chance to get in there and do it's thing.

[jescalan]

The tests run straight from coffee files. The entire project is coffee actually, there's no js and no compiling (except for a task I use to convert before publishing, which was formerly used for js instrumentation before I switched to coffee-coverage).\

This is the project, if that helps at all. package.json points to the lib folder and loads up index.coffee.

[jwalton]

Just gave this a quick try, and I'm seeing statement coverage of 40.2% (361 / 898), which I'm guessing is too low? (I'm also seeing 100 tests passing and 9 pending.)

[jwalton]

I'm wondering if this:

--require test/support/all
--require test/support/helpers

is the problem. I'm wondering if mocha is somehow loading these files without coverage...

[jwalton]

If I do:

--require coffee-coverage/register-istanbul
--require test/support/all
--require test/support/helpers

I get 40% coverage. If I do:

--require test/support/all
--require test/support/helpers
--require coffee-coverage/register-istanbul

I get 18% coverage... So that's maybe not it. :/

[jwalton]

If I hack up coffee-coverage to show what it's instrumenting, then I see:

10:54:42 [jwalton@nimue ~/Development/roots (master)] $ npm run coverage

> [email protected] coverage /Users/jwalton/Development/roots
> mocha && istanbul report

Instrumenting /Users/jwalton/Development/roots/lib/index.coffee
Instrumenting /Users/jwalton/Development/roots/lib/analytics.coffee
Instrumenting /Users/jwalton/Development/roots/lib/global_config.coffee
Instrumenting /Users/jwalton/Development/roots/lib/config.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions/compiled.coffee
Instrumenting /Users/jwalton/Development/roots/lib/extensions/static.coffee
Instrumenting /Users/jwalton/Development/roots/lib/api/bail.coffee
Excluding /Users/jwalton/Development/roots/test/analytics.coffee
Excluding /Users/jwalton/Development/roots/test/cli.coffee
Instrumenting /Users/jwalton/Development/roots/lib/cli/index.coffee
Excluding /Users/jwalton/Development/roots/test/compile.coffee
Excluding /Users/jwalton/Development/roots/test/deploy.coffee
Excluding /Users/jwalton/Development/roots/test/extensions.coffee
Excluding /Users/jwalton/Development/roots/test/index.coffee
Excluding /Users/jwalton/Development/roots/test/new.coffee
Excluding /Users/jwalton/Development/roots/test/template.coffee
Excluding /Users/jwalton/Development/roots/test/watcher.coffee



Instrumenting /Users/jwalton/Development/roots/lib/api/template.coffee
Instrumenting /Users/jwalton/Development/roots/lib/sprout.coffee
  analytics
    ✓ should track events when the __track function is called (309ms)
    ✓ should be able to disable analytics through the api
    ✓ should be able to enable analytics through the api (266ms)
...

But then I never see any more coffee files get instrumented. It's like the .coffee extension handler is somehow getting un-registered. Hrm.

[jescalan]

Huh, very strange. Thanks for investigating this, and glad it's not just me doing something stupid! If there's anything else I can help with here happy to do so, just let me know 😀

[jwalton]

Did you ever get this working, BTW?

[jescalan]

Nope. I've got it working with other projects, but not with this one 😢