Error in modules's nested executeNow call forces subsequent modules to become children

[Eric162]

The executeNow.call is not protected, which means when that if that fails, the rest of processModule is not executed.

qunit/src/module.js

Line 83 in e901be8

executeNow.call( module.testEnvironment, moduleFns );

Because of this, the module which failed becomes the currentModule, AKA the parent of the rest of the modules, and since it has the { skip: true } option, its children will all be skipped (which is now every module loaded after this one).

See this CodePen for an example:
https://codepen.io/eric162/pen/mdPmjJZ?editors=1010

The issue is especially apparent with module.skip, as it would appear that all tests have passed unless a close inspection of the test output is made.

With tools like ember-exam which do load balancing and load modules dynamically, this could mean that 1 module may be skipped, or hundreds/thousands in large test suites. Developers unfamiliar with the usual number of skipped tests in a repo would not know how many are supposed to be skipped, and even those who are familiar may not notice if it happens to be a small increase.

Though this explanation references how Ember uses qunit, the issue may be best resolved here.

Links from codepen comments below.

Ember-cli-test-loader:
https://github.com/ember-cli/ember-cli-test-loader/blob/bb4f22a525d3698454c14f8091be4d90522adfff/addon-test-support/index.js#L19

Ember-qunit:
https://github.com/emberjs/ember-qunit/blob/ef3e5ee10880be849509c1084436b311aa875e58/addon-test-support/test-loader.js#L35

[Krinkle]

I suspect this maybe related to #1407 or #1416, which I'm currently working on fixing for the next patch release.

[Krinkle]

#1416 has been fixed and released (in QUnit 2.13), but I can still reproduce this bug with QUnit 2.14:

https://codepen.io/Krinkle/pen/ExNMxrg