When test fails in async callback it aborts other tests

[pselden]

describe('mocha tests', function() {
    it('should go to the next test when it fails (outside callback)', function(done){
        throw new Error('the other test should still run');
        done();
    });

    it('should go to the next test when it fails (inside callback)', function(done) {

        setTimeout(function() {
            throw new Error('the other test should still run');
            done();
        }, 0);
    });

    it('should get to this test', function(done) {
        done();
    });
});

I expect that it should run all the tests and report 2 failures and 1 success. Instead I get this (only 2 out of 3 tests were run):

mocha ./test/test.test.js

  ..

  ✖ 2 of 3 tests failed:

  0) mocha tests should go to the next test when it fails (outside callback):
     Error: the other test should still run

  at Test.fn (/Users/me/mochatest/test/test.test.js:3:19)
      at Test.run (/usr/local/lib/node_modules/mocha/lib/runnable.js:103:12)
      at Runner.runTest (/usr/local/lib/node_modules/mocha/lib/runner.js:244:10)
      at /usr/local/lib/node_modules/mocha/lib/runner.js:290:12
      at next (/usr/local/lib/node_modules/mocha/lib/runner.js:172:14)
      at /usr/local/lib/node_modules/mocha/lib/runner.js:181:7
      at next (/usr/local/lib/node_modules/mocha/lib/runner.js:133:23)
      at Array.0 (/usr/local/lib/node_modules/mocha/lib/runner.js:149:5)
      at EventEmitter._tickCallback (node.js:192:40)

  1) mocha tests should go to the next test when it fails:
     Error: the other test should still run

  at Timer.ontimeout (/Users/me/mochatest/test/test.test.js:10:23)

This is the same if I'm running multiple files.

[tj]

yeah I'll see if I can fix that, the tricky thing is handling stuff like:

  it('should bar', function(done){
    process.nextTick(function(){
      done();
      setTimeout(function(){
        throw new Error('fail in bar');
      }, 100);
    });
  })

  it('should baz', function(done){
    setTimeout(function(){
      done();
    }, 1000);
  })

that will map it to baz.

but yeah when it's uncaught you lose the stack so recovering from that and continuing tests might end up being too much of a hack

[pselden]

The mapping issue seems like a test case bug rather than a framework issue -- the user should always be calling done after the final callback right?

I'm using should.js for my assertion library and basically can't use it inside any callbacks (expectations from api calls, etc...) because it throws (I ended up wrapping everything in try-catch and calling done(err) with the exception as a messy workaround).

[tj]

well nothing stops some internal api from doing the same.

but no you shouldn't have to try/catch, I don't in any of mine and they are in callbacks as well, i'll take a look, i think i can reproduce this

[tj]

nvm seems fine for me:

  it('should fail', function(done){
    process.nextTick(function(){
      'foo'.should.equal('bar');
    });
  })

outputs:

  ✖ 1 of 47 tests failed:

  0) global leaks should fail:
     AssertionError: expected 'foo' to equal 'bar'
      at Object.equal (/Users/tj/Projects/mocha/node_modules/should/lib/should.js:306:10)
      at Array.<anonymous> (/Users/tj/Projects/mocha/test/globals.js:33:20)
      at EventEmitter._tickCallback (node.js:126:26)

[pselden]

Does it actually run the other tests?

it('should fail', function(done){
   process.nextTick(function(){
      'foo'.should.equal('bar');
  });
});

it('should fail too', function(done){
   should.fail('fail');
});

it('should pass', function(done){
   done();
});

$:mochatest pselden$ mocha test.js -r should

  .

  ✖ 1 of 3 tests failed:

  0)  should fail:
     AssertionError: expected 'foo' to equal 'bar'
      at Object.equal (/usr/local/lib/node_modules/should/lib/should.js:306:10)
      at Array.0 (/Users/pselden/Documents/mochatest/test.js:3:20)
      at EventEmitter._tickCallback (node.js:192:40)

It should be two failures but the other one isn't run. I just upgraded to 0.2.0 to check if that was it but still doesn't work. If you can't reproduce with that test case then I will dig in and start debugging.

I'm on node 0.6.1 if that matters.

[tj]

we need to tweak https://github.com/visionmedia/mocha/blob/master/lib/runner.js#L369 removing "end", clear the timer and continue on

[rdingwall]

I still seem to be getting this problem?? Using https://github.com/visionmedia/mocha/blob/master/mocha.js 1.0.2 5a5f99f

[artem-karpenko]

I experience this problem with mocha 2.2.5.

[alex-di]

Same on 2.3.3

[danielstjules]

The issue highlighted in #94 (comment) is not a problem in 2.3.4. Note that both failures are reported, rather than only 1 in the prior example.

$ mocha --version
2.3.4
danielstjules:~/Desktop

$ cat test.js
var assert = require('assert');

it('should fail', function(done){
   process.nextTick(function() {
      assert(false);
  });
});

it('should fail too', function(done){
   assert(false);
});

it('should pass', function(done){
   done();
});

danielstjules:~/Desktop
$ mocha test.js


  1) should fail
  2) should fail too
  ✓ should pass

  1 passing (12ms)
  2 failing

  1)  should fail:

      Uncaught AssertionError: false == true
      + expected - actual

      -false
      +true

      at test.js:5:7

  2)  should fail too:

      AssertionError: false == true
      + expected - actual

      -false
      +true

      at Context.<anonymous> (test.js:10:4)

[rawqing]

same on 6.0.2
$ ./node_modules/mocha/bin/mocha --version 6.0.2