testing Mocha with wallaby

[boneskull]

Issue description or question

I apologize if I asked this already, but I'm going to give it another try. If we can get some portion of our test suite running, then that's good enough!

I'm running into some issues when trying to run Mocha's unit tests. I think Wallaby must contain its own version of Mocha, or it's just simply being pulled from somewhere else that I'm unaware of. I'll just start with one example for now.

We have this test. It's simply supposed to grab the main file (as in package.json) from the project, then use its exported functions to run tests.

Wallaby reports this:

TypeError: describe is not a function
	at Object.<anonymous> test/unit/required-tokens.spec.js:7

Debug window gives a longer trace:

Runtime error: TypeError: describe is not a function
    at Object.<anonymous> (./test/unit/required-tokens.spec.js:6:14)
    at Module._compile (module.js:660:30)
    at Object.Module._extensions..js (module.js:671:10)
    at Module.load (module.js:573:32)
    at tryModuleLoad (module.js:513:12)
    at Function.Module._load (module.js:505:3)
    at Module.require (module.js:604:17)
    at require (internal/module.js:11:18)
    at Array.forEach (<anonymous>)
    at WebSocket.emit (events.js:160:13)
    at afterWrite (_stream_writable.js:474:3)
    at onwrite (_stream_writable.js:465:7)
    at InflateRaw.afterTransform (_stream_transform.js:89:3)
    at Zlib.processCallback (zlib.js:588:8)

Changing worker count seems to have no positive effect.

Wallaby.js configuration file

'use strict';

module.exports = () => {
  return {
    files: [
      'index.js', 'lib/**/*.js', 'test/setup.js',
      {
        pattern: 'test/node-unit/**/*.fixture.js',
        instrument: false
      }, {
        pattern: 'test/unit/**/*.fixture.js',
        instrument: false
      }
    ],
    filesWithNoCoverageCalculated: ['test/**/*.fixture.js'],
    tests: [
      'test/unit/**/*.spec.js', 'test/node-unit/**/*.spec.js'
    ],
    env: {
      type: 'node',
      runner: 'node'
    },
    testFramework: 'mocha',
    setup (wallaby) {
      require('./test/setup');
    },
    debug:true
  };
};

There are a couple more problems that I'm suspecting are of the same nature; certain methods being undefined such as this.skip() or this.retries() being unavailable, which were recent changes in Mocha.

It should be noted that if Wallaby is looking for node_modules/mocha, it ain't there. But setting NODE_PATH=${__dirname} causes the following to repeat over and over:

TypeError: Mocha is not a constructor
    at WebSocket.emit (events.js:160:13)
    at afterWrite (_stream_writable.js:474:3)
    at onwrite (_stream_writable.js:465:7)
    at InflateRaw.afterTransform (_stream_transform.js:89:3)
    at Zlib.processCallback (zlib.js:588:8)TypeError: Cannot read property 'reporter' of undefined
    at WebSocket.emit (events.js:160:13)
    at afterWrite (_stream_writable.js:474:3)
    at onwrite (_stream_writable.js:465:7)
    at InflateRaw.afterTransform (_stream_transform.js:89:3)
    at Zlib.processCallback (zlib.js:588:8)

Code editor or IDE name and version

WebStorm 2017.3.3
Build #WS-173.4301.22, built on January 15, 2018
Licensed to Mocha /
Subscription is active until November 2, 2018
For non-commercial open source development only.
JRE: 1.8.0_152-release-1024-b11 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
Mac OS X 10.13.2

[ArtemGovorov]

@boneskull
There are always two mocha instances running, one that is running the test suite, another is the mocha under test (in wallaby case it is the source code instrumented for code coverage).

It should be noted that if Wallaby is looking for node_modules/mocha, it ain't there.

Yes, wallaby tries to load mocha from node_modules and if it is not there, then it uses its own embedded version. However, you may specify a path to any mocha version via the testFramework setting.

It should be possible to run mocha unit tests with mocha in wallaby with few changes in the config:

• to specify the version of mocha that wallaby runs, you may use: testFramework: {type: 'mocha', path: __dirname},,
• to eliminate any caching issues, when you add new features/change features in running mocha and would like to immediately use them, you may use the workers: {recycle: true}, setting,
• to expose describe/it on the mocha under test (that is different from running mocha), suite.emit('pre-require', ... call needs to be made,
• to make one the tests pass, cp.execFile needs to be patched to execute in the context of your project (as opposed to the wallaby cache folder with instrumented files).

So the following wallaby config should work for you:

'use strict';

module.exports = () => {
  return {
    files: [
      'index.js', 'lib/**/*.js', 'test/setup.js',
      {
        pattern: 'test/node-unit/**/*.fixture.js',
        instrument: false
      }, {
        pattern: 'test/unit/**/*.fixture.js',
        instrument: false
      }
    ],
    filesWithNoCoverageCalculated: ['test/**/*.fixture.js'],
    tests: [
      'test/unit/**/*.spec.js', 'test/node-unit/**/*.spec.js'
    ],
    env: {
      type: 'node',
      runner: 'node'
    },
    workers: {recycle: true},
    testFramework: {type: 'mocha', path: __dirname},
    setup (wallaby) {
      // running mocha instance is not the same as mocha under test,
      // running mocha is the project's source code mocha, mocha under test is instrumented version of the source code
      const runningMocha = wallaby.testFramework;
      runningMocha.timeout(200);
      // to expose it/describe etc. on the mocha under test
      const mochaUnderTest = new (require('./'))();
      mochaUnderTest.suite.emit('pre-require', global, '', mochaUnderTest);
      // to make test/node-unit/color.spec.js pass, we need to run mocha in the project's folder context
      const childProcess = require('child_process');
      const execFile = childProcess.execFile;
      childProcess.execFile = function () {
        let opts = arguments[2];
        if (typeof opts === 'function') {
          opts = {};
          Array.prototype.splice.call(arguments, 2, 0, opts);
        }
        opts.cwd = wallaby.localProjectDir;
        return execFile.apply(this, arguments);
      };
      require('./test/setup');
    },
    debug: true
  };
};