integrated test runner

[basarat]

refs #177

• Start worker
• provide errors for tested.json
• tested worker should auto boot restart in the beginning.
• instead of auto starting it should start with an explicit command setting the working dir.
• Collect stats at each level
• Inform when tests are running
• Inline fail result info
• Remove mocha stack from stack
• Source maps are crashing : TypeError: Column must be greater than or equal to 0, got -62 evanw/node-source-map-support#148 Fix : Defensive coding to prevent out-of-range column evanw/node-source-map-support#138. Do that as post install script while its not fixed upstream. Does not happen latest node!
• Add a test result overview view
  • Highlight selected test
    • Render suites + pass / fail at each level
• Render tests + duration / pass / fail
• Suite logo
  • It logo
• Update gls / csx
• Update TypeScript
  • Update typescript for file path completions
  • Give async/await for es5 a go ;)
• See what happens when you delete alm.json
• Rerun all test when any file in compilation context changes for now?
• action all TODO: tested
• Memo into testing
  • alm.json
  • alm.json sample for .ts + .tsx
  • Gif about it
  • Clicking on status bar opens the test view
• Breaking change: Need node 6

Both log and error have a content + Stack. Both should display at last location in test file with a mention to origin file if it is different. {lastLocationInFile, stack}

Secondary

• Use _mocha instead of mocha as it forks : Configuring istanbul with mocha gotwarlost/istanbul#44 (comment)

Instrumentation research

mochaInstrumentation that notes down any access to describe (with skip etc), it (with skip etc) and console (with log etc)

IDEA: custom interface

mocha/lib/interfaces contains what are called UIs. You can customize them using https://github.com/mochajs/mocha/wiki/Third-party-UIs

So we could do something like :

mocha --require ourUi.js someTest.ts

where ourUi.js has:

const origBDD = Mocha.interfaces["bdd"];
Mocha.interfaces["bdd"] = /** redefine `describe` and it */

The bdd interface is what thrashes our attempts to override the global describe between tests : https://github.com/mochajs/mocha/blob/4a87a947d0109af74400fd09399eb3e171fab761/lib/interfaces/bdd.js#L41

Final js sample

console.log('---INTERCEPTER Start');

var Mocha = require('mocha');
const origBDD = Mocha.interfaces["bdd"];
Mocha.interfaces["bdd"] = function(suite) {
    // Still do what the original one did to let its `pre-require` pass
    origBDD(suite);

    // And attach our own custom pre-require to fixup context watchers
    suite.on('pre-require', function(context, file, mocha) {
        const origDescribe = context.describe;
        const origDescribeOnly = context.describe.only;
        const origDescribeSkip = context.describe.skip;
        context.describe = function(title){
            console.log("describe title", title)
            return origDescribe.apply(context,arguments);
        }
        context.describe.only = function(title){
            console.log("describe title", title)
            return origDescribeOnly.apply(origDescribe,arguments);
        }
        context.describe.skip = function(title){
            console.log("describe title", title)
            return origDescribeSkip.apply(origDescribe,arguments);
        }

        const origIt = context.it;
        const origItOnly = context.it.only;
        const origItSkip = context.it.skip;
        context.it = function(title){
            console.log("It title", title)
            return origIt.apply(context,arguments);
        }
        context.it.only = function(title){
            console.log("It title", title)
            return origItOnly.apply(origIt,arguments);
        }
        context.it.skip = function(title){
            console.log("It title", title)
            return origItSkip.apply(origIt,arguments);
        }
    });
}

console.log('---INTERCEPTER END');

IDEA figure out which functions mocha calls

We managed describe control with:

/// Mock out describe
const suiteCreate = require('mocha/lib/suite').create;
require('mocha/lib/suite').create = function(parent, title){
    // console.log("Describe:", title, parent.title);
    return suiteCreate.apply(null,arguments);
}

But its not optimal as : stack is 1 removed from the actual describe call.
Could not figure out how to mock out it from require('mocha/lib/test'); as its exported as a function :-/

[tomitrescak]

Wow, you're serious? This will be great when implemented ;) I can;t wait to test that!

[basarat]

Released with major version 2.0.0. One needs node 6 to run alm now ❤️

Docs : https://basarat.gitbooks.io/alm/content/features/testing.html 🌹