RFC: implement unit testing framework

[mikedanese]

Supports tests of type function, and "table tests". Output looks like:

Running tests/arith_bool.jsonnet...
--- PASS 'Test !'
        pass    'case 0'
        pass    'case 1'
        pass    'case 2'
        pass    'case 3'
--- PASS 'Test !='
        pass    'case 0'
        pass    'case 1'
        pass    'case 2'
        pass    'case 3'
--- PASS 'Test &&'
        pass    'case 0'
        pass    'case 1'
        pass    'case 2'
        pass    'case 3'
--- PASS 'Test =='
        pass    'case 0'
        pass    'case 1'
        pass    'case 2'
        pass    'case 3'
--- PASS 'Test if'
        pass    'case 0'
        pass    'case 1'
--- PASS 'Test no implicit conversions'
        pass    'case 0'
        pass    'case 1'
--- PASS 'Test shortcut semantics'
        pass    'case 0'
        pass    'case 1'
--- PASS 'Test ||'
        pass    'case 0'
        pass    'case 1'
        pass    'case 2'
        pass    'case 3'

Running tests/merge.jsonnet...
--- PASS 'Test std.mergePatch'
        pass    'case 0'
        pass    'case 1'
        pass    'override with null'
        pass    'case 3'
        pass    'case 4'
        pass    'case 5'
        pass    'case 6'
        pass    'case 7'
        pass    'case 8'
        pass    'case 9'
        pass    'case 10'
        pass    'case 11'
        pass    'case 12'
        pass    'case 13'

OK

[akazakov]

👍

[akazakov]

typo: evaluate
same 2 lines down

[mikedanese]

Tests can also be written as functions. e.g. these are valid:

local expect = (import "stdlib/truth.jsonnet").expect;
local expectations = (import "stdlib/truth.jsonnet").expectations;

{
    "Test test function passing": function()
        local a = 1 + 2;
        expectations([
            expect()
                .that(a)
                .hasType("number"),
            expect()
                .that(a)
                .isEqualTo(3),
        ]),

    "Test test function failing"():
        local a = 1 + 2;
        expect()
            .that(a)
            .isEqualTo(4),
}

One huge drawback with this approach is that it's still impossible to test routines that throw exceptions

[sparkprime]

This is definitely interesting. In the case of the arith_bools it has more than tripled in size. Ideally there would have been a reduction in size. For the sake of argument, maybe it could look like this:

{
    "Test !": unit.SimpleInputOutput {
        cases: [
            self.Case(!false, true),
            self.Case(!true, false),
            ...
        ]
    }
}

I also think we should try to use this framework for something more "real". How about your Kubernetes stuff? If not, there is a whole bunch of abstract templates in micromanage we can use.

I think there's a big win for unit testing library abstractions (whether mixins or functions) that check that they can be instantiated into either something specific, or at least into something that has a set of expected properties. It would help document their interfaces as well as allowing refactoring with confidence.

[mikedanese]

I'll close this for now since this can be easily implemented in a library.