Test output is confusing and hard to debug

[adamnovak]

I saw that #184 failed its tests, but I couldn't find the failing test by looking at the logs.

It turns out it is somewhere in this section:

    console.log
      A connection has been closed

      at WebSocketConnection.<anonymous> (src/server.js:1085:17)

    console.log
      TubeMapServer stopped.

      at Object.close (src/server.js:1033:17)

  ● produces correct link for view before & after go is pressed

    expect(received).toEqual(expected) // deep equality

    Expected: "localhost?name=snp1kg-BRCA1&xgFile=snp1kg-BRCA1.vg.xg&gamFile=NA12878-BRCA1.sorted.gam&dataPath=default&region=17%3A1-100&bedFile=snp1kg-BRCA1.bed&dataType=built-in"
    Received: "http://localhost?name=snp1kg-BRCA1&xgFile=snp1kg-BRCA1.vg.xg&gamFile=NA12878-BRCA1.sorted.gam&dataPath=default&region=17%3A1-100&bedFile=snp1kg-BRCA1.bed&dataType=built-in"

      273 |   await clickCopyLink();
      274 |   // Ensure link reflects our selected data
    > 275 |   expect(fakeClipboard).toEqual(expectedLinkBRCA1);
          |                         ^
      276 |
      277 |   // Set up dropdown
      278 |   await act(async () => {

      at Object.<anonymous>.it (src/end-to-end.test.js:275:25)

PASS src/components/RegionInput.test.js
PASS src/App.test.js

produces correct link for view before & after go is pressed turns out to be the failing test. There's no explicit mention of it failing, and it's burred among four-line-each console.log reports from other tests passing before and after it. In the terminal it would be red which sort of helps if you can see red, but on Github Actions it isn't.

jestjs/jest#4156 was never really fixed, so we don't have a good way to only see the logs for the suites that fail, unless we manually implement something like jestjs/jest#4156 (comment). jestjs/jest#3322 was also never really fixed, so there's no built-in way to get a list of the failing tests at the end of the run, unless we pull in one of the custom reporters mentioned there.

We should pull in some third-party code or something to make it easier to find the actually-failing tests from the de-colorized logs.

[adamnovak]

It looks like right now to find this you would have to search maybe for "Expected" or for the "●" character (which would also take you to the console logs).

[rmccrear]

There is a test reporter called jest-compact-reporter here.

It reports a list of how many tests ran, passed and failed. Then, it shows only the errors from the failed tests.

You can use it with react-scripts and the default jest test reporter by replacing the "test" script in your package.json file with this:

"test": "react-scripts test --reporters default --reporters jest-compact-reporter",

If you use both of the reporters, as above, you can get all the results from the default test reporter as usual, plus the compact results without all the console.log output interspersed. Then you can search for those failing tests in the default results to see more details.

A failing test of 1 + 1 === 1 would look something like this on GitHub Actions.

[adamnovak]

We have compact reporter now, but this could maybe still be better.

[adamnovak]

I think #410 might have fixed this.