Add xpath support for Problem Matcher

[stefanbuck]

Describe the enhancement
Sniffing through logs using RegExp can be hairy especially if your need multiline line matching. In my particular use case I want to write a Problem Matcher for jest. The output from jest is very human friendly.

 FAIL  ./app.spec.js
  compare a and b
    ✕ should fail (7ms)

  ● compare a and b › should fail

    expect(received).toBe(expected) // Object.is equality

    Expected: "b"
    Received: "a"

      1 | describe('compare a and b', function() {
      2 |     it('should fail', function() {
    > 3 |         expect('a').toBe('b');
        |                     ^
      4 |     });
      5 | });
      6 |

      at Object.<anonymous> (app.spec.js:3:21)

Jest provides a CLI argument to print the test results in JSON. However, parsing this output using RegExp isn't that simple either.

What if the Problem Matcher API would allow us to use xpath to grep values from a json file on disc. Such an API would be useful and quite powerful addition.

Code Snippet

{
    "problemMatcher": [
        {
            "owner": "jest",
            "pattern": [
                {
                    "source": "./path/to/file.json",
                    "file": $.testResults[*].name,
                    "message": $.testResults[*].assertionResults[*].failureMessages
                }
            ]
        }
    ]
}

[ericsciple]

i'll defer to others, but in general we've aligned with vscode problem matchers

[ericsciple]

@chrispat fyi

[stefanbuck]

Is there a way to help moving this forward? I really want to see this happening 😉

[ericsciple]

Problem matchers plugin to how the runner processes streaming output. I'm not sure this fits into that plugin model.

It sounds like you want to process the json file, and create error/warning annotations.

It is possible, but it looks like core.error() doesnt expose the file/line/column parameters today. Even though the runner supports it. As a workaround you could manually echo ::error file=the-source-file-path::the error message to stdout.

[ericsciple]

related to #186

[chrispat]

Problem matchers are for parsing console output and are run line by line. In you case you want a file and the runner won't when to process the file. One option would be to write an action to process the file and create annotations.

[billyvg]

@chrispat I'm working on an action that runs eslint with --fix and commits the fixed files using the octokit client. I'm using eslint's node api to run the linter which returns a JSON. Having an API (as suggested in #186) to add annotations would be great.

I'm also wondering if another option would be to add a problem matcher for the JSON itself? e.g. Create kind of a custom format with the JSON and just calling console.log(). Would the problem matchers be able to pick those up?

[chrispat]

You can pretty easily create your own annotations by issuing commands like https://help.github.com/en/actions/reference/development-tools-for-github-actions#set-a-warning-message-warning. The toolkit function https://github.com/actions/toolkit/blob/master/packages/core/src/command.ts#L19 makes it easy to format the parameters for the command.

The problem matcher is really designed to mostly work against text log output at a single line level. While I am sure it would be possible to build something that would recognize JSON in an output stream I am not sure it is practical or a good use of time right now.