Vale workflow does not fail even if errors are present

[thomas-moulard]

vale reports: Vale: Found 0 suggestion(s), 0 warning(s), and 26 error(s). but the action finishes without reporting an error. Is this the expected behavior?

2019-12-12T16:27:27.9607635Z ##[group]Run errata-ai/[email protected]
2019-12-12T16:27:27.9607806Z with:
2019-12-12T16:27:27.9608047Z   styles: https://github.com/errata-ai/write-good/releases/latest/download/write-good.zip
https://github.com/errata-ai/Google/releases/latest/download/Google.zip

2019-12-12T16:27:27.9608301Z   config: https://gist.githubusercontent.com/thomas-moulard/81e4fc260bddeb4149838b884ba86701/raw/cf331fe24d43fc2f02be4b40c5e83051a9e62130/vale.ini
2019-12-12T16:27:27.9608479Z   files: all
2019-12-12T16:27:27.9608595Z env:
2019-12-12T16:27:27.9608820Z   GITHUB_TOKEN: ***
2019-12-12T16:27:27.9608938Z ##[endgroup]
2019-12-12T16:27:27.9638290Z ##[command]/usr/bin/docker run --name af96b460939e6e3b7641f980748a9c246201f9_842036 --label af96b4 --workdir /github/workspace --rm -e GITHUB_TOKEN -e INPUT_STYLES -e INPUT_CONFIG -e INPUT_FILES -e HOME -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e GITHUB_ACTIONS=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/cross_compile/cross_compile":"/github/workspace" af96b4:60939e6e3b7641f980748a9c246201f9  "https://github.com/errata-ai/write-good/releases/latest/download/write-good.zip
https://github.com/errata-ai/Google/releases/latest/download/Google.zip
" "https://gist.githubusercontent.com/thomas-moulard/81e4fc260bddeb4149838b884ba86701/raw/cf331fe24d43fc2f02be4b40c5e83051a9e62130/vale.ini" "all"
2019-12-12T16:27:29.0454385Z (node:1) ExperimentalWarning: The fs.promises API is experimental
2019-12-12T16:27:29.0484285Z [command]/bin/vale -v
2019-12-12T16:27:29.0947523Z vale version v2.0.0-beta.2
2019-12-12T16:27:29.1000699Z Downloading external config 'https://gist.githubusercontent.com/thomas-moulard/81e4fc260bddeb4149838b884ba86701/raw/cf331fe24d43fc2f02be4b40c5e83051a9e62130/vale.ini' ...
2019-12-12T16:27:29.2875148Z Successfully fetched remote config.
2019-12-12T16:27:29.2897208Z Installing style 'write-good' ...
2019-12-12T16:27:29.2919666Z [command]/bin/vale --no-exit --output=JSON --mode-rev-compat --config=/github/workspace/tmp-1b4DHBUaX0VWo.ini install write-good https://github.com/errata-ai/write-good/releases/latest/download/write-good.zip
2019-12-12T16:27:29.3360053Z Installing style 'Google' ...
2019-12-12T16:27:29.3360780Z [command]/bin/vale --no-exit --output=JSON --mode-rev-compat --config=/github/workspace/tmp-1b4DHBUaX0VWo.ini install Google https://github.com/errata-ai/Google/releases/latest/download/Google.zip
2019-12-12T16:27:29.3801762Z Vale set-up comeplete; using '--no-exit,--output=JSON,--mode-rev-compat,--config=/github/workspace/tmp-1b4DHBUaX0VWo.ini,.'.
2019-12-12T16:27:29.8125700Z Vale: Found 0 suggestion(s), 0 warning(s), and 26 error(s).
2019-12-12T16:27:30.9211134Z Post job cleanup.
2019-12-12T16:27:31.0285539Z [command]/usr/bin/git version
2019-12-12T16:27:31.0353317Z git version 2.24.0
2019-12-12T16:27:31.0391443Z [command]/usr/bin/git config --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2019-12-12T16:27:31.0427691Z http.https://github.com/.extraheader
2019-12-12T16:27:31.0437506Z [command]/usr/bin/git config --unset-all http.https://github.com/.extraheader
2019-12-12T16:27:31.0532231Z Cleaning up orphan processes

logs | raw logs

[jdkato]

Yes and no: the initial job should pass (as shown above), but there should be a second failing job named "Vale" that actually produces the annotations (see here for example).

This is a workaround until annotations are officially added to the toolkit (actions/toolkit#186). They've also since added docs for Problem Matchers, though, which may be a better alternative.