Empty Vale Report on GitHub but not locally

[fjp]

Hi, I am trying to use the errata-ai/vale-action@reviewdog action on GHE but the output is empty:

Running Vale locally with vale --glob='*.{md,txt}' . results in a bunch of expected suggestions/errors in the output report.

The workflow steps for Vale are:

name: lint
on:
  pull_request:
    branches:
      - main
  
jobs:
  vale:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Vale
        uses: errata-ai/vale-action@reviewdog
        with:
          debug: true
          fail_on_error: true
          reporter: github-pr-check
          # added (default), diff_context, file, nofilter
          filter_mode: added
          vale_flags: --glob="*.{md,txt}"

        env:
          # Required, set by GitHub actions automatically:
          # https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret
          GITHUB_TOKEN: ${{secrets.GITHUB_TOKEN}}

The output of the build step is

Run errata-ai/vale-action@reviewdog
/usr/bin/docker run --name e28767c079dabc6d4c2c9503e976582b2fe0_d627cd --label 85e287 --workdir /github/workspace --rm -e GITHUB_TOKEN -e INPUT_DEBUG -e INPUT_FAIL_ON_ERROR -e INPUT_REPORTER -e INPUT_FILTER_MODE -e INPUT_VALE_FLAGS -e INPUT_VERSION -e INPUT_FILES -e INPUT_LEVEL -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e GITHUB_STEP_SUMMARY -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/opt/github-action-runner/_work/_temp/_github_home":"/github/home" -v "/opt/github-action-runner/_work/_temp/_github_workflow":"/github/workflow" -v "/opt/github-action-runner/_work/_temp/_runner_file_commands":"/github/file_commands" -v "/opt/github-action-runner/_work/docs/docs":"/github/workspace" 85e287:67c079dabc6d4c2c9503e976582b2fe0  "latest" "all" "true" "github-pr-check" "true" "error" "added" "--glob=\"*.{md,txt}\""
Installing Vale version 'latest' ...
/bin/tar xz --overwrite -C /github/home -f /opt/github-action-runner/_work/_temp/cbb8a33e-206a-4dbd-83a2-04b7f3d39c9d
Installed version '2.20.[1](.../runs/180368?check_suite_focus=true#step:4:1)' into '/github/home/vale'.
Using Vale 2.20.1
/github/home/vale --glob="*.{md,txt}" sync

Downloading packages [0/6] █                                             0% | 0s
                                                                                
 SUCCESS  Downloaded package 'Google'

Downloading packages [1/6] ████████                                     [17](.../runs/180368?check_suite_focus=true#step:4:17)% | 2s
                                                                                
 SUCCESS  Downloaded package 'proselint'

Downloading packages [2/6] ███████████████                              33% | 3s
                                                                                
 SUCCESS  Downloaded package 'write-good'

Downloading packages [3/6] ██████████████████████                       50% | 4s
                                                                                
 SUCCESS  Downloaded package 'alex'

Downloading packages [4/6] █████████████████████████████                67% | 5s
                                                                                
 SUCCESS  Downloaded package 'Readability'

Downloading packages [5/6] ████████████████████████████████████         83% | 6s
                                                                                
 SUCCESS  Downloaded package 'Joblint'

Downloading packages [6/6] ███████████████████████████████████████████ 100% | 7s
Vale set-up comeplete; using '--output=/lib/rdjsonl.tmpl,--glob="*.{md,txt}",.'.
Running vale with reviewdog 🐶 ...
  /github/home/vale --output=/lib/rdjsonl.tmpl --glob="*.{md,txt}" .
  /bin/reviewdog -f=rdjsonl -name=vale -reporter=github-pr-check -fail-on-error=true -filter-mode=added -level=info
  2022/08/30 12:54:06 [vale] reported: .../runs/[18](.../runs/180368?check_suite_focus=true#step:4:18)0369 (conclusion=success)

Do you have any suggestion what might cause the empty reports or how to debug it?

To me it seems Vale is executed in a directory that contains no content files.

Is there a way to check which path is used to execute vale or to pass the -tee debugging option to reviewdog to debug the issue?

[fjp]

An example of an empty report is found here: https://github.com/ros-mobile-robots/ros-mobile-robots.github.io/pull/5/checks?check_run_id=8102264712

Is there any setting I am missing?

[lithammer]

I had similar issues. And it seems to be related to how vale_flags is used. I looked into the implementation and here's what I think is going on. The flags string is split into an array based on whitespace:

vale-action/src/input.ts

Line 16 in 8dd360d

return flags.split(/\s+/);

And the "splatted" into an argument list here:

vale-action/src/input.ts

Lines 78 to 81 in 8dd360d

	let args: string[] = [
	`--output=${path.resolve(__dirname, 'rdjsonl.tmpl')}`,
	...parse(valeFlags),
	];

So what you end up with in the end is something like this:

["--output=/lib/rdjsonl.tmpl", "--glob=\"*.{md,txt}\""]

The problem with this is the glob pattern will actually include the quotes. And that obviously doesn't match anything.

The solution is either of these two (at least for me):

-vale_flags: --glob="*.{md,txt}"
+vale_flags: --glob=*.{md,txt}

-vale_flags: --glob="*.{md,txt}"
+vale_flags: --glob *.{md,txt}