How to configure policy.yml for a run?

[rajbos]

How can we configure a policy.yml file for a run?
I see there is a template file available, but I do not see it being used. Seems like the container would use it, but we ware not passing it into the container params (I think).

I am trying to run the action internally on GHES, and our runners do not have internet access, so downloading the extra clients is failing:

Error: GetClients: getting OSS-Fuzz repo client: error during InitRepo: repo unreachable: Get "https://api.github.com/repos/google/oss-fuzz": internal error: innerTransport.RoundTrip: internal error: innerTransport.RoundTrip: error in HTTP: dial tcp 140.82.121.6:443: i/o timeout
2023/03/03 15:17:48 error during command execution: GetClients: getting OSS-Fuzz repo client: error during InitRepo: repo unreachable: Get "https://api.github.com/repos/google/oss-fuzz": internal error: innerTransport.RoundTrip: internal error: innerTransport.RoundTrip: error in HTTP: dial tcp w.x.y.z:443: i/o timeout

I'd expect that I can configure the policy with fuzzing disabled and get a step further with testing:

 Fuzzing:
      score: 10
      mode: disabled

[azeemshaikh38]

@laurentsimon may have more insight on the policy. But without internet access I'd expect Scorecard to fail further too. We rely on making GitHub API calls to get the data we need.

[rajbos]

Since it uses the correct environment variable, I was expecting it to make the calls to the correct url 😄
See here:

scorecard-action/options/options.go

Line 69 in c59c116

GithubAPIURL string `env:"GITHUB_API_URL"`

[laurentsimon]

@azeemshaikh38 is correct we need internet access. I'm not that we use GithubAPIURL internally. @spencerschrock @azeemshaikh38 do you know?

re: policy. For the policy to work, you need to use --format sarif and --policy path/to/policy.yml. I think you also need to set ENABLE_SARIF=1 or it will complain.

[azeemshaikh38]

1. scorecard-action/options/options.go

   Line 69 in c59c116

   GithubAPIURL string `env:"GITHUB_API_URL"`

   was meant to help us support GH Enterprise clients. But it hasn't been fully implemented yet, see Support private git servers (like Github Enterprise instances) scorecard#660. @raghavkaul fyi who is working on getting GHE support end-to-end.
2. @laurentsimon do we support inputting a policy file to scorecard-action? IIUC, @rajbos is trying to use scorecard-action on a GHE along with privately hosted Scorecard API.

[laurentsimon]

The GHA does not support the policy as input. It's hardcoded for flexibility on our side, but it's not exposed.

[azeemshaikh38]

@rajbos looks like we do not currently support the usecase you're trying and I don't see us working on supporting this in the short-term. So closing this issue.

[rajbos]

Well, if @raghavkaul is working on an end-to-end setup for GHES instances, I'd like to join that effort. Any links to the work that is being done? Otherwise I'll start one myself by running the container and exposing those inputs.

Would you like/appreciate a PR for that in the future?

[raghavkaul]

@rajbos PRs welcome; there's a change to add GitLab to the RepoClient (ossf/scorecard#2655) which may be a pattern to follow for GHE support. I have some WIP in a branch I can share if that helps, too.

[rajbos]

I'd love to. I have the injection of the policy file into the container running, but are now waiting on an upstream fix to ossf/scorecard before I can continue.

[gabrielkoo]

For those who would like to customize this, I implemented this workaround with an extra Action step:

    - name: Filter SARIF to skip irrelevant rules
        env:
          SCORECARD_SKIPPED_RULE_IDS: "${{ vars.SCORECARD_SKIPPED_RULE_IDS }}"
        run: |
          SCORECARD_SKIPPED_RULE_IDS_IN_JSON=$(echo $SCORECARD_SKIPPED_RULE_IDS | jq -cR 'split(",")')
          # Trim the SARIF file to remove skipped rule detections
          cat results.sarif | jq '.runs[].results |= map(select(.ruleId as $id | '$SCORECARD_SKIPPED_RULE_IDS_IN_JSON' | all($id != .)))' > filteredResults.sarif
          # Print the skipped rule detections
          cat results.sarif | jq '.runs[].results | map(select(.ruleId as $id | '$SCORECARD_SKIPPED_RULE_IDS_IN_JSON' | any($id == .))) | select(. | length > 0)'

Basically, you have to put a comma separated list into the GitHub Action variable SCORECARD_SKIPPED_RULE_IDS.

All checks would still be run, but the jq script would filter out the detections that are false positive/irrelevant to your repositories.

As the exclusions are no longer in the SARIF file, GitHub won’t show the excluded scorecard detection results in the “Code Scanning” repository, as well as not in the “Security” tab.

Lastly, it still prints out the skipped rule detections in case you want to look at.