Pinned_Dependencies check stops on faulty shell files when it is meant to continue

[lehors]

Describe the bug
When the shell parser returns an error the check stops when it should continue to the next file.

Reproduction steps
Steps to reproduce the behavior:

./scorecard --repo=https://github.com/openssl/openssl --checks=Pinned-Dependencies

RESULTS
-------
Aggregate score: ?

Check scores:
|-------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|
| SCORE |        NAME         |             REASON             |                                             DOCUMENTATION/REMEDIATION                                              |
|-------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|
| ?     | Pinned-Dependencies | error parsing shell code:      | https://github.com/ossf/scorecard/blob/71e8698617d25b006cf0935da5e1fb5487bff292/docs/checks.md#pinned-dependencies |
|       |                     | 5:19: a command can only       |                                                                                                                    |
|       |                     | contain words and redirects;   |                                                                                                                    |
|       |                     | encountered (                  |                                                                                                                    |
|-------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|

Expected behavior
Rather than quit on the first faulty shell file the check should continue, process the rest of the files, and still return some score.

Additional context

A simple fix (PR upcoming) gives the following more useful result:

RESULTS
-------
Aggregate score: 6.0 / 10

Check scores:
|--------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|
| SCORE  |        NAME         |             REASON             |                                             DOCUMENTATION/REMEDIATION                                              |
|--------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|
| 6 / 10 | Pinned-Dependencies | dependency not pinned by hash  | https://github.com/ossf/scorecard/blob/71e8698617d25b006cf0935da5e1fb5487bff292/docs/checks.md#pinned-dependencies |
|        |                     | detected -- score normalized   |                                                                                                                    |
|        |                     | to 6                           |                                                                                                                    |
|--------|---------------------|--------------------------------|--------------------------------------------------------------------------------------------------------------------|

[azeemshaikh38]

Hi @lehors, thanks for starting this issue. I was about to respond to your email on ossf-scorecard-dev@ group, but glad that you started this issue as this is easier to track.

Yes, this is a known issue but its not on our roadmap right now due to resource constraints. Happy to discuss possible solutions and/or review PRs for it. @chrismcgehee who is the most familiar with this code.

[evverx]

I ran into this issue and I think the check should keep going as well.

PR upcoming

@lehors are you planning to add something like #1297 to that PR? If so, I'll go ahead and close my PR

[lehors]

The patch to get the check to continue is actually trivial. It took me more time to write the test for it. :-)
I haven't included your fix @evverx so keep your PR open. I agree that's a useful change too!

[lehors]

This was addressed by PR#1312.