On SAST check, evaluate is the project's language is supported by the SAST tools

[diogoteles08]

Is your feature request related to a problem? Please describe.
The project is penalized on SAST check, but the project's language is not supported by any of the SAST tools that Scorecard consider.

Describe the solution you'd like
When evaluating the SAST check, first analyze if the project language is supported by any of the the SAST tools Scorecard consider.

Describe alternatives you've considered
Increase the number of SAST tools considered by Scorecard, but that would not take care of all scenarios.

Additional context
I went through this issue while talking to rust-lang/rust about Scorecard adoption, you can check the discussion here.

This issue is indirectly related to #1726 and #2483

[laurentsimon]

Please feel free to propose the SAST tools we should support, and which language they support. Not sure how long it would take to compile this list, but it would be very valuable. #1487 is also relevant.

[miguelnietoa]

I agree with @diogoteles08.

The same happens with Elixir, that neither CodeQL, SonarCloud nor LGTM support this programming language.

However, in the CI/CD of our Elixir projects in kommitters we use Dialyzer and Credo, but I am not sure if these tools can be fully considered as a SAST tool.

This is a common CI workflow: https://github.com/kommitters/kadena.ex/blob/010f18d213d45c47ca24102471918f4614a7f127/.github/workflows/ci.yml#L53-L60

[diogoteles08]

Another language in this scenario is Dart. Codeql does not support Dart and sonarcloud is still on plans to support it.

For this language, another path would be to support dart-analyze tool in the SAST check. @laurentsimon do you think it makes sense?

\cc @godofredoc

[godofredoc]

Another example is clang-tidy flutter/engine makes extensive use of it in all the supported platforms.

e.g. https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8785261744812716625/+/u/test:_lint_host_debug/stdout

[laurentsimon]

We're very very very interested in supporting more SAST tools, so +1 from me. Overtime we can clean up the code and improve the SAST check to be more ecosystem-aware if need be; and also to capture differences between the "type" of SAST (a linter like clang-tidy != static analysis tool). (Maybe we need a different check for linters altogether)

For CLI tools we need to parse workflow commands. That's also do-able but will need a bit more work.

Feel free to send PRs for SAST and cc me on it

[godofredoc]

\cc @zanderso

[godofredoc]

@laurentsimon even though clang tidy advertises itself as a linter tool it includes multiple memory validations, cast validations, pointer analysis, etc usually included in SAST tools (many of these are checked by CodeQL).

Full list of checks: https://clang.llvm.org/extra/clang-tidy/checks/list.html

[laurentsimon]

Thanks for the info. +1 on what you said. It uses only the AST iirc, but you're right it does a bit more than a "simple" linter.

[github-actions]

Stale issue message - this issue will be closed in 7 days

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[ia0]

What about languages that already perform static analysis with strong guarantees like Rust, OCaml, Haskell? In those cases, a program simply doesn't compile if it has one of the classes of bug that the type system prevents. Those languages don't need an additional SAST tool because they already come with one. And that tool is actually part of the compiler, such that you don't need to run it in addition to the compiler. In particular, a program doesn't compile if the tool rejects the program. So I would extend the wording in the initial post of this issue:

Describe the solution you'd like When evaluating the SAST check, first analyze if the project language is supported by any of the the SAST tools Scorecard consider.

Or if the project language has a SAST tool built in the compiler and thus doesn't need an additional one.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.