[FEEDBACK] Unclear value on dependency-pinning and packing checks on C/C++ projects

[diogoteles08]

Is your feature request related to a problem? Please describe.
When inspected by scorecards, projects written in C get penalized for not having dependency-pinning tools or packings, but C does not have any widely adopted dependency manager, so there is not much they can do.

Describe the solution you'd like
Study the possibility of a smaller penalty for dependency-pinning tools or packing for C projects.

Describe alternatives you've considered

Additional context
This concern was raised while discussing Scorecards adoption with cURL project. You can check the whole discussion on this history of the mailing list, but the part related to this issue says:

We don't have depdencies like that so there is no such tool. You build curl to use the dependencies that are installed in the machine where curl is built/run. We cannot scan for those on GitHub.

[pnacht]

Would it perhaps be possible to use the GH API to fetch the project's main language, and then disable the check for any project where the main language isn't supported by dependabot or renovatebot? (worth noting renovatebot does have beta support for conan, a dependency manager for C).

Some filtering might be necessary in case there's a weird project with so many scripts that the primary language is "shell", but the actual "content" of the repo is in Python, for example.

Also might be relevant to detect other types of objects that dependabot can handle, such as git submodules and Docker/Terraform files. So a project written in C but that uses submodules should still be penalized.

[varunsh-coder]

In addition, might be good to check if project uses GitHub Actions, in which case updating Actions is still a good idea. e.g. I checked and cURL repo does have GitHub Actions.

[pnacht]

Very true! I think the summary here is: the check should only run in repos where it's relevant, and it might be quite valuable for the warning to indicate precisely how dependabot (or renovate) would be useful.

Currently the output for the Dependency-Update-Tool check is:

{
  "name":"Dependency-Update-Tool",
  "score":0,
  "reason":"no update tool detected",
  "details":[
    "Warn: Config file not detected in source location for dependabot, renovatebot, Sonatype Lift, or\n\t\t\tPyUp (Python). We recommend setting this configuration in code so it can be easily verified by others."
  ],
  "documentation":{
    "short":"Determines if the project uses a dependency update tool.",
    "url":"https://github.com/ossf/scorecard/.../docs/checks.md#dependency-update-tool"
  }
}

I think it'd be useful if the "details" could be:

{
  # ...
  "details":[
    "Warn: Config file not detected in source location for dependabot, renovatebot, Sonatype Lift, or\n\t\t\tPyUp (Python). We recommend setting this configuration in code so it can be easily verified by others.",
    "Warn: Project stores Python dependencies in ./requirements.txt. These dependencies can be updated by dependabot or renovatebot.",
    "Warn: Project uses GitHub workflows, which can be updated by dependabot or renovatebot.",
    "Warn: Project includes a Dockerfile, which can be updated by dependabot or renovatebot."
    # ...
  ],
  # ...
}

This would make the dependency update tool's value much clearer for projects that initially don't think the check applies to them.

Or, in projects where it truly doesn't make sense:

{
  "name":"Dependency-Update-Tool",
  "score":-1,
  "reason":"not applicable",
  "details":[
    "Info: Project written in C, which is supported by neither dependabot nor renovatebot.",
    "Info: Project does not use GitHub workflows.",
    "Info: Project does not include a Dockerfile.",
    # ...
  ],
  # ...
}

[diogoteles08]

Just found the issue #1726 very closely related to this one

[laurentsimon]

I like the idea too! We already have an internal APIs to fetch the languages of the repo using the language API from GitHub.

[github-actions]

Stale issue message - this issue will be closed in 7 days

[github-actions]

This issue is stale because it has been open for 60 days with no activity.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.