Add support for NuGet

[JonDouglas]

We've invested time into understanding this space. It seems that for NuGet to thrive with scorecards, we will be heavily dependent on #4177 for many Microsoft packages as they haven't quite moved to GH actions yet. This work however, will work just fine for other prominent packages that are already on GH actions and follow best practices outlined.

Additionally, it seems that no other package manager has implemented the basic pinned dependencies check for ensuring "pinned versions" are included.

Based on the recent SonaType survey, the following are emphasized across various ecosystems:

Therefore, this issue should track the following support (or lack thereof) of the most impactful/implemented today:

• Pinned Dependencies supporting , packages.config, and (CPM). Today this check only does Docker and GH Actions Workflows .yml AFAIK.
• License (May need to check license expression from package too, not just repo)
• Security Policy (Similar to License, SECURITY.md may become a popular paradigm in packages soon)
• Token Permissions
• Vulnerabilities (May need to use NuGet's existing APIs for this per package version as it pulls up all repository vulns, not package specific-enough in case of monorepos)
• Dangerous Workflow
• Binary Artifacts (This check seems to flag any .dll on repos, not really accurate for .NET)

While some of these are repository & CI/CD specific, in the context of a package i.e. scorecard --nuget=System.Text.Json --show-details we have a lot of work to do.

OLD:

The scorecard project currently only supports npm, golang, and pip as far as I could tell. I'm a PM on the NuGet team at Microsoft and would love to help contribute adding support for NuGet in this tool or providing the right guidance to implement support for NuGet. This closely aligns with a proposal I had last year and would love to experiment with this scorecard in .NET:

dotnet/designs#216

Please feel free to reach out to us over at NuGet/Home on GitHub or in this issue. Any steps on how to best contribute adding this support would be greatly appreciated!

[naveensrinivasan]

It would be great to add support for NuGet.

Here is an example of how we are looking for go

scorecard/checks/shell_download_validate.go

Line 591 in 696553b

if isGoUnpinnedDownload(c) {

PR's are welcome!

[laurentsimon]

@JonDouglas that's great, thanks for reaching out!

@naveensrinivasan pointed you to the file where we parse commands and look for unpinned go, pip and npm commands.
This is the place to add it.

Shameless plug: as part of the OSSF Best Practice WG, we have a work stream on package manager's best security practices. We've started with npm and pip. If you're interested in collaborating - or even lead! - a similar effort for NuGet, please let me know.

An idea we've been kicking around is to work with package managers and help them display scorecard information on their website. Would you imagine providing scorecard results on the NuGet's hub//website? I'd love to discuss this direction if it's of interest. You're more than welcome to attend our bi-weekly Thu scorecard meetings!

[laurentsimon]

an additional place to add support for NuGet is in the Packaging check https://github.com/ossf/scorecard/blob/main/checks/packaging.go, which tries to infer if the project published a package. We typically look for known GitHub actions and commands to detect this.

[JonDouglas]

@laurentsimon Let's discuss this further as it's very in-line with some areas I'm investigating and would love to join a scorecard meeting. Where can I get involved for those bi-weekly meetings?

[azeemshaikh38]

Please see https://github.com/ossf/scorecard#connect-with-the-scorecards-community for details about joining the bi-weekly. Bi-weekly's happen Thursdays 1-2pm PST (next instance is on Feb 24th).

[azeemshaikh38]

Also feel free to add an agenda item to the upcoming meet in this doc. You'll need to join [email protected] for being able to modify the doc.

[laurentsimon]

Looking forward to meeting you @JonDouglas!
Thanks @azeemshaikh38 for the prompt info!

[JonDouglas]

Hi all,

I had originally planned to join the meeting later this afternoon, but unfortunately I have a conflict I cannot clear this month. I'll be looking to join the next one on the 10th instead.

[laurentsimon]

Thanks for letting us know. See you in 2 weeks then!

[balteravishay]

@laurentsimon I'm trying to plan the work needed to comply with your reply here:

an additional place to add support for NuGet is in the Packaging check https://github.com/ossf/scorecard/blob/main/checks/packaging.go, which tries to infer if the project published a package. We typically look for known GitHub actions and commands to detect this.

Looking at the packaging code and the IsPackagingWorkflow method it seems as if some support for Nuget is already accounted for when calculating the packaging score.
Do you reckon some more development needs to be added or is that comment stale?
cc. @JonDouglas

[laurentsimon]

Looking at the packaging code and the IsPackagingWorkflow method it seems as if some support for Nuget is already accounted for when calculating the packaging score. Do you reckon some more development needs to be added or is that comment stale? cc. @JonDouglas

Great, it seems it's already supported. No further comment from me on that front then

[walbourn]

Note in my case there's no GitHub action that is going to indicate my project has an associated NuGet package on nuget.org or a VC++ Package on the vcpkg.io.

I do have links in my Release comments that have those links though, so maybe you can scan those as well?

[balteravishay]

thanks for the feedback @walbourn!

Packaging scores in Scorecard, today, work by detecting github/gitlab packaging workflows only, for all ecosystems (correct me if I'm wrong here @laurentsimon). The aim of this check is to detect if the repo has an automatic, standardized packaging and delivery mechanism, rather than to check if "indicate my project has an associated NuGet package on nuget.org".

please review the test case here, noting how a GH workflows that match a run statement with "nuget push" commands is deemed as a packaging workflow. that also complies with the GitHub docs on pushing to nuget, here and with the nuget docs on pushing, here

Perhaps the scenario you're describinh can be better handled as its own packaging enhancement request, aiming all ecosystems, rather than under this issue that is specific to nuget?

Also, I suggest to separate dotnet/nuget from VC++ (both here and in your original issue #2535), as they would likely have rather different implementations, contribution paths and contributors.

[walbourn]

One option would be to use a file in the repo like is done with .onefuzz detection for Fuzzing. Maybe a .nugetorg file that just has text lines for the URL(s) to associated nuget packages?

The scorecard could then walk those URLs to see if the packages are signed.

You can also do validation where you extract the NuGet package metadata and verify that the <repository type="git" url="???" /> points back to the repo you are scoring to make sure it's consistent.

[github-actions]

This issue has been marked stale because it has been open for 60 days with no activity.

[JonDouglas]

Updated this issue with various comments and rationale after doing a small hackathon on this topic. We're investigating incremental improvements such as lock files in #4351 today and bringing more awareness to Microsoft/.NET teams on various other checks.