GH CI: Add Provenance Support (Security Enhancement)

[elliot-huffman]

Feature Request

Description

This allows you to publicly establish where a package was built and who published a package, which can increase supply-chain security for your packages.

This is a super easy publish command change.
Only a NPM parameter needs to be added to the existing publish pipeline. (by default)

Overview

Publish this package using only GitHub actions and enable Provenance to enable the ability cryptographically to attest that the package hasn't been tampered with during build, publish, and transport.

This can be done via npm audit signatures automatically.

NPM Docs:
https://docs.npmjs.com/generating-provenance-statements

Examples

Before: npm publish

After: npm publish --provenance

Consuming attestation signatures:
https://docs.npmjs.com/generating-provenance-statements#verifying-provenance-attestations

[IamLizu]

Interesting 👍

I just learned a new thing thanks to you. Here's a GitHub blog post in case anyone else is also new to this, https://github.blog/security/supply-chain-security/introducing-npm-package-provenance/

[UlisesGascon]

yep! This is a very good idea @elliot-huffman. We planned to do it only once we move the publication to GitHub Actions as it is one of the requirements

To publish a package with provenance, you must build your package with a supported cloud CI/CD provider using a cloud-hosted runner. Today this includes GitHub Actions and GitLab CI/CD. https://docs.npmjs.com/generating-provenance-statements#provenance-limitations

I will move this to discussions and change the title as we might want to do it for all the packages.

[elliot-huffman]

I am cool with doing a PR to make a NPM publish GH action.

This is what I would base my PR off of:
https://github.com/Software-Hardware-Integration-Lab/ObjectID-Converter/blob/main/.github/workflows/NPM-Publish.yml

[ljharb]

What would be the benefit?

All npm provenance guarantees you is the publish was done from a specific workflow file, which is of virtually zero value (npm has cryptographically verified that the tarball isn't tampered for over a decade). Additionally, it requires using a single factor which makes it objectively less secure than publishing locally with 2FA/MFA.

[UlisesGascon]

I am cool with doing a PR to make a NPM publish GH action.

We need to solve a 2FA in general (not just for provenance) first, so makes sense to wait a bit until we had a decision made on this.

[elliot-huffman]

What would be the benefit?

All npm provenance guarantees you is the publish was done from a specific workflow file, which is of virtually zero value (npm has cryptographically verified that the tarball isn't tampered for over a decade). Additionally, it requires using a single factor which makes it objectively less secure than publishing locally with 2FA/MFA.

I personally believe that a cryptographic attestation of package/publish health is very impactful for supply chain security. Without that crypto attestation, how would you be able to quickly tell if there was tamper on the package build on a local system?
You could not do it quickly (you would have to review all code).

The benefit would be that there is a crypto attestation that it isn't being done with a local dev/custom dev system.

Using custom dev/local dev systems to publish means that all of the junk that is also on that system, such as viruses/threat actors could also inject their own junk into the build process.

GH Actions is hypervisor isolated and a blank environment on execution which eliminates all but the most complex host-based attacks from the publish process.

Provenance is not single factor, it is MFA. A refresher on what MFA is:

• Somewhere you are
• Something you have
• Something you know
• Something you are

GH Actions is somewhere you are (GH Actions execution environment), something you know (the NPM key) and something you have (PKI on the OAuth key)

This effectively increases the security to 3 factors on publish.

It would also improve transparency for the build process to allow the community to audit the build process end to end. Right now, we can't see where the packages are being generated or how they are being generated.

[elliot-huffman]

I am cool with doing a PR to make a NPM publish GH action.

We need to solve a 2FA in general (not just for provenance) first, so makes sense to wait a bit until we had a decision made on this.

What MFA issues are you experiencing right now? Since this is a popular package, doesn't it force you to use MFA?

[ljharb]

What good is an attestation if you already trust the publisher? (or don't trust)

Provenance is single-factor because to publish to npm from CI, you need a publish token, which is a single factor from npm's perspective, which is the only perspective that matters.

[elliot-huffman]

What good is an attestation if you already trust the publisher? (or don't trust)

Provenance is single-factor because to publish to npm from CI, you need a publish token, which is a single factor from npm's perspective, which is the only perspective that matters.

Did you not read the NPM JS docs and GitHub docs?
It clearly isn't single factor.
I think you are confusing Authorization with Authentication.

Tokens are issued after MFA. An access token is a cryptographic attestation (not unlike provenance) that the principal has authenticated, and the token also has authorization data.
The GH Action runner has to MFA to get the token and get provenance.

Please review this diagram from GitHub:
https://github.blog/wp-content/uploads/2023/04/Diagram.png

Also please read the link that @IamLizu posted for the GH blog.
Also please read the NPM JS docs.
Also please review access token architecture here: https://learn.microsoft.com/en-us/entra/identity-platform/access-tokens (GH implements the same architecture as Entra, Entra has more docs explaining in more details what happens with tokens and how to use them).

Please also review the principal of assume breach here:
https://www.microsoft.com/en-us/security/blog/2021/01/19/using-zero-trust-principles-to-protect-against-sophisticated-attacks-like-solorigate

Assume breach
Our final principle is to Assume Breach, building our processes and systems assuming that a breach has already happened or soon will. This means using redundant security mechanisms, collecting system telemetry, using it to detect anomalies, and wherever possible, connecting that insight to automation to allow you to prevent, respond and remediate in near-real-time.

Sophisticated analysis of anomalies in customer environments was key to detecting this complex attack. Customers that used rich cloud analytics and automation capabilities, such as those provided in Microsoft 365 Defender, were able to rapidly assess attacker behavior and begin their eviction and remediation procedures.

Importantly, organizations such as Microsoft who do not model “security through obscurity” but instead model as though the attacker is already observing them are able to have more confidence that mitigations are already in place because threat models assume attacker intrusions.

More telemetry to detect breach is necessary to catch more sophisticated attacks.

[ljharb]

The only thing that npm requires to actually do the publish from CI, with or without provenance, is that single token. That it was generated by an authenticated actor doesn't make it suddenly multi-factor.

You're right that the action is doing stuff to generate the provenance data written to sigstore, but that's irrelevant - only the publish matters here.

[ljharb]

Additionally, can you point me to a single attack in the npm ecosystem that provenance would have prevented? Nobody at Microsoft/Github, Google, or the OpenSSF has ever been able to provide even a single example.

[elliot-huffman]

Additionally, can you point me to a single attack in the npm ecosystem that provenance would have prevented? Nobody at Microsoft/Github, Google, or the OpenSSF has ever been able to provide even a single example.

UA-Parser-JS (October 22 2021). A threat actor was able to breach the account and upload non-attested versions. The attestation process would have not been successful due to custom builds for that and would have been detected.

The attestation forces transparency and would have immediately been axed by the community.

[ljharb]

How would it have been detected? I can make a workflow that takes a URL as input and downloads a tarball and publishes that, and get full provenance, and then delete all the action logs (which expire in 3 months anyways) - and the malicious actor in question could have done the same on ua-parser-js.

[elliot-huffman]

The only thing that npm requires to actually do the publish from CI, with or without provenance, is that single token. That it was generated by an authenticated actor doesn't make it suddenly multi-factor.

You're right that the action is doing stuff to generate the provenance data written to sigstore, but that's irrelevant - only the publish matters here.

If the token is stored in a GH Secret, the process is one way and can only be accessed from a GH Actions Runner (using the same MFA methods as above).
The GH Secret is MFA and that means as long as a malicious admin doesn't store the secret elsewhere, the only way to get that token is MFA. Essentially encapsulating the SFA to MFA by making MFA mandatory to retrieve the token.

[ljharb]

That's just not how it works. Anyone with write access to the repo can retrieve the secret in a way that can't be traced later.

No matter what, publishing to npm with an automation token is single-factor.

[elliot-huffman]

How would it have been detected? I can make a workflow that takes a URL as input and downloads a tarball and publishes that, and get full provenance, and then delete all the action logs (which expire in 3 months anyways) - and the malicious actor in question could have done the same on ua-parser-js.

You are adding extra steps into the example.
The example I gave 100% could have been detected by provenance. Provenance by itself is not a complete solution, it is a partial solution.
Provenance would force threat actors to make their attack process more complicated and more detectable.
There is no one stop solution for security, good security requires multiple overlapping security controls. Provenance helps with providing a clean baseline, it is not a replacement for keeping your account secure. You need both.

If we are going to start modifying the example I gave then:
Threat actor then modifies the repo code to publish via provenance.

To mitigate this additional step the TA takes in your fictional scenario, ideally you would also implement a Privileged Access Workstation for approving PRs and starting the publish/deploy process. All code dev is rendered untrusted with that model and PAWs bring trust to the merge process and publish process by micro-segmentation.

This would make it almost impossible for TAs to slip code through the PR process.

[ljharb]

This is all security theater imo - the only true mechanism for verifying and detecting this kind of attack is one that requires zero maintainer effort whatsoever: post-publish heuristics-based comparison of tarball and repo contents.

[elliot-huffman]

That's just not how it works. Anyone with write access to the repo can retrieve the secret in a way that can't be traced later.

No matter what, publishing to npm with an automation token is single-factor.

Unless you isolate it to a GH Environment requiring approval and only specific access on your specified tags/branches. this prevents everyone from accessing the GH Secret.

It is actually part of the security best practices GH has for secrets:
https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions

[elliot-huffman]

This is all security theater imo - the only true mechanism for verifying and detecting this kind of attack is one that requires zero maintainer effort whatsoever: post-publish heuristics-based comparison of tarball and repo contents.

It isn't security theater, I used to assist on an IR team at Microsoft and I have 100% seen attacks like this and even more complex. These attacks are not theoretical, they are real. That is why MSFT is mandating GH to improve security capabilities. These technologies were created through blood sweat and customer breaches.

No matter what, publishing to npm with an automation token is single-factor.

Please see above. If you can create a threat model demonstrating how this is possible as single factor using the architecture I provided, I will concede.

[ljharb]

I mean, I can add a dependency that does whatever malicious thing I'd have wanted the main package to do merely by having write access, no matter how else you've locked things down - so what is provenance preventing, exactly?

[elliot-huffman]

I mean, I can add a dependency that does whatever malicious thing I'd have wanted the main package to do merely by having write access, no matter how else you've locked things down - so what is provenance preventing, exactly?

It is preventing local dev machines from being publishing sources and cryptographically attesting that the build environment is what you see is what you get. Nothing gets hidden.

Dev machines like yours or mine can't be considered clean and have to be assumed as breached. Provenance attests that the environment is clean and NOT a dev machine. It also makes the publish/build pipeline auditable.

If a threat actor puts malware into the repo itself, provenance doesn't prevent that, and it will happily sign it. It wasn't built to prevent that.
Good repo permissions/environment isolation/code review/SAST/DAST prevent that type of attack.

[ljharb]

Github Actions has had far more breaches (nonzero) than my local machine has (zero) :-) I think the "have to be assumed as breached" part is a pretty large flaw in your thinking here (or at least, only applying that thinking to dev machines instead of every machine)

[elliot-huffman]

Github Actions has had far more breaches (nonzero) than my local machine has (zero) :-) I think the "have to be assumed as breached" part is a pretty large flaw in your thinking here (or at least, only applying that thinking to dev machines instead of every machine)

Correction GH Actions was not breached. It was customer environments that were breached due to secrets leaking and other misconfigurations.

If you leave your door wide open, and a robber robs you. You don't sue the door company.

I do apply this thinking to all computers (see PAW). That is outside the scope of this thread though