python2Packages.certifi: init at 2019.11.28 #127453

dotlambda · 2021-06-19T09:13:28Z

Motivation for this change

Things done

cc @tollb @NixOS/nixops-committers

dotlambda · 2021-06-19T09:18:26Z

To be honest, I don't like providing an outdated version of certifi for Python 2 because we shouldn't ship outdated security-related stuff. Maybe it's better not to fix this and mark NixOps 1.7 as broken.
Also, I will unsubscribe from this pull request. Any necessary modifications can be force-pushed to my branch by @NixOS/nixops-committers who should also merge this pull request if they think it's the best solution or create a new one otherwise.
This has been the very last time I put any work into fixing some Python 2 package required by NixOps!

r-rmcgibbo · 2021-06-19T12:21:28Z

Result of nixpkgs-review pr 127453 at 304839a9 run on x86_64-linux 1

11 packages marked as broken and skipped:

gnuradio3_7
gnuradio3_7Packages.ais
gnuradio3_7Packages.gnuradio
gnuradio3_7Packages.gsm
gnuradio3_7Packages.limesdr
gnuradio3_7Packages.nacl
gnuradio3_7Packages.osmosdr
gnuradio3_7Packages.rds
natron
ocropus
rainbowstream

3 packages failed to build:

12 packages built successfully:

bitbucket-cli
buttersink
cloudmonkey
datadog-agent
euca2ools
fmbt
git-fast-export
mercurial_4
mx
nixops
tsung
zabbix-cli

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

mweinelt · 2021-06-19T13:21:44Z

This poses a certain security risk, so I decided to set knownVulnerabilities.

Ekleog · 2021-06-20T19:08:22Z

pkgs/development/python-modules/certifi/2019.11.28.nix

+    license = licenses.isc;
+    maintainers = with maintainers; [ ]; # NixOps team
+    knownVulnerabilities = [
+      "Python 2 supported for certifi ended in 2019 and shipping an outdated certificate store carries a non-negligble risk"


Can anyone describe an actual threat model based on “shipping an outdated certificate store”? I can't think of any that would actually be a problem for normal users, and non-normal users already have much worse to worry about.

IMO adding knownVulnerabilities when there are no actually known vulnerabilities only makes people more used to working around it, especially for a package as advertised as nixops, and so is a net negative for the security of NixOS users.

Now, if there were some specific known vulnerability (eg. “this CA that was there in 2019 was removed after having issued malicious certificates that can still be used”), I would totally understand this item; but IMO as currently described this is lacking motivation and it'd be better to have no knownVulnerabilities than this one.

easy. Some CA get's their private key stolen and now the thief can issue new certificates. Alternatively (way more common) the CA is seen to be not trustworthy due to weird behavior or bogus certificates issued by it. CAs that are not trustworthy or compromised might issue certificates that can be used to stage successful man-in-the-middle attacks. This impacts all tls secured communication that uses a certificate store that includes this CA (passwords/downloads/...). Unless you have support for OCSP, revocation is done via two ways:

certificate revocation lists (that are sparely used). In this case the issuing CA revokes the compromised CA and publishes a file that includes this information.

removing the CA from certificate stores like this one. The only way to handle compromised self-signed CA.
Hence keeping the certificate store up to date is important.

For reference: https://sslmate.com/certspotter/failures

Hm, what about re-using the bundle from the cacert package instead? This would not only solve this issue but also would consolidate the CA bundle to a single location. For example when writing integration tests involving Python software, one usually needs to not only override cacert but also python*Packages.certifi to inject testing certificates.

The approach suggested by @aszlig may also be applicable to the boto 2.49.0 package, as I believe it has an embedded cacert file that is even further out-of-date: https://raw.githubusercontent.com/boto/boto/2.49.0/boto/cacerts/cacerts.txt. I'm not certain when this embedded cacerts.txt is used by boto versus the one provided by python2Packages.certifi.

If someone can confirm that boto's embedded cacerts.txt is actually used (especially through nixops 1.7), we might want to add a knownVulnerabilities to the boto package as well -- for consistency with python2Packages.certifi. Or, provide updated certificates in both packages. I took a quick look at arch and debian to see if they had updated the certs in their boto 2.49.0 packages, and it doesn't seem that they have. So, maybe I am mistaken.

@zeri42 Thank you, your link confirms that as of today there is no known instance of a CA having an issue that would warrant the knownVulnerabilities toggle since the version of certifi that is being considered here, and so that reverting updates should not be a security vulnerability :) (though it is only one source, of course)

(And also, even if they were in most cases I'd keep my position: an attacker stealing the private key from a CA is not a reasonable threat model for a normal NixOS user, which is the reason why I wrote my first message)

This being said I'm going to step out of this discussion: I'm only focussed on “knownVulnerabilities here would be bad”; there are for sure much better things that could be done by improving stuff in various ways :)

nlewo · 2021-06-21T07:20:54Z

@dotlambda Note this would also fixes the python2Packages.requests library which depends on certifi.

dpausp · 2021-06-24T09:16:09Z

We could also use a recent version of certifi and disable the test for the contents() function. The function is broken but it didn't even exist on 2019.11.28. This wouldn't break existing Python 2.7 code where the new function has never worked. Ugly, but having newer certificates is better, I think.

tomfitzhenry · 2021-06-25T23:33:14Z

This will also fix buttersink failing to build: #128151.

prusnak · 2021-06-29T12:12:02Z

This will also fix gnuradio 3.7 shipped with nixos-21.05: #128620

It's not clear when the `certifi` PR[1] will be merged, and I don't want to have to wait for a new hydra build because of it. `nixopsUnstable` is unusable in its current from due to almost lacking documentation, and so I have little choice but to remove `nixops` right now until `certifi` is merged. [1]:NixOS/nixpkgs#127453

adisbladis · 2021-07-02T08:32:44Z

@dotlambda I took the liberty of reworking this PR a bit.
We now use 2019.11.28 for building the package but inherit the cert bundle from the maintained python3-only version.

I think this is a decent solution with minimal maintenance.

prusnak · 2021-07-02T08:37:02Z

@adisbladis I wonder whether the package version of this hybrid should be 2019.11.28 (source code version as reported by __init__.py) or 2021.05.30 (the version of cacert.pem)

dotlambda · 2021-07-02T09:03:38Z

The version should be 2019.11.28 because the API has since been changed.

pkgs/development/python-modules/certifi/python2.nix

adisbladis · 2021-07-02T09:32:29Z

Alright, version set as 2019.11.28.

We use the sources from 2019.11.28 to build, but inherit the cert bundle from the maintained Python3-only version of certifi. Co-authored-by: adisbladis <[email protected]>

clefru · 2021-07-03T18:54:59Z

clefru@57e65ad has a commit that patches Python 2 support back into a recent certifi. Works for me so far (with nixops)

adisbladis · 2021-07-05T21:22:03Z

@clefru I thought about adopting a similar approach but opted against it because of the added maintenance burden.

I think for now I'll just merge this and if you think your approach is sufficiently better could you please create a separate PR?

github-actions · 2021-07-05T21:26:12Z

Successfully created backport PR #129374 for release-21.05.

dotlambda requested review from FRidh and jonringer as code owners June 19, 2021 09:13

dotlambda removed request for FRidh and jonringer June 19, 2021 09:13

github-actions bot added the 6.topic: python label Jun 19, 2021

dotlambda added the backport release-21.05 label Jun 19, 2021

ofborg bot added 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 labels Jun 19, 2021

dotlambda added the 9.needs: maintainer Package or module needs active maintainers label Jun 19, 2021

mweinelt added the 1.severity: security Issues which raise a security issue, or PRs that fix one label Jun 19, 2021

dotlambda mentioned this pull request Jun 20, 2021

pythonPackages.certifi: does not support python2 #127561

Merged

11 tasks

Ekleog requested changes Jun 20, 2021

View reviewed changes

dotlambda mentioned this pull request Jun 24, 2021

[21.05] pythonPackages.certifi: does not support python2 #127563

Merged

11 tasks

tomfitzhenry mentioned this pull request Jun 25, 2021

buttersink recently fails to build on nixos-21.05 #128151

Closed

prusnak mentioned this pull request Jun 29, 2021

Situation of gnuradio 3.7 #128620

Closed

adisbladis force-pushed the certifi-python2 branch from f81cc2b to 0af5adc Compare July 2, 2021 08:30

prusnak requested changes Jul 2, 2021

View reviewed changes

pkgs/development/python-modules/certifi/python2.nix Outdated Show resolved Hide resolved

adisbladis force-pushed the certifi-python2 branch from 0af5adc to 0b5fe23 Compare July 2, 2021 09:31

python2Packages.certifi: init at 2019.11.28

ada27b8

We use the sources from 2019.11.28 to build, but inherit the cert bundle from the maintained Python3-only version of certifi. Co-authored-by: adisbladis <[email protected]>

dotlambda force-pushed the certifi-python2 branch from 0b5fe23 to ada27b8 Compare July 3, 2021 12:50

ofborg bot added the 8.has: package (new) This PR adds a new package label Jul 3, 2021

adisbladis merged commit e75cc16 into NixOS:master Jul 5, 2021

github-actions bot mentioned this pull request Jul 5, 2021

[Backport release-21.05] python2Packages.certifi: init at 2019.11.28 #129374

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python2Packages.certifi: init at 2019.11.28 #127453

python2Packages.certifi: init at 2019.11.28 #127453

dotlambda commented Jun 19, 2021

dotlambda commented Jun 19, 2021

r-rmcgibbo commented Jun 19, 2021

mweinelt commented Jun 19, 2021

Ekleog Jun 20, 2021

zeri42 Jun 22, 2021 •

edited

Loading

aszlig Jun 23, 2021

tollb Jun 23, 2021

Ekleog Jun 24, 2021

nlewo commented Jun 21, 2021

dpausp commented Jun 24, 2021

tomfitzhenry commented Jun 25, 2021

prusnak commented Jun 29, 2021

adisbladis commented Jul 2, 2021

prusnak commented Jul 2, 2021

dotlambda commented Jul 2, 2021

adisbladis commented Jul 2, 2021

clefru commented Jul 3, 2021 •

edited

Loading

adisbladis commented Jul 5, 2021

github-actions bot commented Jul 5, 2021

python2Packages.certifi: init at 2019.11.28 #127453

python2Packages.certifi: init at 2019.11.28 #127453

Conversation

dotlambda commented Jun 19, 2021

Motivation for this change

Things done

dotlambda commented Jun 19, 2021

r-rmcgibbo commented Jun 19, 2021

mweinelt commented Jun 19, 2021

Ekleog Jun 20, 2021

Choose a reason for hiding this comment

zeri42 Jun 22, 2021 • edited Loading

Choose a reason for hiding this comment

aszlig Jun 23, 2021

Choose a reason for hiding this comment

tollb Jun 23, 2021

Choose a reason for hiding this comment

Ekleog Jun 24, 2021

Choose a reason for hiding this comment

nlewo commented Jun 21, 2021

dpausp commented Jun 24, 2021

tomfitzhenry commented Jun 25, 2021

prusnak commented Jun 29, 2021

adisbladis commented Jul 2, 2021

prusnak commented Jul 2, 2021

dotlambda commented Jul 2, 2021

adisbladis commented Jul 2, 2021

clefru commented Jul 3, 2021 • edited Loading

adisbladis commented Jul 5, 2021

github-actions bot commented Jul 5, 2021

zeri42 Jun 22, 2021 •

edited

Loading

clefru commented Jul 3, 2021 •

edited

Loading