MacOS Catalina Abort Trap 6: requires cryptography >=2.8 or asn1crypto >= 1.0.0

[joelfrederico]

I've traced an issue here. MacOS Catalina has an issue where it fails to import OpenSSL with abort trap 6. This is due to dependencies. The dependencies have fixed the issue, but OpenSSL hasn't updated its dependencies.

It looks like pyopenssl has two options: work around the old versions (unlikely) or update its dependencies.

The direct dependency cryptography should work for versions >= 2.8, where cryptography dropped dependency on asn1crypto. (https://github.com/pyca/cryptography/blob/master/CHANGELOG.rst)

Alternatively, pyopenssl could specify that asn1crypto needs to be versions >=1.0.0. (wbond/asn1crypto#158)

This is causing problems for a large variety of packages that depend on pyopenssl, including perhaps most notably pip.

[tiran]

pip, requests, and urllib3 don't depend on PyOpenSSL. Don't install requests with requests[security] dependency. The ssl module in Python's stdlib has been able to create secure connections for many years.

[reaperhulk]

Even ignoring @tiran's point I'm not sure what we can do here. pyOpenSSL only holds a transitive dependency to asn1crypto through outdated versions of cryptography. Updating the dep fixes the issue (which is admittedly more of a challenge if pip is crashing). We could raise our minimum dep to cryptography 2.8, but that is a pretty big hammer to fix an issue that isn't really pyOpenSSL's.

[tiran]

I have reported the issue with requests and requested to drop implicit monkey-patch of urllib3 with PyOpenSSL.

How did you install PyOpenSSL? I'm getting cryptography 2.8 without asn1crypto when I install PyOpenSSL in a new virtual env.

[joelfrederico]

Regardless of any other points, it's possible to have pyopenssl installed with cryptography<2.8 and asn1crypto<1.0.0. Then it's impossible to import OpenSSL on Catalina.

There are ways to get there- before installing PyOpenSSL, install asn1crypto<1.0 and then cryptography<2.8. This is in fact happening to a variety of people.

There are two arguments to this. One, is that cryptography<2.8 and asn1crypto<1.0.0 shouldn't be available on Catalina. The other is that pyopenssl should specify dependencies that work.

Arguably, cryptography and asn1crypto should have specified those versions to only work with environment markers for darwin releases less than 19.0.0. But this gets into the issue- how are you supposed to know that a future OS change will break your build? Everybody would have to specify their max compatibility, but almost nobody does that.

At any rate, for cryptography and asn1crypto to fix this now, they'd have to go back and create new wheels for every past version that specify their max os for darwin as <19.0.0. I think we can all agree they're not going to do that.

I think pyopenssl can either fix it or not. If it isn't fixed, then it will be possible to end up with a broken pyopenssl. The practical effect is that somebody else will have to deal with this, and in this case, it will have to be everybody who uses pyopenssl. That includes some popular packages. Their options will be to fix pyopenssl's dependencies, or to switch away from pyopenssl. Either way, not a great look for pyopenssl.

[joelfrederico]

Of course, maybe that's what pyca wants? They do recommend using cryptography instead of pyopenssl.

[reaperhulk]

We couldn't create new wheels even if we wanted to. PyPI does not allow replacement of artifacts.

I am personally fine with bumping the minimum version in pyOpenSSL and doing a release, but that will only help people who aren't pinned to an older version. It's unclear to me how many people aren't pinning but are still getting old cryptography, but this is an easy change so ¯\(ツ)/¯

[joelfrederico]

Ah. Yeah. Version pinning. Um... eesh. I mean, what I'm really advocating for is to make the non-eager pip upgrades work. Everybody else in their chain needs to upgrade anyways... It's sort of ugly, all of this. I might more properly put the blame on the python installer that didn't eager upgrade all of the packages?

It's really not clear to me what should happen.

[reaperhulk]

@joelfrederico Yeah this is an ugly situation. We can do a pyOpenSSL 19.1 to make pip upgrade cryptography, so that should at least help a few people avoid this bug.