Deprecate or update formulae that depend on Python 2

[danielnachun]

With the announcement of https://developer.apple.com/documentation/macos-release-notes/macos-12_3-release-notes#Python, I think we are at the point where we need to deprecate or update all remaining formulae that depend on Python 2

Update:

Needs PR

• app-engine-python - (see below comment) - either switch this formula to use https://github.com/GoogleCloudPlatform/appengine-python-standard as the new upstream, or deprecate this formula and make a new one.
• cfv - old upstream is dead, there is maintenance fork that is supposed to add Python 3 support, but hasn't made a stable release yet: Python 3 Support cfv-project/cfv#8 - we should be able to update, though we can't wait indefinitely for this.
• viewvc - the 1.3.0 release will support Python 3: https://github.com/viewvc/viewvc/milestone/9 - we should update when available.

Open PR

Fixed

• aubio - has a hardcoded reference in waf build scripts to /usr/bin/python - we should try to update, to see if it is compatible with Python 3. Arch Linux build script uses Python 3: https://github.com/archlinux/svntogit-community/blob/packages/aubio/trunk/PKGBUILD: aubio: fix build on linux #95472
• bde - our formula uses waf scripts that don't support Python 3 - we should update this to use the new build system which does support Python 3: https://bloomberg.github.io/bde/library_information/build.html - bde: use brewed python #96951.
• blastem - there are upstream commits to add Python 3 support - being updated here: blastem: use Python 3 and remove unneeded resources #95337.
• blockhash - calls to waf in formula need to explicitly use Python 3 - we should update: blockhash: use python3 #95411
• cppcms - C++ package that only uses Python 2 for test scripts - we can try to patch these using upstream commits, but for now the only thing needed for this to build without Python 2 is an edit to the CMakelists - we should update - cppcms: use brewed python #97007.
• czmq - seems to support Python 3 - we should update. (czmq: remove macOS dependency #94095).
• ffmpeg2theora - depends on libkate, and is using a patch from Debian for Python 3 support - development appears stalled - we should deprecate.
• git-cinnabar - upstream repo explicitly says it supports Python 3 - we should update. (git-cinnabar: use python3 #93971)
• hsd - currently depends on node@10, which needs Python 2, but upstream says it should work with node@14: nodejs v15 is NOT supported handshake-org/hsd#577 - we should update: hsd: depend on node@14 #95412
• libkate - builds a Python 2 tool KateDJ. Optional: Development seems stalled - we should deprecate. Removed Python 2 bindings in libkate: avoid installing python2 KateDJ #99939
• libming - upstream issue for Python 3 support has not been resolved: python3 support libming/libming#177, but we can probably disable Python bindings - we should update: libming: remove python bindings #95474
• libpgm - we are using an old upstream that is unmaintained - the new upstream has much newer releases that explicitly support Python 3: https://github.com/steve-o/openpgm - we should update: libpgm 5.3.128 #95457
• lv2 - seems to support Python 3 - we should update. lv2: use brewed python #96190
• norm - waf build script needs Python 2 and it appears to also build Python 2 bindings. Does not seem to have commits for Python 3 support, despite being actively developed. I think we need to open an upstream issue asking about Python 3 support.
• opensearch-dashboards - currently depends on node@10, will switch to node@14 with 2.0 release: opensearch-dashboards 1.3.0 #97888 (comment) - we should update when available. - opensearch-dashboards 2.0.0 #102390
• prestodb - launcher.py script tries to use /usr/bin/env python - we should update this to use brewed Python directly: prestodb: use [email protected] #95408
• rethinkdb - now had upstream commits to remove v8 dependency - need to clarify a if a new 2.4.x point release is coming with these changes, as there are many commits. -- rethinkdb 2.4.2 #100256
• saldl - calls to waf in formula need to explicitly use Python 3 - we should update: saldl: depend on [email protected] #95413
• serialosc - waf build script shebang uses Python 2. Seems to run by passing to python3 - serialosc: fix build on Linux #99919
• snap7 - test calls python - should update to use a better test. snap7: remove python call from test #96322
• speedtest-cli - updated to support Python 3: speedtest-cli: depend on [email protected] #95332
• spidermonkey - should now support Python 3: https://bugs.gentoo.org/698910#c16 - we should update (spidermonkey: depend on [email protected] #94820).
• termbox - waf build scripts need Python 2, and project is now unmaintained - there is a maintained fork under termbox organization termbox 1.1.4 #100259
• traefik@1 - old version of traefik. Was able to build w/o Python 2 - traefik@1: build on Linux #99931
• traildb - waf build scripts need Python 2, but an upstream commit has been made to make this compatible with Python 3 traildb/traildb@053ed8e - we should update: traildb: build with python3 #95469
• udis86 - needs to depend on [email protected], and specify Python path with --with-python - we should update: udis86: depend on [email protected] #95414
• volatility - supports Python 3 with volatility 3 (https://github.com/volatilityfoundation/volatility3) which uses a different repo - either update existing formula, or create a new one for volatility3 and deprecate this formula. (volatility 2.0.0 #94275)
• weighttpd - waf build system needs Python 2, but we can change it to autotools build and it should work - we should update: weighttp: build with autotools #95415
• woof - commit after 2012-05-31 version supports Python 3 - we should update (woof 20220202 #94258)

Deprecate:

• appscale-tools - development appears stalled - we should deprecate. appscale-tools: deprecate #99937
• brew-pip - development appears stalled - we should deprecate.
• [email protected] - old version of cassandra, which supports Python 3 in newer version - we should deprecate.
• eventql - needs Python 2 at least to build its own copy of spidermonkey - the project is unmaintained: Roadmap/Progress Update? eventql/eventql#366 - we should deprecate. eventql: deprecate #99941
• fsh - was last updated in 2001, so it definitely won't support Python 3 - we should deprecate. fsh: deprecate #96658
• git-remote-hg - development appears stalled - we should deprecate. git-remote-hg: deprecate #99940
• kibana@6 - old version of kibana, which is now disabled due to incompatible licensing - we should deprecate.
• mergepbx - seems to be written in Python 2 and development has stalled - we should deprecate. mergepbx: deprecate #96510
• nu-smv - uses a Python 2 script in the build process. There is no upstream VCS repo, and the last update was in 2015 - we should deprecate. nu-smv: deprecate #99936
• [email protected] - old version of numpy, only needed for opencv@2 - we should deprecate. [email protected]: deprecate #96336
• rfcmarkup - development seems to have stalled - we should deprecate. rfcmarkup: deprecate #99935
• ship - build dependency on Python 2. The upstream repo says it is now unmaintained because it has been superseded by kots - we should deprecate. ship: deprecate #97741
• [email protected] - old version of subversion that should be deprecated. [email protected]: deprecate #97742
• [email protected] - old version of v8 that should be deprecated.

[danielnachun]

Thanks for all of the great work on this so far! I dug up my old spreadsheet of unbottled formulae on Linux (which is now outdated to some extent), and checked all the formulae I had noted as failing to build due to some sort of Python-related issue. This has revealed a whole slew of formulae with build or runtime dependencies on Python 2 that could not be easily identified as such without trying to build on Linux. There's plenty more to do now, and please feel free to suggest changes to my recommendations.

[Moisan]

brew deps ship --tree --include-build shows that ship does not depend on Python 2 anymore.

[danielnachun]

@Moisan Unfortunately the dependency on Python 2 for ship is very indirect. ship depends on an old version of node-sass (4.9.3) which in turn depends on an old version of node-gyp that needs Python 2. However, I somehow failed to notice on the README at the Github page for that it says right at the top that ship is now unmaintained because it has been superseded by kots. For that reason we should deprecate this, and I will update the checklist accordingly.

[AlvinKuruvilla]

Can we move RethinkDB to the unsure category? I just rechecked the Github repository, and it's still getting commits. The last one was 22 days ago when writing this. The only thing that seems to have stalled is the releases. The RethinkDB python driver also remains maintained, though the last commit was in September of last year.

[danielnachun]

The reason rethinkdb depends on Python 2 to build is because it includes an old version of v8. However, there was a PR open to switch from v8 to quickjs rethinkdb/rethinkdb#6978, and it has now been merged after months of inactivity. Assuming that the commits can be applied cleanly to the current stable release, this would remove the Python 2 build dependency from rethinkdb. The future of that database is still very uncertain given that the parent company which developed it is out of business, but if it has ongoing development then it's probably okay to leave undeprecated for now.

[bayandin]

norm - waf build script needs Python 2 and it appears to also build Python 2 bindings. Does not seem to have commits for Python 3 support, despite being actively developed. I think we need to open an upstream issue asking about Python 3 support.

Will be fixed by #97001

[Moisan]

All files installed with app-engine-python require Python 2. Upstream recently released a version for Python 3 in a different repo.

[danielnachun]

All files installed with app-engine-python require Python 2. Upstream recently released a version for Python 3 in a different repo.

That's good to know. We probably need to decide if we should switch the existing formula to use the new upstream, or deprecate the current formula and make a new one altogether.

[cho-m]

For termbox, do we want to consider a fork. From repology, I see that maintainer of https://github.com/termbox/termbox has migrated the FreeBSD and NixOS packages. Also sounded like they were requesting switch for Alpine and Debian (NixOS/nixpkgs#99581 (comment)).

[Bo98]

For termbox, do we want to consider a fork. From repology, I see that maintainer of https://github.com/termbox/termbox has migrated the FreeBSD and NixOS packages. Also sounded like they were requesting switch for Alpine and Debian (NixOS/nixpkgs#99581 (comment)).

I'm ok to consider switching to that fork, but it's worth noting the future of that seems to be in termbox2 instead, which may warrant a separate formula (though that's not to say we can't both switch to the fork and ship termbox2).

Note that there is this we can apply if there's any objections to the fork: nsf/termbox@6fe63ac

[Bo98]

We're now running 12-arm64-##########-# CI nodes with macOS 12.3 installed. This can be used to see if any PRs would fail without Python 2, but it is not used for bottle uploads so it is not yet critical if it fails.

The existing CI machines will be updated next month, at which point Python 2 will no longer be available. You can consider us now in the transition period.

[SMillerDev]

With CI now missing python 2 we no longer need to track this in an issue since CI will warn us.