[Python 2 has been deprecated] - Formalise Python support policy

[felix-hilden]

Status update

November 1st, 2021 @ 10:36AM EST by @ichard26

We've deprecated Python 2 support starting with version 21.10b0. Assuming things go as planned the support will be dropped sometime in January of 2022 along side the first stable release of Black.

---

From the discussions in #2247. The new FAQ (once merged) contains information on the supported Python versions. @ichard26 mentioned that it would be nice to have such documentation elsewhere in the docs too.

As discussed, the support plan seems to essentially be: "support whatever CPython versions are currently not EOL'd". This is for running Black. Then another question is formatting files, which is currently also done for 2.7.

• Should the Python version support of Black be considered identical to CPython official support?
• Does file formatting follow that, or do we intend to support older versions too?

[cooperlees]

I'd like to propose dropping Python 2 formatting support in the near future. It's still causing formatting decisions that are not beneficial to modern python3 users - e.g. fr"" vs rf"" strings.

[felix-hilden]

To go a bit further, would we also like for supported formatting versions follow the supported runtime versions? Currently black/mode/TargetVersion has versions from 3.3 onwards too.

[cooperlees]

To go a bit further, would we also like for supported formatting versions follow the supported runtime versions? Currently black/mode/TargetVersion has versions from 3.3 onwards too.

I believe this is the goal. e.g. as soon as Python 3.6 is EOL'd we'll remove that target version and CI from black + no longer maintain comparability etc.

[septatrix]

I strongly believe that teams which still use Python 2 can simply fix their black version to a version still supporting py2. They have to do that for other tools anyway (like pip which also does not support py2 any longer).

[pradyunsg]

Honestly, I don't see any good reason to maintain support for a single day beyond EoL.

The main beneficiaries of extended post-EoL support for Python versions are organisations/institutions. These have taken on risk by staying on EoL software. One of the risk factors is lack of support in tooling that they don't pay for. The onus is on them to figure out how to keep their lights on. However they do so, it must cost them $$$ -- either by paying maintainers enough to justify their effort spent maintaining that support, by having the developers they have hired maintain the support for them internally, or by paying a redistributor to maintain that support for them.

From https://www.curiousefficiency.org/posts/2015/04/stop-supporting-python26.html:

upstream open source developers shouldn't feel obliged to donate their own time to help end users cope with organisations that aren't yet able to upgrade their internal infrastructure in a more timely fashion.

So, yes, please feel free to drop support for EoL Python versions in the first release you make after the EoL date. If someone complains, tell them that they are not obligated to get support on EoL software for free.

[ichard26]

To be honest, I'm +1 to drop Python 2 formatting support because it's been over a year since March 2020 and even major projects have dropped Python 2 support. While the burden isn't that high so far, it's going to get worse. The main thing is that it would have to be well communicated (as much it's impossible) and not just something we do on a random day :)

IMO, by the end of the year seems fine to me. If you want something locked to a release, either stable (assuming we do it soon), the one after that is also fine by me!

In particular I'd like to drop Python 2 related tests in the test suite (which make the test suite ... annoying to run properly to say the least). Most users still stuck on Python 2 are probably commercial users and we aren't paid, so ... it's basically what @pradyunsg said (well from a formatting support view, I'll post my comments about runtime version support later).

Dropping 3.3 to 3.5 (inclusive) formatting support is also fine by me, but those have less of a burden (but then again, those versions are rarely used).

[JelleZijlstra]

Maybe in the next release we can print a warning if people are using Black to format Python 2 or 3.3-3.5 code, similar to what pip did.

[ichard26]

Maybe in the next release we can print a warning if people are using Black to format Python 2 or 3.3-3.5 code, similar to what pip did.

I'm happy with this, although I do believe we should figure out a support policy that we can point to so we don't look like we're just dropping support randomly :)

Detecting Python 2 should be easy but detecting 3.3-3.5 might a fair bit harder, IIRC the target-version autodetection falls back to Python 3.x if it can't find any sort of version indicator from the AST. That would led to a fair bit of false-positives ... I'm not sure about this part of the code though, so maybe everything is fine and my concerns are moot :)

Should the Python version support of Black be considered identical to CPython official support?

Sadly while going with the EoL approach makes the most sense to me, distros like Ubuntu 18.04 LTS exist and they kinda mean 3.6 is going to be in hot demand for a long time. Right now it's fine since Black's Python requirement is >=3.6.2 and 3.6 is NOT EoL'd yet, but it's going to be by the EOY. So I'm 50-50 on dropping support runtime support for 3.6 on EoL.

Does file formatting follow that, or do we intend to support older versions too?

I don't really care (other than dropping Python 2) but I think it would be better to make it consistent by some sort of logic so even we drop the ball on communicating changes, we should be fine (tm). Matching or trailing runtime support (by a version or two) ... I don't care much.

Thoughts?

[septatrix]

Detecting Python 2 should be easy but detecting 3.3-3.5 might a fair bit harder, IIRC the target-version autodetection falls back to Python 3.x if it can't find any sort of version indicator from the AST. That would led to a fair bit of false-positives ... I'm not sure about this part of the code though, so maybe everything is fine and my concerns are moot :)

IIRC black defaults to produce code for the version it is running under except when told to do otherwise (using e.g. flags)

Should the Python version support of Black be considered identical to CPython official support?

Sadly while going with the EoL approach makes the most sense to me, distros like Ubuntu 18.04 LTS exist and they kinda mean 3.6 is going to be in hot demand for a long time. Right now it's fine since Black's Python requirement is >=3.6.2 and 3.6 is NOT EoL'd yet, but it's going to be by the EOY. So I'm 50-50 on dropping support runtime support for 3.6 on EoL.

However these distros also ship a python3-black package or similar which pins black to a fixed version. Even if that is not the case they are able to get a supported version with pip.

[pradyunsg]

distros like Ubuntu 18.04 LTS exist

Worth repeating -- just because a distribution of your software is being supported by another group doesn't mean you have to support it too.

[felix-hilden]

So it seems that dropping Python 2 is obvious, and Jelle's suggestion of printing warnings in the next release seems reasonable. In terms of mostly-compatible 3.3-5 code, I think simply removing the target version options and special cases might be enough. We could say that we guarantee nothing for those versions, but if the code is compatible, then why not format it after all.

I do recognise 3.6 will die more slowly with Ubuntu, so there is value in keeping it, even though we wouldn't have to do it for the leeches that are all commercial parties and big organisations 😏 (/s) I'm somewhat indifferent about it now, because compared to Py2 it's a way smaller issue. I'd like to have a stricter policy, meaning Python support = Black support, but it shouldn't be too much trouble to feed our Ubuntu users some improvements too 😄 (and let our developers develop 😉).

[ichard26]

[...] even though we wouldn't have to do it for the leeches that are all commercial parties and big organisations 😏 (/s)

While I have some gripes with $big-corp stretching FOSS resources thin, I haven't reached that level of jaded (yet, probably) :P

IIRC black defaults to produce code for the version it is running under except when told to do otherwise (using e.g. flags)

Seems like Black defaults to target-version=*?

❯ cat run.py
import black
black.main(["src/black/trans.py", "-v", "--experimental-string-processing", "-l", "79"])

~/programming/oss/black on mypyc-support-pt2 [$!?] via Python v3.8.5 (black) 
❯ black run.py --check --config=tests/empty.toml --fast -q
detected-versions={<TargetVersion.PY39: 9>, <TargetVersion.PY38: 8>, <TargetVersion.PY33: 3>, <TargetVersion.PY27: 2>, <TargetVersion.PY34: 4>, <TargetVersion.PY36: 6>, <TargetVersion.PY35: 5>, <TargetVersion.PY37: 7>}

Looking at the numbers, there's not as many Python 3.6 users out there as I originally believed. Today 15% of downloads were on 3.6:

https://pypistats.org/packages/black

It's still a lot of users, yes, but given that it's trending downwards, maybe dropping 3.6 on EoL will be fine? The argument of "you shouldn't expect support on EoL support for free" does appeal to me too. Also, I just remembered it's only the updates that they will be missing out on, and that doesn't feel that bad to me :) No strong feelings here though, so if other maintainers feel differently, I won't object.

---

Anyway, I still want to push forward on an official support policy that we can write down (and point to if needed).

I'd like to propose dropping Python 2 formatting support in the near future.

When exactly? By the end of the year? X releases after the first release that warns on Python 2?

Does file formatting follow that, or do we intend to support older versions too?

For the sake of simplicity (and throwing an idea out there so people can either agree or object - ie. push issue forward 😉), I don't care and would probably choose "runtime support = formatting support". Once again, I don't feel strongly (although if you dropped Python 2, it's kinda standard to drop 3.3-3.5 too).

[Xtrem532]

My 2 cents:
When a Python version is EOL'ed, remove all testing/CI for that version, add a warning to all Outputs in/for that Version, and treat that Python version as if it didn't exist. I would not remove the version specific code unless it gets in the way of a new feature/version.

[ichard26]

Here's my proposal, let's deprecate Python 2 in the next release and remove all related Python 2 code after December 1st, 2021. Python 3.3-5 will remain untouched for now. Related PR here: #2523.

I'm happy to change the dates if needed ^^

[graingert]

Here's my proposal, let's deprecate Python 2 in the next release and remove all related Python 2 code after December 1st, 2021. Python 3.3-5 will remain untouched for now. Related PR here: #2523.

I'm happy to change the dates if needed ^^

Please wait one additional month and drop support in v22b0, then it's clear black21 supports python 2 and black22 does not