Dropping Python 3.7

[dopplershift]

We've reached a point where the NEP29 suggestion for dropping 3.7 (December 2021) is much different from the official EOL (June 2023). We probably should decide what the policy is actually going to be, since it's becoming a relevant question for some builds.

#1947 seemed to indicate some confusion on whether we're following NEP29 or not.

[hmaarrfk]

I thought we had this discussion and an important point was to ensure that we worked with the non-scientific community as well:
conda-forge/conda-forge.github.io#1616

[dopplershift]

Thanks, I had forgotten about that issue. I think there is still an issue with actually firmly establishing what our schedule/policy will be going forward. The issue I linked above is a practical case of:

1. Needing to rebuild an older version against our updated pins due to upstream moving on
2. Confusion about whether we're following NEP29 or not

So while that issue indicates we shouldn't drop it now, there's still a question of: when are we dropping 3.7? What are we doing when 3.11 comes out? (Beta 1 drops first week of May, official release 5 months later)

[jakirkham]

Typically we've dropped an old version when adding a new one.

[chrisburr]

I don't think we necessarily need a formal policy for this do we? In the case of Python 3.6 it made sense as end of life was close and it was an issue for the OpenSSL migration (only 3.7+ supports OpenSSL 3).

When 3.11 is released we might want to reduce the size of the build matrix, especially for ppc64le/aarch64. That said, I'd also be happy with keeping it for another year given that Python is moving towards having 5 years of support and one release per year. Especially as it seems to be getting a decent number of downloads for the latest patch.

Regardless, if an issue arises that causes Python 3.7 support to become annoying we should keep the option for dropping it sooner.

[jakirkham]

Not a formal one, but informally we have tended to keep 3 Python versions around and drop the oldest one when adding a new one. There may be other reasons like approaching EOL or other assessed costs. The reason for this informal policy is as you say keeping the build matrix reasonable.

Individual packages as always can drop support for things they no longer support or things that cause issues.

[isuruf]

As discussed in the core meeting today, let's drop 3.7 when 3.11 comes out and we start the migration in October. We can make an announcement and people can voice their concerns in this issue.

[jakirkham]

Submitted PR ( conda-forge/conda-forge.github.io#1815 ) to announce the intention to drop Python 3.7

Edit: Here is the announcement entry.

[bkpoon]

Will there be a way to still build Python 3.7 versions of a package on the conda-forge channel by modifying conda_build_config.yaml? We have users that use conda packages in Google Colab notebooks with condacolab and write algortihms integrating with AlphaFold (machine learning for structural biology). As of today, both are still on Python 3.7. Thanks!

[jakirkham]

@jakevdp, do you know who we should talk to about updating Google Colab's Python version?

[jakevdp]

I would ask on this issue: googlecolab/colabtools#2165 One of the Colab team members might be able to repond there regarding the current ETA.

[jakirkham]

Thanks for the speedy reply, Jake! 😄

Have posted a comment here ( googlecolab/colabtools#2165 (comment) ) asking for an ETA as well as outlining the situation. Others should feel free to include additional info.

[bkpoon]

It looks like the update might be intertwined with an update to the Ubuntu image (googlecolab/colabtools#1880). I guess we'll wait and see.

We might switch to binder, but we'll miss the GPU.

[jakirkham]

The existing packages will still be there. So those can still be used.

We just won't be building more things for Python 3.7.

We need to make room for the new Python version. Plus we are building for different kinds of hardware. Not to mention PyPy support.

While we would like to support more Python versions and implementations. CI resource are a limitation. Maybe in a future world where Python has a stable ABI across minor versions and implementations (like using HPy), this could be possible. At present these are the constraints we are operating within.

[bkpoon]

No problem, our production releases use explicit conda environments and I can switch over to building on our own Azure organization if we really need to maintain Python 3.7 support for a little while longer.

[h-vetinari]

I would ask on this issue: googlecolab/colabtools#2165 One of the Colab team members might be able to repond there regarding the current ETA.

Looking at the discussion there, this has been brought up with urgency for a while already (numpy 1.22 dropped 3.7 support almost a year ago), but the maintainers there have not responded to escalating pleas (and people cancelling their subscriptions), for about half a year, so it doesn't look too good unfortunately. But that's not something we can do anything about.

[chrisburr]

Closing as Python 3.7 was dropped from the build matrix in #3573

Existing versions will still be available for Python 3.7 and noarch: python packages are unaffected by this as they have their own package specific version pins. If any conda-forge package maintainers specifically need to keep Python 3.7 support they can ask on gitter for help.

[vallsv]

Hi, is there somehow a way to pinpoint some project to avoid the robot to remove python 3.7?
Any idea how could i revert a project which was already cleaned up?
Thanks.

[vallsv]

@chrisburr Hi, could you tell me what is gitter, and how could i ask to keep Python 3.7 for a specific project for now? See above.

[dopplershift]

@vallsv Here is all the documentation on contacting the conda-forge team, including Gitter.

[jakirkham]

FWIW Google Colab is now on Python 3.8 ( googlecolab/colabtools#2165 (comment) )! 🎉

[ngam]

Are we following NEP29? If so, are we dropping 3.8 soon? Projects have begun dropping 3.8 and it's actually not trivial to fix the pinning, so we may need to adopt a community-wide solution soon instead of disrupting individual feedstocks. Maybe we should open a new issue to discuss? I think we generally need to follow a policy and it seems NEP29 is the best available one out there (but it could be disruptive to other communities, e.g., non-scientific ecosystems as hmaarrfk pointed out above). I am not taking a position specifically, but just highlighting the challenges of not adopting an official policy as we seem to be doing now.

[jakirkham]

Typically we drop a Python version around the same time we are adding a new one

As noted in these discussions before we have many more Python packages than just those affected by NEP 29 (IOW outside Scientific Python/PyData ecosystem), so that policy ends up not covering those use cases

At the end of the day recipe maintainers are always able to skip Python versions and otherwise affect version constraints as needed

[h-vetinari]

Typically we drop a Python version around the same time we are adding a new one

Thing is that python releases have been moving together due to the new 12 month cadence, as opposed to 18 month previously. We're now getting to the point where around the time we add 3.12, 3.8 still has well over a year before it's EOL (end of 2024). We may have to accept an extra python version in the build matrix...

[ngam]

Yeah, so the policy unofficially is to follow the Python version EOL, right? Can we update the numpy versions associated with the python versions then? Or should we keep them intact until we address the Python versions? I will just open a PR and see what people think.

At the end of the day recipe maintainers are always able to skip Python versions and otherwise affect version constraints as needed

Yes, understood, as linked in my comment. However, the point is that we should minimize that to protect the integrity of the entire ecosystem. At least that was my understanding of the wide pinning...

[h-vetinari]

to protect the integrity of the entire ecosystem

How long we maintain packages for a given python version shouldn't affect integrity. Numpy already dropped support for 3.8 in the last release and the world hasn't stopped.

Not sure what scenario you have in mind, but AFAICT it's mostly a question of CI resources whether we want to support 5 python versions at a time.

My point (not addressed to you particularly) is that previously, with an 18 month cadence, 4 releases covered 6 years of real time, and as a new release appeared, it was more than time to drop the oldest one. Now with a 12 month cadence, 4 releases only cover 4 years, and thus less than the official upstream support.

Already for dropping python 3.7 half a year before its EOL, there were people asking if we can keep or readd support. I think that's going to become even more relevant this time.

[ngam]

Sorry, I actually misunderstood the numpy--python zip situation. My only concern here is the numpy part and I didn't realize it was possible to move the numpy pin ahead (like you did, and I followed your suite in #4730) and the leave the python pin intact.

On the general python debate, I usually find myself agreeing with you, so add my vote to yours 😉

[jni]

Hi folks,

The existing packages will still be there. So those can still be used.

When I try [conda|mamba|micromamba] create -n 37 python=3.7 ipython, I get a message that Python 3.7 doesn't exist. Search confirms that.

I remember reading a while back that packages for old Python versions would be siloed somewhere (to keep solving size reasonable), but I can't remember where — is that still the case?

Is the problem just that Python 3.7 was never built for osx-arm64?

Thank you! 🙏

[chrisburr]

Is the problem just that Python 3.7 was never built for osx-arm64?

Yes.