Packages that can be noarch

[isuruf]

By dropping support for python<3.8 or by using noarch_platforms as in #1839

Here we collect some packages. PRs are welcome to help

• future (Noarch future-feedstock#28)
• win_inet_pton (make noarch with __win win_inet_pton-feedstock#17)
• click (Noarch click-feedstock#44)
• importlib_metadata ([bot-automerge] importlib_metadata v5.0.0 importlib_metadata-feedstock#109)
• mock (Noarch mock-feedstock#38)
• pluggy Make noarch pluggy-feedstock#32
• pep517 (Noarch pep517-feedstock#16)
• pkgconfig (make noarch (py27 is no longer supported) pkgconfig-feedstock#35)
• sh ( Convert to noarch: python sh-feedstock#23 )
• desert (uses dataclasses on py<37) Make Noarch desert-feedstock#8
• manhole Make noarch manhole-feedstock#7
• openpathsampling Make noarch openpathsampling-feedstock#21
• curtsies Make noarch and v0.4.1 curtsies-feedstock#23
• hypothesis make noarch: python hypothesis-feedstock#752
• httpie make noarch: python httpie-feedstock#29
• constructor make noarch constructor-feedstock#63

Harder ones

• pytest (by making two noarch packages. One for py<311 and one for py>=311) make noarch pkg pytest-feedstock#156
• django (by adding a backports.zoneinfo empty package for py>=3.9) make noarch package django-feedstock#150
• zaber-motion

@jaimergp: Feedstocks that have a py{<,<=}3{6,7,8} selector without compiler dependencies, still not noarch and no python_impl mentions (cs.github.com query). Search results say 110+ feedstocks but you can only retrieve 100 at a time so here we are. Some of them might not apply for other reasons( could not review all of them).

• adaptive
• adaptive-scheduler
• aio-pika
• aiopeewee
• aiosmtpd
• airflow
• ansi2html
• astral
• astroid
• bempp-cl
• bgen-reader
• bluesky-httpserver
• cached_path
• catalogue
• certbot
• chalice
• cloudpathlib
• cmd2
• cogapp
• compas_view2
• confection
• dandi
• databases
• datalad
• dicomweb-client
• dirty-equals
• docx2pdf
• dvc
• ecco_v4_py
• eliot
• exchangelib
• flake8-annotations
• flake8-typing-imports
• fortls
• glue-core
• glueviz
• gunicorn
• inspiral-range
• jaraco.text
• jupyter-docx-bundler
• keyring
• kombu
• mcculw
• mdacli
• mne-qt-browser
• mnelab
• montreal-forced-aligner
• moreorless
• mss
• multigifview
• napari-tiler
• nipype
• openconnect-sso
• openstacksdk
• osprofiler
• panda3d_viewer
• pdir2
• pluggy
• poetry ( MNT: Add noarch: python poetry-feedstock#76 )
• porespy
• psychopy
• ptpython
• pyannote.core
• pyannote.database
• pykakasi
• pynwb
• pyodesys
• pystow
• pytz-deprecation-shim
• qcodes
• resonance
• robotframework-browser
• rpaframework
• rtree
• sanic
• sanic-plugins-framework
• sensormapgeo
• single-source
• sos-notebook
• spleeter
• sqlacodegen
• stitchm
• tramway
• trio_asyncio
• trio-asyncio
• tzlocal
• unzip-http
• uwsift
• vega
• virtualenv
• vyper
• web3
• xonsh
• xraylarch
• xrt
• zxpy

cc @conda-forge/core

[h-vetinari]

Would be amazing to have pytest be noarch!

hypothesis might be a candidate as well. It recently got made yesarch again, but would presumably be noarch-capable if the django workaround for backports.zoneinfo arrives.

[jakirkham]

It's worth noting that with pytest, it also has some conditional Windows dependencies. So we probably get 4 noarch packages out of it. Still that is probably worthwhile.

Perhaps if we wanted to simplify that, we could turn the backport packages it depends on into noarch packages with the Python version conditioning and then always depend on them in pytest (or anywhere else).

[BastianZim]

Desert: conda-forge/desert-feedstock#8
Pluggy: conda-forge/pluggy-feedstock#32

[BastianZim]

And some more
conda-forge/manhole-feedstock#7
conda-forge/openpathsampling-feedstock#21
conda-forge/curtsies-feedstock#23

Also, the "CircleCI Pipeline" error pops up on all of them. Is there any way to remove this?

[hmaarrfk]

I think this package could be made a noarch platform, but would require a platform for every package.

Basically, it is a C library, that is loaded with ctypes into a python package:
https://github.com/conda-forge/zaber-motion-feedstock

[h-vetinari]

Some very nice PRs from Isuru:

• allow selectors in noarch_platforms recipes conda-smithy#1682
• make noarch package django-feedstock#150
• make noarch: python hypothesis-feedstock#752
• make noarch pkg pytest-feedstock#156

[jaimergp]

I made some queries on cs.github.com and found like... 100 feedstocks we could noarchify. I haven't reviewed all cases though, but I am more or less confident most would be eligible. Check the first message!

[BastianZim]

Slightly related project that popped up a couple of times but was never really taken up. If we can create one source of truth for packages where the selectors can be ignored, we could add them all to Grayskull, and I could just run Grayskull on that list.

There is already https://github.com/conda-incubator/grayskull/blob/6955333ee01f83ba6ae6e8dc76ce1f576e7c762e/grayskull/strategy/config.yaml#L465 and https://conda-forge.org/docs/maintainer/knowledge_base.html#non-version-specific-python-packages but it's not connected.

[beckermr]

I can fix the circle stuff. We can delete the webhooks.

[ocefpaf]

Folks, how are we handling the old jupyter extensions that have a pre/post script?

cc @xhochy

[jakirkham]

Folks, how are we handling the old jupyter extensions that have a pre/post script?

cc @bollwyvl

[BastianZim]

I made some queries on cs.github.com and found like... 100 feedstocks we could noarchify.

Poetry is already noarch

[jakirkham]

Poetry is already noarch

Looks like that just happened today. Likely the list was made before today and the box wasn't checked yet. Have now checked it

[bollwyvl]

Folks, how are we handling the old jupyter extensions that have a pre/post script?

Haven't looked, but we can likely emit appropriate ${PREFIX}/share/jupyter and ${PREFIX}/etc/jupyter/jupyter_notebook_config.d/<pkg_name>.json that would have the same effect as whatever the scripts were doing.

[carterbox]

I think this package could be made a noarch platform, but would require a platform for every package.

Basically, it is a C library, that is loaded with ctypes into a python package: https://github.com/conda-forge/zaber-motion-feedstock

@hmaarrfk, TomoPy is like this. I have one noarch python package which depends on platform specific packages which contain the shared libraries. Could have separate feedstocks, but I just have one and build the same noarch package every time.

[jaimergp]

I can fix the circle stuff. We can delete the webhooks.

@beckermr Is it enough to delete the webhooks at e.g. https://github.com/conda-forge/curtsies-feedstock/settings/hooks ?

[BastianZim]

Is it enough to delete the webhooks at e.g.

Is there any way someone outside of core can do that? I have so many packages that have that, pinging core each time is going to spam your inboxes...

[beckermr]

Yes deleting the webhooks is fine.

@BastianZim If you want to write an admin migration to do that, feel free at conda-forge/admin-migrations.

[ocefpaf]

Is there any way someone outside of core can do that? I have so many packages that have that, pinging core each time is going to spam your inboxes...

I'm OK if you ping me. Every migration we find tons of packages that can be noarch. Last one I converted dozens and it will be easier to merge someone else PR than doing it myself.

@BastianZim If you want to write an admin migration to do that, feel free at conda-forge/admin-migrations.

That would be quite complicated b/c the new semi-noarch recipes are not a one-size fits all.

[BastianZim]

If you want to write an admin migration to do that, feel free at conda-forge/admin-migrations.

Ahh true, forgot that. Is there any guideline on that? I've never written one.

[BastianZim]

I'm OK if you ping me.

Thank you! But the CircleCI stuff also appears in my normal feedstocks so it would be quite a lot...?

That would be quite complicated b/c the new semi-noarch recipes are not a one-size fits all.

I think this is just about removing the CircleCI hook?

[ocefpaf]

I think this is just about removing the CircleCI hook?

Ah. Then it is definitely worth a try with a migrator.

[BastianZim]

Rest would be nice but probably impossible. Although, if we have a list of empty packages we can ignore (#1840 (comment)), I can run Grayskull on most and probably automate ~80% of the PRs to the point where they just need to be merged.

[beckermr]

@ocefpaf I am talking about an admin migration at conda-forge/admin-migrations, not a bot migration.

[jaimergp]

org:conda-forge path:meta.yaml "- python" "- pip" "# [win" NOT compiler NOT noarch: returns 240 matches in cs.github.com. These have the potential to be pure Python packages with Windows-only dependencies that are not currently noarch... A migrator might indeed be the best solution.

These feedstocks are using MxP jobs when they could be using 2... (M = number of enabled architectures, between 3 and 6; P = number of Pythons, currently 3 or 4). This is 9 jobs at best, 24 at worst!

[jaimergp]

org:conda-forge path:meta.yaml /pywin32\s+#\s+\[win/ NOT compiler NOT noarch: gives you 20 results for potential noarch packages that are not just because they require pywin32, but we could use the pywin32-on-windows metapackage instead!

[jakirkham]

Couldn't we do the same thing with pywin32 dependents as other OS specific dependencies? Namely add an OS specific branch and still build with noarch?

[jaimergp]

Yes, sure, that will always work. But it's a simpler modification to replace just pywin32 in those cases (and a single job instead of two). It could even be done at the repodata level if we had the right metadata.

[jakirkham]

Maybe we can move conda-forge-ci-setup to noarch

[h-vetinari]

Maybe we can move conda-forge-ci-setup to noarch

I'm not sure that's compatible with work such as conda-forge/conda-forge-ci-setup-feedstock#210? Or would the suggestion be - for debugging purposes - to switch off noarch when preparing a PR?

[jakirkham]

Have different thoughts on how to approach that as well, but that's a discussion for a different issue

[mbargull]

Just an FYI:

• Added in conda 4.8: __osx
• Added in conda 4.9: __win, __unix
• Added in conda 4.10: __linux

We appear to still have people a non-negligible amount of users with conda 4.9 (more than all of <4.9 combined).
We thus might not want to make broad use of __linux just yet.
(But our main use case here is to have a __win/__unix distinction so that shouldn't impose much of a restriction.)

[jakirkham]

Wonder if we could just create those as arch packages and stick them under a special label (like legacy) and advise users on older Conda versions to use them. As they will encounter an error message about a missing package, we could put this in the docs under FAQ so it is easily discoverable.

[mbargull]

@jakirkham, not a bad idea.
I'm +1 on offering (otherwise non-intrusive) ways to reduce the amount of brick walls users could hit when they want to upgrade old installations.
We'd have to do some testing beforehand, though (1. does anaconda.org/anaconda-client still allow uploads of __* packages, 2. how do conda/mamba behave when updated if there is still a conda-forge/label/legacy/__* package installed, etc.).

[jakirkham]

Even if the uploads are blocked, we could include the recipe in the docs. It is probably a couple lines. Once built it should be installable from the local package cache.

[jaimergp]

Do we need the actual packages or can we mock them in a repodata patch? Just an idea in case that's easier to implement.

[jakirkham]

Thinking about this more I think it is on end users to build these packages themselves. Not all cases check simply for the existence of these packages. Some of them (like __osx or __glibc) check the version as well. So it really depends on end users to know their system and build these packages with the right versions baked in. That way they behave correctly when performing a solve.

[hmaarrfk]

Is the following issue known: conda-forge/spyder-kernels-feedstock#87

[bollwyvl]

Looking at what ipykernel does is probably relevant in the Spyder case. I figure we would have heard if it wasn't working.

[hmaarrfk]

Is https://github.com/conda-forge/xcb-proto-feedstock a candidate? It seem like it puts things outside the python directory so I'm not sure if it would work. But the noarch platforms would help reduce the build matrix size quite a bit.

[jaimergp]

Is https://github.com/conda-forge/xcb-proto-feedstock a candidate?

Maybe but you'd need packages for all the platforms (six so far?).