Clarify guidance on Python 3 :: Only packages

[jaraco]

In jaraco/backports.entry_points_selectable#5, Brandon raised the concern about packages that are Python 3 only (or that have PEP 420 support, i.e. Python >= 3.3). I can provide some background on the concern and some proposals here.

[jaraco]

background

Since this project was published, a lot of work has been done on namespace packages. The best guidance is probably the packaging users guide on namespace packages. The tl;dr is that pkg_resources-style namespace packages are deprecated and incompatible with PEP 420 packages, but pkgutil-style are compatible and interoperable with PEP 420 (native).

Fortunately, this project recommends pkgutil-style namespace packages, meaning that any backports project can use pkgutil-style packages if they require compatibility with older Pythons and native ones if they depend on Python 3.3+, and they'll be interoperable with other backports packages.

So your goal would simply be to update its description on PyPI so that it could offer advice to all of its daughter projects about — switching to PEP 420 namespace packaging if they detect Python ≥ 3.3 or something like that?

I'd like not to recommend a solution that dynamically provides different package layouts based on different Pythons. That would mean, for example, that they would have to publish different wheels for different Python versions. I can conceive of how it could be possible, but I've not seen it in the wild and I don't think it's worth the trouble.

Instead, I'd recommend that packages (primarily) use native namespace packages if their package requires Python 3.3 or later, but if their package supports Python 3.2 or earlier, it should use pkgutil-style technique.

How does that sound?

[jaraco]

On further consideration, I'm reluctant to recommend native namespace packages until some of the kinks get ironed out. There are issues like pytest-dev/pytest#3396, which mean that pytest doesn't work with native namespace packages.

I was going to recommend configparser to migrate to native namespace packages to prove interoperability and the technique, but immediately stumbled on the pytest incompatibility (https://github.com/jaraco/configparser/actions/runs/1535957601).

[brandon-rhodes]

Thank you for doing all of this research! I'm glad that the existing pkgutil-style recommendation in the docs, if somewhat old-fashioned for folks on Python 3, at least does still interoperate with the new PEP 420. And thanks very much for the extra investigation that revealed the problems that pytest looks likely to face with the new approach to namespace packages.

I will invite you to become a contributor to this repository, so that you can make progress against the new issues you’ve opened. Once you’re ready to release (it's fine if that's not for a while — if we're not ready for PEP 420 yet, maybe things stay as they are on PyPI?), I can head over to PyPI and try to figure out what to press to give you permission to release the package yourself for the first time!

We need to catch up sometime! Hopefully we'll see each other at a conference sometime next year.