python3: Circular dependency between python3-email and python3-urllib

[jefferyto]

Maintainer: me, @commodo
Environment: N/A

Description:
In the Python 3 modules, urllib imports email and email imports from urllib.

In our packages, python3-urllib depends on python3-email (#8307) but python3-email does not depend on python3-urllib.

If python3-email has a dependency on python3-urllib, then make will complain about the circular dependency, but if python3-email does not depend on python3-urllib, then installing python3-email only and importing the email module will result in a run-time error. (I found this when testing for #12941.)

Not sure how this should be addressed ☹️

[jefferyto]

I ran the stdlib dependency script on Python itself - I haven't gone through the results in detail yet but it looks like there may be a number of imports that are not captured as package dependencies (not sure if there are any other circular dependencies).

[commodo]

Not sure how this should be addressed

One idea would be to merge the 2 packages into a single one.
Maybe python3-urllib for the final merged package? Or some other name?

I ran the stdlib dependency script on Python itself - I haven't gone through the results in detail yet but it looks like there may be a number of imports that are not captured as package dependencies (not sure if there are any other circular dependencies).

I'll admit that when I split the packages into small chunks, I did not take into account these imports much.
It was more empirical. If some package [usually a folder] has a few hundred of kilobytes, it would become it's own package.
I was trying to figure out a good-enough mechanism for reducing the Python/Python3 minimum-required-to-start size [python[3]-base, which was different for Python & Python3], then python[3]-light size, then something to make combine python[3]-light + all the splits back into a "full" Python/Python3 package [the files/python[3]-package-xxxx.mk auto-includes].
Size was more crucial than import sanity.

In the back of my head I always knew this is far from perfect, but if I was going to try to make it perfect, it would take too long.
After ~5 years, it doesn't look like people needed more than what's currently being done.
AFAICT,some people just use python3 as a dependency, others try to go from python3-base / python3-light.
Both are equally fine from my side, I guess.

I still feel, that putting CPython with all libs in places with small flash-sizes is weird, but if people want to do that, and this package split helps, then sure.

[jefferyto]

One idea would be to merge the 2 packages into a single one.
Maybe python3-urllib for the final merged package? Or some other name?

Merging would be fine but the two modules don't seem to be related, e.g. from the name python3-urllib I wouldn't guess it also has the email module.

After ~5 years, it doesn't look like people needed more than what's currently being done.
AFAICT,some people just use python3 as a dependency, others try to go from python3-base / python3-light.
Both are equally fine from my side, I guess.

I think for packagers, those who don't care about install size will just specify python3. It takes a lot of work to figure out what stdlib dependencies are necessary, that's why I wrote the script.

For users, I think if/when they see a module missing error, they just install the appropriate package and move on. I don't think many of them will report an issue or open a PR to fix the dependency.

---

I'll go through the script results in more detail later (may take a few days); we should have a better understand after and can then decide what to do (if anything).

[commodo]

@jefferyto

any more thoughts on this?

[jefferyto]

The problem is larger than just email and urllib: #20462 (comment)

This isn't your responsibility anymore, you can unsubscribe from the issue if you like.