Incompatible with Python 3.12 #104

mweinelt · 2023-12-04T16:13:54Z

We are moving towards Python 3.12 over in nixpkgs, and noticed this package relies somewhat heavily on distutils, which has been deprecated since 3.10, see https://peps.python.org/pep-0632/.

Now I see that master recently started testing against 3.12, but I don't understand how that succeeds. For me, it fails like this on 4.2:

 ======================================================================
 ERROR: pyroma (unittest.loader._FailedTest.pyroma)
 ----------------------------------------------------------------------
 ImportError: Failed to import test module: pyroma
 Traceback (most recent call last):
   File "/nix/store/ly93xq4pckid6ykivk07ln04bgjf41hv-python3-3.12.0/lib/python3.12/unittest/loader.py", line 129, in loadTestsFromName
     module = __import__(module_name)
              ^^^^^^^^^^^^^^^^^^^^^^^
   File "/build/source/pyroma/__init__.py", line 4, in <module>
     from pyroma import projectdata, distributiondata, pypidata, ratings
   File "/build/source/pyroma/projectdata.py", line 8, in <module>
     from distutils import core
 ModuleNotFoundError: No module named 'distutils'

The text was updated successfully, but these errors were encountered:

hosiet · 2024-12-21T16:40:09Z

Ping. Is there any progress after one year?

hugovk · 2024-12-21T16:57:53Z

Nothing to report, I think we need a way to reproduce it to begin with.

This is what I get on macOS. There is an AttributeError: module 'distutils.util' has no attribute 'run_2to3' but the tests still pass:

❯ python3.12 -m pip install -e ".[test]"
...
❯ python3.12 -bb -X dev -W ignore::UserWarning:setuptools.dist -m unittest -v pyroma.tests
/private/tmp/pyroma/pyroma/tests.py:6: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import resource_filename, resource_string
test_complete (pyroma.tests.DistroDataTest.test_complete) ... /private/tmp/pyroma/pyroma/distributiondata.py:23: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
  tar.extractall(path, members, numeric_owner=numeric_owner)
Getting metadata for wheel...
Getting metadata for wheel...
Getting metadata for wheel...
Getting metadata for wheel...
ok
test_complete (pyroma.tests.ProjectDataTest.test_complete) ... Getting metadata for wheel...
ok
test_complete (pyroma.tests.PyPITest.test_complete) ... Found complete version 1.1.0
Downloading xmldiff-1.1.0.tar.gz to verify distribution
Starting new HTTPS connection (1): files.pythonhosted.org:443
https://files.pythonhosted.org:443 "GET /packages/ab/4a/cab33ec593f433b7cded3cec85d1f7c900261fd503bd894e16782f7c0d09/xmldiff-1.1.0.tar.gz HTTP/11" 200 39258
Getting metadata for wheel...
ok
test_distribute (pyroma.tests.PyPITest.test_distribute) ... Found distribute version 0.6.15
Downloading distribute-0.6.15.tar.gz to verify distribution
Starting new HTTPS connection (1): files.pythonhosted.org:443
https://files.pythonhosted.org:443 "GET /packages/07/a6/8f6b270b7e6a1be3df78b174e2437fdb631cfcce16f82c2f8b7affb50ee2/distribute-0.6.15.tar.gz HTTP/11" 200 397027
/private/tmp/pyroma/pyroma/distributiondata.py:23: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
  tar.extractall(path, members, numeric_owner=numeric_owner)
Getting metadata for wheel...
Creating isolated environment: venv+pip...
Installing packages in isolated environment:
- setuptools >= 40.8.0
Getting build dependencies for wheel...
Exception raised during metadata preparation
Traceback (most recent call last):
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 81, in _get_data
    return get_build_data(path)
           ^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 62, in get_build_data
    metadata = build_metadata(path, isolated=isolated)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 38, in build_metadata
    return build.util.project_wheel_metadata(path, isolated=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/util.py", line 49, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 225, in get_requires_for_build
    with self._handle_backend(hook_name):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 353, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel
<string>:13: ResourceWarning: unclosed file <_io.TextIOWrapper name='MANIFEST.in' mode='r' encoding='UTF-8'>
ResourceWarning: Enable tracemalloc to get the object allocation traceback
Exception when running setup.
Traceback (most recent call last):
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 81, in _get_data
    return get_build_data(path)
           ^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 62, in get_build_data
    metadata = build_metadata(path, isolated=isolated)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 38, in build_metadata
    return build.util.project_wheel_metadata(path, isolated=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/util.py", line 49, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 225, in get_requires_for_build
    with self._handle_backend(hook_name):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 353, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 216, in run_setup
    exec(f.read(), glocals, glocals)
  File "<string>", line 27, in <module>
AttributeError: module 'distutils.util' has no attribute 'run_2to3'
'distutils.core.setup()' was never called -- perhaps 'setup.py' is not a Distutils setup script?
Traceback (most recent call last):
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 81, in _get_data
    return get_build_data(path)
           ^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 62, in get_build_data
    metadata = build_metadata(path, isolated=isolated)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 38, in build_metadata
    return build.util.project_wheel_metadata(path, isolated=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/util.py", line 49, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 225, in get_requires_for_build
    with self._handle_backend(hook_name):
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/build/_builder.py", line 353, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 244, in get_setuppy_data
    distro = run_setup("setup.py", stop_after="config")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/pyroma/pyroma/projectdata.py", line 224, in run_setup
    raise RuntimeError(
RuntimeError: 'distutils.core.setup()' was never called -- perhaps 'setup.py' is not a Distutils setup script?
ok
test_complete (pyroma.tests.RatingsTest.test_complete) ... Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_custom_test (pyroma.tests.RatingsTest.test_custom_test) ... Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_lacking (pyroma.tests.RatingsTest.test_lacking) ... Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_markdown (pyroma.tests.RatingsTest.test_markdown) ... ok
test_minimal (pyroma.tests.RatingsTest.test_minimal) ... Getting metadata for wheel...
lists of files in version control and sdist do not match!
missing from sdist:
  randomfile.txt
no MANIFEST.in found; you can run 'check-manifest -c' to create one
suggested MANIFEST.in rules:
  include *.txt
ok
test_only_config (pyroma.tests.RatingsTest.test_only_config) ... ok
test_pep517 (pyroma.tests.RatingsTest.test_pep517) ... Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_pep621 (pyroma.tests.RatingsTest.test_pep621) ... Getting metadata for wheel...
Creating isolated environment: venv+pip...
Installing packages in isolated environment:
- flit_core >=3.4,<4
Getting build dependencies for wheel...
Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_setup_config (pyroma.tests.RatingsTest.test_setup_config) ... Getting metadata for wheel...
lists of files in version control and sdist match
ok
test_skip_tests (pyroma.tests.RatingsTest.test_skip_tests) ... ok

----------------------------------------------------------------------
Ran 14 tests in 7.644s

OK

What do you get? What OS? What commands did you run?

mweinelt · 2024-12-25T21:24:28Z

Linux (NixOS unstable). We encounter this when running pytest or upon importing the module.

  File "/build/source/pyroma/__init__.py", line 5, in <module>
    from pyroma import projectdata, distributiondata, pypidata, ratings
  File "/build/source/pyroma/projectdata.py", line 10, in <module>
    from distutils import core
ModuleNotFoundError: No module named 'distutils'

https://github.com/regebro/pyroma/blob/master/pyroma/projectdata.py#L10

hugovk · 2024-12-27T20:21:00Z

How are you installing? Does setuptools get installed?

mweinelt · 2024-12-27T20:44:57Z

With python -m build and python -m installer. And yes, we have setuptools in the PYTHONPATH, but version 75.3.0.

hosiet · 2024-12-31T03:59:07Z

The logic is simple: For Python 3.12+, the distutils module should be considered gone, as seen from https://peps.python.org/pep-0632/ . Any lines like import distutils should be ultimately removed in favor of functionalities provided by setuptools.

A stopgap method is available as setuptools._distutils as a replacement for distutils as can be found in https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html . But still this needs a new version of setuptools, and the embedded copy itself is not a long-term solution. Usage of setuptools._distutils should be avoided.

hugovk · 2024-12-31T06:19:24Z

PRs welcome for the stopgap and long-term fixes.

regebro · 2024-12-31T12:42:52Z

One main question is if we should try to keep support for setup.py at all, or just drop it now.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Incompatible with Python 3.12 #104

Incompatible with Python 3.12 #104

mweinelt commented Dec 4, 2023 •

edited by hugovk

Loading

hosiet commented Dec 21, 2024

hugovk commented Dec 21, 2024 •

edited

Loading

mweinelt commented Dec 25, 2024 •

edited by hugovk

Loading

hugovk commented Dec 27, 2024

mweinelt commented Dec 27, 2024

hosiet commented Dec 31, 2024

hugovk commented Dec 31, 2024

regebro commented Dec 31, 2024

Incompatible with Python 3.12 #104

Incompatible with Python 3.12 #104

Comments

mweinelt commented Dec 4, 2023 • edited by hugovk Loading

hosiet commented Dec 21, 2024

hugovk commented Dec 21, 2024 • edited Loading

mweinelt commented Dec 25, 2024 • edited by hugovk Loading

hugovk commented Dec 27, 2024

mweinelt commented Dec 27, 2024

hosiet commented Dec 31, 2024

hugovk commented Dec 31, 2024

regebro commented Dec 31, 2024

mweinelt commented Dec 4, 2023 •

edited by hugovk

Loading

hugovk commented Dec 21, 2024 •

edited

Loading

mweinelt commented Dec 25, 2024 •

edited by hugovk

Loading