meson: build fails with python 3.12 #253751

davidak · 2023-09-07T00:44:48Z

Describe the bug

error: builder for '/nix/store/mk0py0cfw91phxhihpfgana0pjmqhzpw-meson-1.2.0.drv' failed with exit code 8;

See log

...
Executing subproject bar 

Project name: bar
Project version: undefined
C++ compiler for the host machine: g++ (gcc 12.3.0 "g++ (GCC) 12.3.0")
C++ linker for the host machine: g++ ld.bfd 2.40
C++ compiler for the build machine: g++ (gcc 12.3.0 "g++ (GCC) 12.3.0")
C++ linker for the build machine: g++ ld.bfd 2.40
Could not introspect Python (['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python3', '/build/source/mesonbuild/scripts/python_info.py']): exit code 1
Program stdout:


Program stderr:

Traceback (most recent call last):
File "/build/source/mesonbuild/scripts/python_info.py", line 16, in <module>
  import distutils.command.install
ModuleNotFoundError: No module named 'distutils'


test cases/python/6 failing subproject/subprojects/bar/meson.build:3:26: Exception: <PythonExternalProgram 'python3' -> ['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python3']> is not a valid python or it is missing distutils

Subproject subprojects/bar is buildable: NO (disabling)
Dependency bar from subproject bar found: NO (subproject failed to configure)
Could not introspect Python (['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python3', '/build/source/mesonbuild/scripts/python_info.py']): exit code 1
Program stdout:


Program stderr:

Traceback (most recent call last):
File "/build/source/mesonbuild/scripts/python_info.py", line 16, in <module>
  import distutils.command.install
ModuleNotFoundError: No module named 'distutils'


test cases/python/6 failing subproject/meson.build:5:26: ERROR: <PythonExternalProgram 'python3' -> ['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python3']> is not a valid python or it is missing distutils


Traceback (most recent call last):
File "/build/source/run_tests.py", line 315, in run_configure_inprocess
  returncode = mesonmain.run(commandlist, get_meson_script())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 285, in run
  return CommandLineParser().run(args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 196, in run
  return errorhandler(e, command)
         ^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 40, in errorhandler
  raise e
File "/build/source/mesonbuild/mesonmain.py", line 194, in run
  return options.run_func(options)
         ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/msetup.py", line 358, in run
  app.generate()
File "/build/source/mesonbuild/msetup.py", line 183, in generate
  return self._generate(env, capture, vslite_ctx)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/msetup.py", line 228, in _generate
  intr.run()
File "/build/source/mesonbuild/interpreter/interpreter.py", line 2997, in run
  super().run()
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 164, in run
  self.evaluate_codeblock(self.ast, start=1)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
  raise e
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
  self.evaluate_statement(cur)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_statement
  self.assignment(cur)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 635, in assignment
  value = self.evaluate_statement(node.value)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
  return self.method_call(cur)
         ^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 535, in method_call
  obj = self.evaluate_statement(invocable)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
  return self.method_call(cur)
         ^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 550, in method_call
  res = obj.method_call(method_name, args, kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreter/interpreterobjects.py", line 814, in method_call
  ret = method(state, args, kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 109, in wrapped
  ret = f(*wrapped_args, **wrapped_kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
  return f(*nargs, **wrapped_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 596, in wrapper
  return f(*wrapped_args, **wrapped_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/modules/python.py", line 437, in find_installation
  python = self._find_installation_impl(state, display_name, name_or_path, required)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/modules/python.py", line 398, in _find_installation_impl
  raise mesonlib.MesonException(sanitymsg)
mesonbuild.utils.core.MesonException: <PythonExternalProgram 'python3' -> ['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python3']> is not a valid python or it is missing distutils



====================================== python: 7 install path ======================================


(inprocess) $ setup --prefix /usr --libdir lib 'test cases/python/7 install path' '/build/source/b 87a5fe311e' --backend=ninja
Build started at 2023-09-07T00:26:10.979266
Main binary: /nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python
Build Options: -Dprefix=/usr -Dlibdir=lib -Dbackend=ninja
Python system: Linux
The Meson build system
Version: 1.2.0
Source dir: /build/source/test cases/python/7 install path
Build dir: /build/source/b 87a5fe311e
Build type: native build
Project name: install path
Project version: undefined
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Could not introspect Python (['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python', '/build/source/mesonbuild/scripts/python_info.py']): exit code 1
Program stdout:


Program stderr:

Traceback (most recent call last):
File "/build/source/mesonbuild/scripts/python_info.py", line 16, in <module>
  import distutils.command.install
ModuleNotFoundError: No module named 'distutils'


test cases/python/7 install path/meson.build:9:22: ERROR: <PythonExternalProgram 'python3' -> ['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python']> is not a valid python or it is missing distutils


Traceback (most recent call last):
File "/build/source/run_tests.py", line 315, in run_configure_inprocess
  returncode = mesonmain.run(commandlist, get_meson_script())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 285, in run
  return CommandLineParser().run(args)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 196, in run
  return errorhandler(e, command)
         ^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/mesonmain.py", line 40, in errorhandler
  raise e
File "/build/source/mesonbuild/mesonmain.py", line 194, in run
  return options.run_func(options)
         ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/msetup.py", line 358, in run
  app.generate()
File "/build/source/mesonbuild/msetup.py", line 183, in generate
  return self._generate(env, capture, vslite_ctx)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/msetup.py", line 228, in _generate
  intr.run()
File "/build/source/mesonbuild/interpreter/interpreter.py", line 2997, in run
  super().run()
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 164, in run
  self.evaluate_codeblock(self.ast, start=1)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 190, in evaluate_codeblock
  raise e
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 182, in evaluate_codeblock
  self.evaluate_statement(cur)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 198, in evaluate_statement
  self.assignment(cur)
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 635, in assignment
  value = self.evaluate_statement(node.value)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 200, in evaluate_statement
  return self.method_call(cur)
         ^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/interpreterbase.py", line 550, in method_call
  res = obj.method_call(method_name, args, kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreter/interpreterobjects.py", line 814, in method_call
  ret = method(state, args, kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 109, in wrapped
  ret = f(*wrapped_args, **wrapped_kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 277, in wrapper
  return f(*nargs, **wrapped_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/interpreterbase/decorators.py", line 596, in wrapper
  return f(*wrapped_args, **wrapped_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/modules/python.py", line 437, in find_installation
  python = self._find_installation_impl(state, display_name, name_or_path, required)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/build/source/mesonbuild/modules/python.py", line 398, in _find_installation_impl
  raise mesonlib.MesonException(sanitymsg)
mesonbuild.utils.core.MesonException: <PythonExternalProgram 'python3' -> ['/nix/store/q5fi2vmnii9vjpqynp56vp0734yhpaig-python3-3.12.0b4/bin/python']> is not a valid python or it is missing distutils

Steps To Reproduce

Steps to reproduce the behavior:

nix-shell -I nixpkgs=./ -p python312Packages.meson

Expected behavior

build successful

Additional context

the python 3.12 version of meson is not shown in the search. maybe this is known not to work. i don't see it marked as broken

https://search.nixos.org/packages?channel=unstable&show=python311Packages.meson&from=0&size=50&sort=relevance&type=packages&query=meson

Notify maintainers

@AndersonTorres

Metadata

nixpkgs @ 48992ab

system: "x86_64-linux"
host os: Linux 6.3.8, NixOS, 22.11 (Raccoon), 22.11.4588.93fddcf640c
multi-user?: yes
sandbox: yes
version: nix-env (Nix) 2.11.1
channels(root): "home-manager-22.11.tar.gz, nixos-22.11, nixos-hardware, nixos-unstable"
nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos

The text was updated successfully, but these errors were encountered:

ConnorBaker · 2023-09-28T17:26:45Z

Maybe related: https://docs.python.org/3.12/whatsnew/3.12.html#distutils

EDIT:

Meson tracking issue: PEP 632: Proposed deprectation of distutils mesonbuild/meson#7702
Meson PR: python module: migrate away from the use of distutils mesonbuild/meson#11133 (comment)

dnicolodi · 2023-09-29T14:14:50Z

The released meson version depends on distutils which has been removed from Python 3.12. The next meson version will remove the dependency on distutils. setuptools embeds a copy of distutils, thus, until the next meson version is released, you may want to install setuptools on Python 3.12 to get the distutils module.

haampie · 2023-10-02T16:02:48Z

Installing setuptools is not enough, you have to import it too to let setuptools mess with sys.path s.t. import disutils works.

So, meson should import setuptools before importing disutils if distutils fails to import directly, and meson doesn't do this

dnicolodi · 2023-10-02T16:19:13Z

No, it is not necessary to import setuptools. setuptools have some magic (details provided on request) to inject its distutils copy in the Python path. The solution I proposed is tested. Do you have any evidence that it does not work?

haampie · 2023-10-02T18:27:34Z

I'm not an expert on how nix deals with python package installs, but the setuptools hack where sys.path is modified through

<setuptools prefix>/lib/pythonX.Y/site-packages/distutils-precedence.pth

at startup only works when setuptools prefix == python prefix. That is, setuptools must be installed as a system site package.

That assumption may not hold in nix, it certainly does not in the Spack package manager which like nix also puts python packages in their own prefixes, and lets python locate those through PYTHONPATH.

So, if PYTHONPATH is all you have, you need to import setuptools to "activate" the same hack, s.t. import distutils work.

AndersonTorres · 2023-10-03T23:08:36Z

Should we wait the next release that fixes this?

dwilliam62 · 2024-07-13T06:46:54Z

Another report of this error with more details. Problem is with meson It has distuils as a dependency
That has been removed from Python v3.12 (nice) Have to wait for updated meson or add 'setuputils' in python.

#253751

Fix copied from: https://github.com/NixOS/nixpkgs/pull/325909/files Related: NixOS/nixpkgs#325909 and NixOS/nixpkgs#253751

jwillikers · 2024-11-19T12:36:14Z

This has been fixed, right?

davidak added the 0.kind: bug Something is broken label Sep 7, 2023

D3vil0p3r mentioned this issue May 1, 2024

Build failure: python312packages.bsddb3 #308232

Closed

LuNeder mentioned this issue Jul 15, 2024

Build failure: emojione #326959

Closed

LuNeder added a commit to LuNeder/compiz-reloaded-nix that referenced this issue Jul 15, 2024

Tries to fix, again

a2d6187

Fix copied from: https://github.com/NixOS/nixpkgs/pull/325909/files Related: NixOS/nixpkgs#325909 and NixOS/nixpkgs#253751

CertainLach mentioned this issue Sep 13, 2024

Qubes packages (Its alive!) #341215

Draft

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

meson: build fails with python 3.12 #253751

meson: build fails with python 3.12 #253751

davidak commented Sep 7, 2023

ConnorBaker commented Sep 28, 2023 •

edited

Loading

dnicolodi commented Sep 29, 2023

haampie commented Oct 2, 2023

dnicolodi commented Oct 2, 2023

haampie commented Oct 2, 2023 •

edited

Loading

AndersonTorres commented Oct 3, 2023

dwilliam62 commented Jul 13, 2024 •

edited

Loading

jwillikers commented Nov 19, 2024

meson: build fails with python 3.12 #253751

meson: build fails with python 3.12 #253751

Comments

davidak commented Sep 7, 2023

Describe the bug

Steps To Reproduce

Expected behavior

Additional context

Notify maintainers

Metadata

ConnorBaker commented Sep 28, 2023 • edited Loading

dnicolodi commented Sep 29, 2023

haampie commented Oct 2, 2023

dnicolodi commented Oct 2, 2023

haampie commented Oct 2, 2023 • edited Loading

AndersonTorres commented Oct 3, 2023

dwilliam62 commented Jul 13, 2024 • edited Loading

jwillikers commented Nov 19, 2024

ConnorBaker commented Sep 28, 2023 •

edited

Loading

haampie commented Oct 2, 2023 •

edited

Loading

dwilliam62 commented Jul 13, 2024 •

edited

Loading