psutil fails to compile against MUSL C library #872

yegorich · 2016-08-18T10:37:26Z

Following error occurs, when compiling psutil against MUSL. Any idea?

/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DPSUTIL_VERSION=431 -I/home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
In file included from /home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/kernel.h:4:0,
                 from /home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/ethtool.h:16,
                 from psutil/_psutil_linux.c:35:
/home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/sysinfo.h:7:8: error: redefinition of ‘struct sysinfo’
 struct sysinfo {
        ^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:10:8: note: originally defined here
 struct sysinfo {
        ^
psutil/_psutil_linux.c: In function ‘psutil_linux_sysinfo’:
psutil/_psutil_linux.c:244:17: warning: passing argument 1 of ‘sysinfo’ from incompatible pointer type
     if (sysinfo(&info) != 0)
                 ^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:26:5: note: expected ‘struct sysinfo *’ but argument is of type ‘struct sysinfo *’
 int sysinfo (struct sysinfo *);
     ^
psutil/_psutil_linux.c: In function ‘init_psutil_linux’:
psutil/_psutil_linux.c:644:15: warning: unused variable ‘v’ [-Wunused-variable]
     PyObject *v;
               ^
error: command '/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc' failed with exit status 1

The text was updated successfully, but these errors were encountered:

giampaolo · 2016-08-18T21:13:17Z

What's MUSL?

On Aug 18, 2016 12:37 PM, "Yegor Yefremov" [email protected] wrote:

Following error occurs, when compiling psutil against MUSL. Any idea?

/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc
-fno-strict-aliasing -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
-D_FILE_OFFSET_BITS=64 -Os -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -fPIC -DPSUTIL_VERSION=431 -I/home/user/Documents/
versioned/buildroot/bsp/host/usr/x86_64-buildroot-linux-
musl/sysroot/usr/include/python2.7 -c psutil/_psutil_linux.c -o
build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
In file included from /home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/kernel.h:4:0,
from /home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/ethtool.h:16,
from psutil/_psutil_linux.c:35:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/linux/sysinfo.h:7:8:
error: redefinition of ‘struct sysinfo’
struct sysinfo {
^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:10:8:
note: originally defined here
struct sysinfo {
^
psutil/_psutil_linux.c: In function ‘psutil_linux_sysinfo’:
psutil/_psutil_linux.c:244:17: warning: passing argument 1 of ‘sysinfo’
from incompatible pointer type
if (sysinfo(&info) != 0)
^
In file included from psutil/_psutil_linux.c:21:0:
/home/user/Documents/versioned/buildroot/bsp/host/
usr/x86_64-buildroot-linux-musl/sysroot/usr/include/sys/sysinfo.h:26:5:
note: expected ‘struct sysinfo *’ but argument is of type ‘struct sysinfo *’
int sysinfo (struct sysinfo *);
^
psutil/_psutil_linux.c: In function ‘init_psutil_linux’:
psutil/_psutil_linux.c:644:15: warning: unused variable ‘v’
[-Wunused-variable]
PyObject *v;
^
error: command '/home/user/Documents/versioned/buildroot/bsp/host/usr/bin/x86_64-linux-gcc'
failed with exit status 1

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#872, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAplLFXcm6IbMvfDbxQf8-erdd6d3GToks5qhDXngaJpZM4JnW6d
.

yegorich · 2016-08-19T05:27:26Z

@giampaolo MUSL is a new C library, that tries to strictly follow the standards (POSIX etc.). These rules often lead to compilation issues, when non-stadard headers will be included etc. MUSL is for example quasi standard C library for OpenWrt/LEDE projects.

In the case of psutil, there is glibc leftover issue with struct sysinfo. It is defined in linux/sysinfo.h and in sys/sysinfo.h. OpenWrt project provides following patch for the kernel header, but this is not applicable for psutil of course. If I remove #include <sys/sysinfo.h> from psutil/_psutil_linux.c I can compile the package, but I also get following warning:

psutil/_psutil_linux.c: In function ‘psutil_linux_sysinfo’:
psutil/_psutil_linux.c:244:5: warning: implicit declaration of function ‘sysinfo’ [-Wimplicit-function-declaration]
if (sysinfo(&info) != 0)

So I don't know, if it is a proper solution.

ConorIA · 2016-10-26T01:31:47Z

Here's my output on Alpine Linux, using MUSL

# pip install psutil
Collecting psutil
  Using cached psutil-4.4.1.tar.gz
Installing collected packages: psutil
  Running setup.py install for psutil ... error
    Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-s1iYoK/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-d3266X-record/install-record.txt --single-version-externally-managed --compile:
    /tmp/tmpp7WhDB.c:1:27: fatal error: linux/ethtool.h: No such file or directory
     #include <linux/ethtool.h>
                               ^
    compilation terminated.
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psposix.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psosx.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/__init__.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pssunos.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pswindows.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_psbsd.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_pslinux.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_compat.py -> build/lib.linux-x86_64-2.7/psutil
    copying psutil/_common.py -> build/lib.linux-x86_64-2.7/psutil
    creating build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_memory_leaks.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/__init__.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/runner.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_system.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-2.7/psutil/tests
    copying psutil/tests/test_process.py -> build/lib.linux-x86_64-2.7/psutil/tests
    running build_ext
    building 'psutil._psutil_linux' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/psutil
    gcc -fno-strict-aliasing -Os -fomit-frame-pointer -g -DNDEBUG -Os -fomit-frame-pointer -g -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_VERSION=441 -DPSUTIL_LINUX=1 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -I/usr/include/python2.7 -c psutil/_psutil_linux.c -o build/temp.linux-x86_64-2.7/psutil/_psutil_linux.o
    psutil/_psutil_linux.c:19:27: fatal error: linux/version.h: No such file or directory
     #include <linux/version.h>
                               ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-s1iYoK/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-d3266X-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-s1iYoK/psutil/

DamionWaltermeyer · 2017-01-11T18:46:47Z

bump getting the same issue with Alpine linux:3.4.

giampaolo · 2017-01-11T19:38:02Z

Please provide instructions on how to build/compile psutil by using MUSL on Linux (I'm on Ubuntu).

DamionWaltermeyer · 2017-01-11T20:37:17Z

I have a solution to the problem for Alpine linux. I can't be more specific with the MUSL portion outside of that.
apk add linux-headers
pip install psutil

i'd already entered apk add --upgrade py-pip and psuti, so they may be needed as well.

yegorich · 2017-01-12T08:44:01Z

The easiest way to reproduce the issue would be to use Buildroot:

git clone git://git.buildroot.net/buildroot
cd buildroot
git checkout d8a19c5e4969a5d6485d3704cc72310989109150
vi local.mk (it should have following content: PYTHON_PSUTIL_OVERRIDE_SRCDIR=/home/user/Documents/versioned/psutil, just provide the path to your psutil clone)
create .config with the content shown below
invoke make

BR2_x86_i686=y
BR2_TOOLCHAIN_BUILDROOT_MUSL=y
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON_PSUTIL=y

giampaolo · 2017-01-12T18:54:35Z

How about this?
http://askubuntu.com/a/391037
It appears you may need linux-libc-dev and / or linux-headers packages installed.
Those instructions are for ubuntu, not sure what's the corresponding package name on alpine linux.

DamionWaltermeyer · 2017-01-12T19:01:15Z

That's what I mention above. Installing linux-headers in alpine fixed it for me.

giampaolo · 2017-01-12T19:02:52Z

Oh sorry, I missed that. Closing this out then.

yegorich · 2017-01-13T08:20:12Z

@giampaolo have you tried to build psutil in Buildroot following the steps I've provided? The issue with musl is still there.

@DamionWaltermeyer and @ConorIA had unrelated issue, because kernel headers were missing.

Suppress inclusion of linux/sysinfo.h to fix redefinition of struct sysinfo that musl libc defines in sys/sysinfo.h, which least to the following build failure (paths abbreviated): In file included from .../usr/include/linux/kernel.h:4:0, from .../usr/include/linux/ethtool.h:16, from psutil/_psutil_linux.c:35: .../usr/include/linux/sysinfo.h:7:8: error: redefinition of ‘struct sysinfo’ struct sysinfo { ^ In file included from psutil/_psutil_linux.c:21:0: .../usr/include/sys/sysinfo.h:10:8: note: originally defined here struct sysinfo { ^ Fixes giampaolo#872

This is required to build psutil. Without this patch, I get: psutil/_psutil_posix.c:29:29: fatal error: linux/types.h: No such file or directory After this patch, 'docker build' completes successfully. See also: giampaolo/psutil#872

vinzjaystone · 2023-09-15T07:50:34Z

I have a solution to the problem for Alpine linux. I can't be more specific with the MUSL portion outside of that. apk add linux-headers pip install psutil

i'd already entered apk add --upgrade py-pip and psuti, so they may be needed as well.

Thank you so much for sharing.
This works on my end.

giampaolo closed this as completed Jan 12, 2017

baruchsiach mentioned this issue Feb 16, 2017

Fix build with musl libc #979

Merged

njsmith mentioned this issue Jul 20, 2018

Install linux-headers in dockerfile vmprof/vmprof-server#36

Merged

adewaleo mentioned this issue Apr 2, 2019

[wip] az feedback Azure/azure-cli#8829

Merged

2 tasks

pangzineng mentioned this issue Dec 18, 2019

fail to install dependency in requirements.txt alvations/cliffjumper#1

Closed

giampaolo added the bug label Nov 18, 2020

kdave mentioned this issue Jan 10, 2022

common/utils.c: drop include on sys/sysinfo.h kdave/btrfs-progs#433

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

psutil fails to compile against MUSL C library #872

psutil fails to compile against MUSL C library #872

yegorich commented Aug 18, 2016 •

edited by giampaolo

Loading

giampaolo commented Aug 18, 2016

yegorich commented Aug 19, 2016 •

edited

Loading

ConorIA commented Oct 26, 2016

DamionWaltermeyer commented Jan 11, 2017

giampaolo commented Jan 11, 2017

DamionWaltermeyer commented Jan 11, 2017

yegorich commented Jan 12, 2017 •

edited

Loading

giampaolo commented Jan 12, 2017

DamionWaltermeyer commented Jan 12, 2017

giampaolo commented Jan 12, 2017

yegorich commented Jan 13, 2017

vinzjaystone commented Sep 15, 2023

psutil fails to compile against MUSL C library #872

psutil fails to compile against MUSL C library #872

Comments

yegorich commented Aug 18, 2016 • edited by giampaolo Loading

giampaolo commented Aug 18, 2016

yegorich commented Aug 19, 2016 • edited Loading

ConorIA commented Oct 26, 2016

DamionWaltermeyer commented Jan 11, 2017

giampaolo commented Jan 11, 2017

DamionWaltermeyer commented Jan 11, 2017

yegorich commented Jan 12, 2017 • edited Loading

giampaolo commented Jan 12, 2017

DamionWaltermeyer commented Jan 12, 2017

giampaolo commented Jan 12, 2017

yegorich commented Jan 13, 2017

vinzjaystone commented Sep 15, 2023

yegorich commented Aug 18, 2016 •

edited by giampaolo

Loading

yegorich commented Aug 19, 2016 •

edited

Loading

yegorich commented Jan 12, 2017 •

edited

Loading