pydarshan can't load libdarshan-util-*.so from pypi on Cori&Perlmutter (NERSC/LBL) #656
Comments
|
So, the base reason is that this code is directly opening the libdarshan-util.so library but it depends on the libz. This library is not loaded directly, thus it relies on the loader to find it, either with LD_LIBRARY_PATH, ld.so.conf or PYTHONPATH or some other means. |
|
@bebosudo Can you provide an ldd of the libdarshan-util-0ca4caca.so file? |
|
Hi @kevin-harms, here it's a ldd on both .so files from pypi: |
|
Let me check if a SciPy wheel that claims to be
I can reproduce the issue locally It looks like vendoring And it appears that later versions of the standard do allow linking instead of vendoring perhaps: pypa/auditwheel#334 See also i.e., : pypa/auditwheel#161 If I try to repair our wheel: But this seems to work:
So, the Some conclusions might be:
|
|
So do we need to link libdarshan-util with -rpath so that it can be repaired during installation? Is there some general support in pypi deployment to handle loading of non-system library dependencies? |
|
Normally you should vendor non-system libs, which we already do for When I look at what other libs do for their shared objects that depend on vendored libs, for example SciPy points shared objects at the
If we look at the result of my detailed
( While the original version we ship without
Produces no So, to me the conclusion is the same--we should follow standard/formal Python wheel build procedures more closely and Do other Python libraries do this? Yes--here is the And, yes, |
|
Thanks to @jakobluettgau for digging into this in #665. We've updated our wheels for PyDarshan to be based on a newer manylinux version that seems to prefer system libz. He and I have both tested and things appear to be working with these changes. @bebosudo, would you mind trying out the new updates to ensure we've got this cleaned up. We temporarily put the release on test.pypi so you can test out before we release a new version with the changes. You should be able to pip install as follows:
Then maybe just make sure you aren't seeing anything bomb when importing darshan? |
|
I confirm that build 3.3.1.1 from the test pypi repo works: $ python3 -m ensurepip --user
$ python3 -m pip install -U --user pip
$ python3 -m pip -V
pip 21.3.1 from /global/homes/c/chiusole/.local/lib/python3.6/site-packages/pip (python 3.6)
$ python3 -m pip install darshan==3.3.1.0
$ python3 -m pip install -i https://test.pypi.org/simple/ darshan==3.3.1.1
$ python3
Python 3.6.12 (default, Nov 25 2020, 20:33:10) [GCC] on linux
>>> import darshan
>>> report = darshan.DarshanReport('/path/to/just-a-log-file.darshan', read_all=False)
>>> report.read_all_generic_records()
>>> darshan.enable_experimental()
>>> report.summarize()
>>> report.summary
{'agg_ioops': {'STDIO': {'STDIO_OPENS': 1, 'STDIO_FDOPENS': 0, ...}}}
>>> ^D
$ ldd ~/.local/lib/python3.6/site-packages/darshan.libs/libdarshan-util-9e5e50e2.so.0.0.0
linux-vdso.so.1 (0x00002aaaaaad3000)
libz.so.1 => /lib64/libz.so.1 (0x00002aaaaacd3000)
libc.so.6 => /lib64/libc.so.6 (0x00002aaaaaeea000)
/lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)Does this mean that now pydarshan depends on libz being provided by the system? |
That sounds right, or rather that is the decision the Python packaging ecosystem appears to have made for the newer wheel standard the team is testing there. I believe the reason is that there's a tendency to ignore the vendored version of |
…ad-libdarshan-util Update PyDarshan wheel building process to manylinux2014 (fix for #656)
|
Just double checked to make sure this is still resolved for our latest 3.4.0.0 release of pydarshan, and all looks well: I'm going to close this for now, but please let us know if you have more issues. |
Installing pydarshan from pypi on Cori or Perlmutter causes pydarshan to fail at import time:
This can be solved by manually providing the path to the libdarshan-util-.so and libz-.so* library in
LD_LIBRARY_PATH:The problem seems to be around this block:
darshan/darshan-util/pydarshan/darshan/discover_darshan.py
Lines 187 to 200 in c95f1a3
By adding a breakpoint at line 195 I can see that ffi.dlopen is complaining about not finding
libz-eb09ad1d.so.1.2.3:Even though it's in the same directory of
libdarshan-util-0ca4caca.so:This is not related to conda: all commands above are executed using the system Python 3.6 on Cori, here are the steps:
On a different linux box, running Centos 7.3 and python 3.6, darshan from pypi works fine instead:
Cori and Perlmutter are both running SLES 15. Let me know if you need other details.
CC: @glennklockwood
The text was updated successfully, but these errors were encountered: