Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ImportError: dlopen: cannot load any more object with static TLS #277

Open
davidtong28 opened this issue Jul 19, 2023 · 5 comments
Open

ImportError: dlopen: cannot load any more object with static TLS #277

davidtong28 opened this issue Jul 19, 2023 · 5 comments
Labels
package Packaging of code

Comments

@davidtong28
Copy link

command:

poppunk --create-db --output database --r-files list.tsv --threads 8

output:

Traceback (most recent call last):
  File "/home/davidtong28/.conda/envs/poppunk_2.6/bin/poppunk", line 11, in <module>
    sys.exit(main())
  File "/home/davidtong28/.conda/envs/poppunk_2.6/lib/python3.10/site-packages/PopPUNK/__main__.py", line 231, in main
    from .models import loadClusterFit, BGMMFit, DBSCANFit, RefineFit, LineageFit
  File "/home/davidtong28/.conda/envs/poppunk_2.6/lib/python3.10/site-packages/PopPUNK/models.py", line 15, in <module>
    from sklearn import utils
  File "/home/davidtong28/.conda/envs/poppunk_2.6/lib/python3.10/site-packages/sklearn/__init__.py", line 84, in <module>
    from .utils._show_versions import show_versions
  File "/home/davidtong28/.conda/envs/poppunk_2.6/lib/python3.10/site-packages/sklearn/utils/_show_versions.py", line 13, in <module>
    from ._openmp_helpers import _openmp_parallelism_enabled
ImportError: dlopen: cannot load any more object with static TLS

Poppunk version: 2.6.0
@johnlees
Copy link
Member

This may be an interaction between our use of libgomp and sklearn's.
See scikit-learn/scikit-learn#14485 and pytorch/pytorch#2575 (comment)

Can you please confirm how you installed the package, include the output of conda list, and the version of pp-sketchlib (from running sketchlib).

@johnlees
Copy link
Member

Also whether you are running on OS X (and whether intel or M1) or linux

@davidtong28
Copy link
Author

Hi,
Thanks for reaching back.
I am using a linux system.
I installed everything via conda create -y -n poppunk_2.6 poppunk=2.6.0
here is the output of conda list

# packages in environment at /home/davidtong28/.conda/envs/poppunk_2.6:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aom                       3.5.0                h27087fc_0    conda-forge
at-spi2-atk               2.38.0               h0630a04_3    conda-forge
at-spi2-core              2.40.3               h0630a04_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
biopython                 1.81                     pypi_0    pypi
boost                     1.78.0          py310hc4a4660_4    conda-forge
boost-cpp                 1.78.0               h6582d0a_3    conda-forge
brotli                    1.0.9                    pypi_0    pypi
brotli-bin                1.0.9                h166bdaf_9    conda-forge
brotli-python             1.0.9           py310hd8f1fbe_9    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.1               hd590300_0    conda-forge
ca-certificates           2023.05.30           h06a4308_0
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.16.0            hbbf8b49_1016    conda-forge
cairomm-1.16              1.16.2               h7e731d7_1    conda-forge
certifi                   2023.5.7           pyhd8ed1ab_0    conda-forge
cffi                      1.15.1                   pypi_0    pypi
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.0                    pypi_0    pypi
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    3.0.0                    pypi_0    pypi
dav1d                     1.2.1                hd590300_0    conda-forge
dbus                      1.13.18              hb2f20db_0
dendropy                  4.6.1              pyhdfd78af_0    bioconda
docopt                    0.6.2                      py_1    conda-forge
epoxy                     1.5.10               h166bdaf_1    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
ffmpeg                    6.0.0           gpl_hdbbbd96_103    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.41.0                   pypi_0    pypi
freetype                  2.12.1               hca18f0e_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
gdk-pixbuf                2.42.10              h6b639ba_2    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
glib                      2.76.4               hfc55251_0    conda-forge
glib-tools                2.76.4               hfc55251_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gnutls                    3.7.8                hf3e180e_0    conda-forge
graph-tool                2.57            py310haee70ea_0    conda-forge
graph-tool-base           2.57            py310h9dc9c16_0    conda-forge
graphite2                 1.3.14               h295c915_1
gtk3                      3.24.38              hf583ffa_0    conda-forge
h5py                      3.8.0                    pypi_0    pypi
harfbuzz                  7.3.0                hdb3a94d_0    conda-forge
hdbscan                   0.8.33                   pypi_0    pypi
hdf5                      1.12.2          nompi_h4df4325_101    conda-forge
hicolor-icon-theme        0.17                 ha770c72_2    conda-forge
icu                       72.1                 hcb278e6_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
joblib                    1.3.0              pyhd8ed1ab_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.4                    pypi_0    pypi
krb5                      1.21.1               h659d440_0    conda-forge
lame                      3.100             h166bdaf_1003    conda-forge
lcms2                     2.15                 haa2dc70_1    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.0.6                hcb278e6_1    conda-forge
libass                    0.17.1               hc9aadba_0    conda-forge
libblas                   3.9.0           17_linux64_openblas    conda-forge
libbrotlicommon           1.0.9                h166bdaf_9    conda-forge
libbrotlidec              1.0.9                h166bdaf_9    conda-forge
libbrotlienc              1.0.9                h166bdaf_9    conda-forge
libcblas                  3.9.0           17_linux64_openblas    conda-forge
libcups                   2.3.3                h4637d8d_4    conda-forge
libcurl                   8.2.0                hca28451_0    conda-forge
libdeflate                1.18                 h0b41bf4_0    conda-forge
libdrm                    2.4.114              h166bdaf_0    conda-forge
libedit                   3.1.20221030         h5eee18b_0
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.4                h6a678d5_0
libgcc-ng                 13.1.0               he5830b7_0    conda-forge
libgfortran-ng            13.1.0               h69a702a_0    conda-forge
libgfortran5              13.1.0               h15d22d2_0    conda-forge
libgirepository           1.76.1               h8bf6c18_0    conda-forge
libglib                   2.76.4               hebfc3b9_0    conda-forge
libgomp                   13.1.0               he5830b7_0    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libjpeg-turbo             2.1.5.1              h0b41bf4_0    conda-forge
liblapack                 3.9.0           17_linux64_openblas    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libopenblas               0.3.23          pthreads_h80387f5_0    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpciaccess              0.17                 h166bdaf_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
librsvg                   2.56.1               h98fae49_0    conda-forge
libsqlite                 3.42.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libtiff                   4.5.1                h8b53f26_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libva                     2.19.0               hd590300_0    conda-forge
libvpx                    1.13.0               hcb278e6_0    conda-forge
libwebp-base              1.3.1                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxkbcommon              1.5.0                h5d7e998_3    conda-forge
libxml2                   2.11.4               h0d562d8_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
mandrake                  1.2.2                    pypi_0    pypi
matplotlib                3.7.1                    pypi_0    pypi
matplotlib-base           3.7.1           py310he60537e_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nettle                    3.8.1                hc379101_1    conda-forge
networkx                  3.1                pyhd8ed1ab_0    conda-forge
numpy                     1.25.1                   pypi_0    pypi
openblas                  0.3.23          pthreads_h855a84d_0    conda-forge
openh264                  2.3.1                hcb278e6_2    conda-forge
openjpeg                  2.5.0                hfec8fc6_2    conda-forge
openssl                   3.1.1                hd590300_1    conda-forge
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandas                    2.0.3                    pypi_0    pypi
pango                     1.50.14              heaa33ce_1    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pillow                    10.0.0                   pypi_0    pypi
pip                       23.2               pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
platformdirs              3.9.1              pyhd8ed1ab_0    conda-forge
plotly                    5.15.0             pyhd8ed1ab_0    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
poppunk                   2.6.0                    pypi_0    pypi
pp-sketchlib              2.1.1                    pypi_0    pypi
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
pycairo                   1.24.0                   pypi_0    pypi
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygobject                 3.44.1                   pypi_0    pypi
pyparsing                 3.1.0              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1                    pypi_0    pypi
python                    3.10.12         hd12c33a_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
rapidnj                   2.3.2                h4ac6f70_4    bioconda
readline                  8.2                  h8228510_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
scikit-learn              1.3.0                    pypi_0    pypi
scipy                     1.11.1                   pypi_0    pypi
setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
sigcpp-3.0                3.4.0                hcb278e6_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sparsehash                2.0.4                hcb278e6_1    conda-forge
svt-av1                   1.6.0                h59595ed_0    conda-forge
tenacity                  8.2.2              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
treeswift                 1.1.37             pyh7cba7a3_0    bioconda
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
unicodedata2              15.0.0                   pypi_0    pypi
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
wayland                   1.22.0               h8c25dac_1    conda-forge
wheel                     0.40.0             pyhd8ed1ab_1    conda-forge
x264                      1!164.3095           h166bdaf_2    conda-forge
x265                      3.5                  h924138e_3    conda-forge
xkeyboard-config          2.39                 hd590300_0    conda-forge
xorg-compositeproto       0.4.2             h7f98852_1001    conda-forge
xorg-damageproto          1.2.1             h7f98852_1002    conda-forge
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-inputproto           2.3.2             h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.8.6                h8ee46fc_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxaw               1.0.14               h7f98852_1    conda-forge
xorg-libxcomposite        0.4.6                h0b41bf4_1    conda-forge
xorg-libxcursor           1.2.0                h0b41bf4_1    conda-forge
xorg-libxdamage           1.1.5                h7f98852_1    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxi                1.7.10               h7f98852_0    conda-forge
xorg-libxinerama          1.1.5                h27087fc_0    conda-forge
xorg-libxmu               1.1.3                h7f98852_0    conda-forge
xorg-libxpm               3.5.16               hd590300_0    conda-forge
xorg-libxrandr            1.5.2                h7f98852_1    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-libxt                1.3.0                hd590300_0    conda-forge
xorg-libxtst              1.2.3             h7f98852_1002    conda-forge
xorg-randrproto           1.5.0             h7f98852_1001    conda-forge
xorg-recordproto          1.14.2            h7f98852_1002    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-util-macros          1.19.3               h7f98852_0    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.4.2                h5eee18b_0
zlib                      1.2.13               hd590300_5    conda-forge
zstandard                 0.19.0                   pypi_0    pypi
zstd                      1.5.5                hc292b87_0

Sincerely,

David

@johnlees
Copy link
Member

The solution from the linked issues seem to be to update your glibc, see scikit-learn/scikit-learn#14485 (comment)

@johnlees johnlees added the package Packaging of code label Nov 30, 2023
@flyskywhy
Copy link

As pytorch/pytorch#2575 said That's also why changing import order can fix things, because if you change it in a way that loads all your "static TLS" libraries first, then future "dynamic TLS" libraries will resize the DTV like normal, I just insert import cv2 at the 1st line of https://github.com/rupeshs/fastsdcpu/blob/main/src/app.py and solve the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
package Packaging of code
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@flyskywhy @johnlees @davidtong28