Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test builds at travis failing since release of pluggy v0.12 #220

Closed
jaladh-singhal opened this issue May 30, 2019 · 18 comments
Closed

Test builds at travis failing since release of pluggy v0.12 #220

jaladh-singhal opened this issue May 30, 2019 · 18 comments

Comments

@jaladh-singhal
Copy link

Hi all,
Our test builds at StarKit began to unexpectedly fail since last 3 days. I checked this by doing some random changes in readme (https://travis-ci.org/jaladh-singhal/starkit/jobs/538727910) and they were still failing with same error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/starkit-test-k6s1wpex/lib.linux-x86_64-3.6/starkit/_astropy_init.py", line 111, in test
    coverage=coverage, open_files=open_files, **kwargs)
  File "/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages/astropy/tests/runner.py", line 518, in run_tests
    return super(TestRunner, self).run_tests(**kwargs)
  File "/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages/astropy/tests/runner.py", line 206, in run_tests
    return pytest.main(args=args, plugins=self.keywords['plugins'])
  File "_pytest.config", line 38, in main
  File "_pytest.config", line 117, in _prepareconfig
  File "_pytest.vendored_packages.pluggy", line 724, in __call__
  File "_pytest.vendored_packages.pluggy", line 338, in _hookexec
  File "_pytest.vendored_packages.pluggy", line 333, in <lambda>
  File "_pytest.vendored_packages.pluggy", line 595, in execute
  File "_pytest.vendored_packages.pluggy", line 249, in _wrapped_call
  File "_pytest.helpconfig", line 28, in pytest_cmdline_parse
  File "_pytest.vendored_packages.pluggy", line 278, in get_result
  File "_pytest.vendored_packages.pluggy", line 264, in __init__
  File "_pytest.vendored_packages.pluggy", line 596, in execute
  File "_pytest.config", line 852, in pytest_cmdline_parse
  File "_pytest.config", line 957, in parse
  File "_pytest.config", line 918, in _preparse
  File "_pytest.vendored_packages.pluggy", line 501, in load_setuptools_entrypoints
  File "/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2433, in load
    self.require(*args, **kwargs)
  File "/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2456, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (importlib-metadata 0.0.0 (/home/travis/miniconda2/envs/starkit/lib/python3.6/site-packages), Requirement.parse('importlib-metadata>=0.12'), {'pluggy'})
The command "python setup.py $SETUP_CMD" exited with 1.

I further investigated and found that pluggy has been recently updated to v0.12.0 (on May 28) and since then only our travis builds are failing that are using SETUP_CMD='test --coverage'. FYI, it is an Astropy testing framework that runs pytest by passing option coverage to generate a coverage report.
As much as I could understand, I noticed that pluggy is never used by us directly, it is somehow used by pytest under the hood (I may be wrong)?! Thus I resorted here to inform you about the issue to help us solve the problem. Is it somehow related to issue #205?

Thanks in advance. Please let's try to fix this soon :)
@nicoddemus
Copy link
Member

Hi @jaladh-singhal,

Bummer, sorry about this breakage.

Weird, for some reason your conda environment has a importlib-metadata 0.0.0 installed, which is definitely wrong.

Can you please execute conda list after $ source activate starkit? Can you reproduce this issue locally?

@jaladh-singhal
Copy link
Author

@nicoddemus OK. Thank you for very quick response. :)
Let me try to reproduce this locally. And I'll get back to you with my findings!

@nicoddemus
Copy link
Member

Please also execute $ conda info so we can see exactly which channels you are using. 👍

@jaladh-singhal
Copy link
Author

@nicoddemus Yes I was able to reproduce this same issue locally.

Here is the result of conda list:

# packages in environment at /home/jpolygon/miniconda3/envs/sktest:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.12                     py_0    conda-forge
asn1crypto                0.24.0                py36_1003    conda-forge
astropy                   1.3.3               np112py36_0    conda-forge
atomicwrites              1.3.0                      py_0    conda-forge
attrs                     19.1.0                     py_0    conda-forge
babel                     2.7.0                      py_0    conda-forge
backcall                  0.1.0                      py_0    conda-forge
blas                      1.1                    openblas    conda-forge
bleach                    3.1.0                      py_0    conda-forge
blosc                     1.16.3               hf484d3e_0    conda-forge
bzip2                     1.0.6             h14c3975_1002    conda-forge
ca-certificates           2019.3.9             hecc5488_0    conda-forge
certifi                   2019.3.9                 py36_0    conda-forge
cffi                      1.12.3           py36h8022711_0    conda-forge
chardet                   3.0.4                 py36_1003    conda-forge
corner                    2.0.1                    pypi_0    pypi
coverage                  4.5.3            py36h14c3975_0    conda-forge
coveralls                 1.5.1                      py_0    conda-forge
cryptography              2.6.1            py36h72c5cf5_0    conda-forge
cycler                    0.10.0                     py_1    conda-forge
dbus                      1.13.6               he372182_0    conda-forge
decorator                 4.4.0                      py_0    conda-forge
defusedxml                0.5.0                      py_1    conda-forge
docopt                    0.6.2                      py_1    conda-forge
docutils                  0.14                  py36_1001    conda-forge
entrypoints               0.3                   py36_1000    conda-forge
expat                     2.2.5             hf484d3e_1002    conda-forge
fontconfig                2.13.1            he4413a7_1000    conda-forge
freetype                  2.10.0               he983fc9_0    conda-forge
gettext                   0.19.8.1          hc5be6a0_1002    conda-forge
glib                      2.58.3            hf63aee3_1001    conda-forge
gst-plugins-base          1.14.4            hdf3bae2_1001    conda-forge
gstreamer                 1.14.4            h66beb1c_1001    conda-forge
h5py                      2.9.0           nompi_py36hf008753_1102    conda-forge
hdf5                      1.10.4          nompi_h3c11f04_1106    conda-forge
icu                       58.2              hf484d3e_1000    conda-forge
idna                      2.8                   py36_1000    conda-forge
imagesize                 1.1.0                      py_0    conda-forge
importlib_metadata        0.17                     py36_0    conda-forge
ipykernel                 5.1.1            py36h24bf2e0_0    conda-forge
ipython                   7.5.0            py36h24bf2e0_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.4.2                      py_0    conda-forge
jedi                      0.13.3                   py36_0    conda-forge
jinja2                    2.10.1                     py_0    conda-forge
jpeg                      9c                h14c3975_1001    conda-forge
jsonschema                3.0.1                    py36_0    conda-forge
jupyter                   1.0.0                      py_2    conda-forge
jupyter_client            5.2.4                      py_3    conda-forge
jupyter_console           6.0.0                      py_0    conda-forge
jupyter_core              4.4.0                      py_0    conda-forge
kiwisolver                1.1.0            py36hc9558a2_0    conda-forge
libffi                    3.2.1             he1b5a44_1006    conda-forge
libgcc-ng                 8.2.0                hdf63c60_1  
libgfortran-ng            7.3.0                hdf63c60_0  
libiconv                  1.15              h516909a_1005    conda-forge
libpng                    1.6.37               hed695b0_0    conda-forge
libsodium                 1.0.16            h14c3975_1001    conda-forge
libstdcxx-ng              8.2.0                hdf63c60_1  
libuuid                   2.32.1            h14c3975_1000    conda-forge
libxcb                    1.13              h14c3975_1002    conda-forge
libxml2                   2.9.9                h13577e0_0    conda-forge
lzo                       2.10              h14c3975_1000    conda-forge
markupsafe                1.1.1            py36h14c3975_0    conda-forge
matplotlib                3.1.0                    py36_0    conda-forge
matplotlib-base           3.1.0            py36h5f35d83_0    conda-forge
mistune                   0.8.4           py36h14c3975_1000    conda-forge
more-itertools            4.3.0                 py36_1000    conda-forge
nbconvert                 5.5.0                      py_0    conda-forge
nbformat                  4.4.0                      py_1    conda-forge
nbsphinx                  0.4.2                      py_0    conda-forge
ncurses                   6.1               hf484d3e_1002    conda-forge
notebook                  5.7.8                    py36_1    conda-forge
numexpr                   2.6.9           py36h637b7d7_1000    conda-forge
numpy                     1.12.1          py36_blas_openblash1522bff_1001  [blas_openblas]  conda-forge
numpydoc                  0.8.0                      py_1    conda-forge
openblas                  0.3.3             h9ac9557_1001    conda-forge
openssl                   1.1.1b               h14c3975_1    conda-forge
packaging                 19.0                       py_0    conda-forge
pandas                    0.24.2           py36hf484d3e_0    conda-forge
pandoc                    2.7.2                         0    conda-forge
pandocfilters             1.4.2                      py_1    conda-forge
parso                     0.4.0                      py_0    conda-forge
pcre                      8.41              hf484d3e_1003    conda-forge
pexpect                   4.7.0                    py36_0    conda-forge
pickleshare               0.7.5                 py36_1000    conda-forge
pip                       19.1                     py36_0    conda-forge
pluggy                    0.12.0                     py_0    conda-forge
prometheus_client         0.6.0                      py_0    conda-forge
prompt_toolkit            2.0.9                      py_0    conda-forge
pthread-stubs             0.4               h14c3975_1001    conda-forge
ptyprocess                0.6.0                   py_1001    conda-forge
py                        1.8.0                      py_0    conda-forge
pycparser                 2.19                     py36_1    conda-forge
pygments                  2.4.2                      py_0    conda-forge
pyopenssl                 19.0.0                   py36_0    conda-forge
pyparsing                 2.4.0                      py_0    conda-forge
pyqt                      5.9.2            py36hcca6a23_0    conda-forge
pyrsistent                0.15.2           py36h516909a_0    conda-forge
pysocks                   1.7.0                    py36_0    conda-forge
pytables                  3.4.4           py36h5d69a80_1004    conda-forge
pytest                    4.5.0                    py36_0    conda-forge
pytest-cov                2.7.1                      py_0    conda-forge
python                    3.6.7             h381d211_1004    conda-forge
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2019.1                     py_0    conda-forge
pyyaml                    5.1              py36h14c3975_0    conda-forge
pyzmq                     18.0.1           py36hc4ba49a_1    conda-forge
qt                        5.9.7                h52cfd70_2    conda-forge
qtconsole                 4.5.1                      py_0    conda-forge
readline                  7.0               hf8c457e_1001    conda-forge
requests                  2.22.0                   py36_0    conda-forge
scipy                     1.2.1           py36_blas_openblash1522bff_0  [blas_openblas]  conda-forge
send2trash                1.5.0                      py_0    conda-forge
setuptools                41.0.1                   py36_0    conda-forge
sip                       4.19.8          py36hf484d3e_1000    conda-forge
six                       1.12.0                py36_1000    conda-forge
snowballstemmer           1.2.1                      py_1    conda-forge
sphinx                    1.8.5                    py36_0    conda-forge
sphinx_bootstrap_theme    0.6.4                      py_0    conda-forge
sphinxcontrib-websupport  1.1.2                      py_0    conda-forge
sqlite                    3.28.0               h8b20d00_0    conda-forge
terminado                 0.8.2                    py36_0    conda-forge
testpath                  0.4.2                   py_1001    conda-forge
tk                        8.6.9             h84994c4_1001    conda-forge
tornado                   6.0.2            py36h516909a_0    conda-forge
tqdm                      4.32.1                     py_0    conda-forge
traitlets                 4.3.2                 py36_1000    conda-forge
urllib3                   1.24.3                   py36_0    conda-forge
wcwidth                   0.1.7                      py_1    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.33.4                   py36_0    conda-forge
widgetsnbextension        3.4.2                 py36_1000    conda-forge
xorg-libxau               1.0.9                h14c3975_0    conda-forge
xorg-libxdmcp             1.1.3                h516909a_0    conda-forge
xz                        5.2.4             h14c3975_1001    conda-forge
yaml                      0.1.7             h14c3975_1001    conda-forge
zeromq                    4.3.1             hf484d3e_1000    conda-forge
zipp                      0.5.0                      py_0    conda-forge
zlib                      1.2.11            h14c3975_1004    conda-forge

And conda info:

active environment : sktest
    active env location : /home/jpolygon/miniconda3/envs/sktest
            shell level : 2
       user config file : /home/jpolygon/.condarc
 populated config files : 
          conda version : 4.6.14
    conda-build version : not installed
         python version : 3.7.1.final.0
       base environment : /home/jpolygon/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/jpolygon/miniconda3/pkgs
                          /home/jpolygon/.conda/pkgs
       envs directories : /home/jpolygon/miniconda3/envs
                          /home/jpolygon/.conda/envs
               platform : linux-64
             user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.1 Linux/4.15.0-46-generic ubuntu/16.04.6 glibc/2.23
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

@nicoddemus
Copy link
Member

Thanks @jaladh-singhal for posting this.

importlib_metadata        0.17                     py36_0    conda-forge

Strange, seems fine to me. @asottile can you take a look?

@jaladh-singhal meanwhile, I think as a workaround you can add pluggy==0.11 to your list of packages to fallback to the previous version.

@jaladh-singhal
Copy link
Author

@nicoddemus OK. Can you explain it to me that we only listed pytest in our environment file, then how pluggy gets installed (listed in conda list)? (This may sound silly as I'm new to this)

And could I explicitly list pluggy==0.11 along with pytest in our environment file?

@nicoddemus
Copy link
Member

@nicoddemus OK. Can you explain it to me that we only listed pytest in our environment file, then how pluggy gets installed (listed in conda list)? (This may sound silly as I'm new to this)

No worries: pluggy gets installed because pytest declares it as a dependency. Usually during development one only lists the direct packages one depends on, not other packages that those packages also depend on, otherwise things get unwieldy fast.

For example, pytest depends on these packages: py, pluggy, wcwidth, more-itertools, attrs, colorama, six. Every other packages depends on many others.

And could I explicitly list pluggy==0.11 along with pytest in our environment file?

Exactly. 👍

I suggest also adding a link back to this as a comment, to justify the pin.

@jaladh-singhal
Copy link
Author

No worries: pluggy gets installed because pytest declares it as a dependency. Usually during development one only lists the direct packages one depends on, not other packages that those packages also depend on, otherwise things get unwieldy fast.

Thank you for wonderful explaination. Now I got it! :)

I suggest also adding a link back to this as a comment, to justify the pin.

Great suggestion.

@asottile
Copy link
Member

asottile commented May 30, 2019 •
edited
Loading

This looks like either :

  • not installing from a wheel
  • too-old setuptools at install time (I-m uses declarative metadata)

I'll look into it more when I'm at a computer

@asottile
Copy link
Member

_pytest.vendored_packages.* is from an ancient version of pytest

@nicoddemus
Copy link
Member

_pytest.vendored_packages.* is from an ancient version of pytest

Good catch. Strangely conda list shows pytest 4.5... 🤔

@asottile
Copy link
Member

I suspect the conda environment isn't actually being used and instead some ancient system packages installed by travis are being picked up

@asottile
Copy link
Member

Additionally the conda package is broken:

(starkit) root@723f4227e25c:/tmp# curl -o i-m.tar.bz2 --location https://anaconda.org/conda-forge/importlib_metadata/0.17/download/linux-64/importlib_metadata-0.17-py36_0.tar.bz2

...

(starkit) root@723f4227e25c:/tmp# tar --list -f i-m.tar.bz2  | grep dist-info
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/INSTALLER
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/WHEEL
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/LICENSE
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/METADATA
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/RECORD
lib/python3.6/site-packages/importlib_metadata-0.0.0.dist-info/top_level.txt

@nicoddemus
Copy link
Member

Additionally the conda package is broken:

Ouchie! Will check this out.

@nicoddemus nicoddemus mentioned this issue May 30, 2019
Closed
@nicoddemus
Copy link
Member

I've fixed the conda package, it should be available in a few minutes. I will ping here when it is. 👍

Thanks @asottile! I have the feeling this is the source of the entire problem actually.

@nicoddemus
Copy link
Member

@jaladh-singhal the fixed package is up! Please try again. Your conda list should contain this line:

importlib_metadata        0.17                     py36_1    conda-forge

(Note: py36_1 instead of py36_0)

@asottile
Copy link
Member

🎉 nice!

@jaladh-singhal
Copy link
Author

@nicoddemus @asottile Thank you so much, now it works absolutely fine.

Yes @nicoddemus it contains py36_1 now!

Brilliant work guys - thanks for such a fast fix (now no need for workaround) 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nicoddemus @asottile @jaladh-singhal