Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pythonPackages.matplotlib: fix headless detection #124439

Merged
merged 3 commits into from
May 26, 2021
Merged

Conversation

rnhmjoj
Copy link
Contributor

@rnhmjoj rnhmjoj commented May 25, 2021

Motivation for this change

The default backend is chosen based on the content of the $DISPLAY
variable and a successfull call to libX11, loaded via dlopen().
The test fails because dlopen looks in /usr/lib and /lib, so matplotlib
falls back to a headless backend.

To reproduce try running:

$ nix-shell -I nixpkgs=$PWD -p \
  'python3.withPackages (p: [ p.matplotlib ])' --run python
>>> import matplotlib.pyplot as plt
>>> assert plt.get_backend() == "TkAgg"
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via the above method
  • Tested compilation of all pkgs that depend on this change
  • Tested execution of all binary files (none)
  • Added a release notes entry if the change is major or breaking (it's not)
  • Fits CONTRIBUTING.md.

@rnhmjoj rnhmjoj requested review from FRidh and jonringer as code owners May 25, 2021 23:38
@ofborg ofborg bot requested review from lovek323 and veprbl May 25, 2021 23:48
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501+ 10.rebuild-linux: 501-1000 labels May 25, 2021
@r-rmcgibbo
Copy link

r-rmcgibbo commented May 26, 2021

Result of nixpkgs-review pr 124439 at fa2fa9db run on x86_64-linux 1

57 packages marked as broken and skipped:
  • cassandra_2_1
  • cassandra_2_2
  • hadoop
  • hadoop_2_7
  • hadoop_2_8
  • hadoop_2_9
  • hydrus
  • impressive
  • linuxPackages-libre.virtualbox
  • linuxPackages_hardkernel_4_14.virtualbox
  • ...
3 packages failed to build:
363 packages skipped due to time constraints:
  • DisnixWebService
  • R
  • abcl
  • adapta-gtk-theme
  • adoptopenjdk-icedtea-web
  • alloy (alloy5)
  • alloy4
  • antlr
  • antlr3 (antlr3_5)
  • antlr3_4
  • ...
164 packages built successfully:
  • anki
  • ankisyncd
  • asciidoc-full-with-plugins
  • corrscope
  • datasette (python38Packages.datasette)
  • deeptools
  • evillimiter
  • fdroidserver
  • flatcam
  • fop
  • graph-cli
  • igraph
  • image-analyzer
  • jre8
  • liberation-sans-narrow
  • liberation_ttf (liberation_ttf_v2)
  • mavproxy
  • mnemosyne
  • pitivi
  • psrecord
  • pyspread
  • python38Packages.Pweave
  • python38Packages.androguard
  • python38Packages.ase
  • python38Packages.basemap
  • python38Packages.bayespy
  • python38Packages.booleanoperations
  • python38Packages.castepxbin
  • python38Packages.colorcet
  • python38Packages.dask-image
  • python38Packages.deap
  • python38Packages.defcon
  • python38Packages.dufte
  • python38Packages.fipy
  • python38Packages.fontmath
  • python38Packages.fontparts
  • python38Packages.fontpens
  • python38Packages.fonttools
  • python38Packages.glymur
  • python38Packages.holoviews
  • python38Packages.hvplot
  • python38Packages.image-match
  • python38Packages.imagecorruptions
  • python38Packages.ipydatawidgets
  • python38Packages.ipympl
  • python38Packages.librosa
  • python38Packages.lightning
  • python38Packages.markdown-it-py
  • python38Packages.matplotlib
  • python38Packages.mdit-py-plugins
  • python38Packages.memory_profiler
  • python38Packages.mplfinance
  • python38Packages.mutatormath
  • python38Packages.nbsmoke
  • python38Packages.nbval
  • python38Packages.osmpythontools
  • python38Packages.persim
  • python38Packages.phik
  • python38Packages.pims
  • python38Packages.pint
  • python38Packages.psautohint
  • python38Packages.pyalgotrade
  • python38Packages.pycm
  • python38Packages.pycocotools
  • python38Packages.pymatgen
  • python38Packages.pyregion
  • python38Packages.pyspice
  • python38Packages.pytest-astropy
  • python38Packages.pytest-astropy-header
  • python38Packages.pytest-mpl
  • python38Packages.pytest-regressions
  • python38Packages.python-csxcad
  • python38Packages.python-openems
  • python38Packages.pytomlpp
  • python38Packages.qcelemental
  • python38Packages.regional
  • python38Packages.ripser
  • python38Packages.scapy
  • python38Packages.scikit-bio
  • python38Packages.scikitimage
  • python38Packages.seaborn
  • python38Packages.showit
  • python38Packages.skidl
  • python38Packages.slicedimage
  • python38Packages.spyder-kernels_0_5
  • python38Packages.spyder_3
  • python38Packages.stytra
  • python38Packages.tadasets
  • python38Packages.ufoprocessor
  • python38Packages.vidstab
  • python38Packages.wordcloud
  • python39Packages.Pweave
  • python39Packages.androguard
  • python39Packages.ase
  • python39Packages.bayespy
  • python39Packages.booleanoperations
  • python39Packages.castepxbin
  • python39Packages.colorcet
  • python39Packages.dask-image
  • python39Packages.datasette
  • python39Packages.deap
  • python39Packages.defcon
  • python39Packages.dufte
  • python39Packages.fontmath
  • python39Packages.fontparts
  • python39Packages.fontpens
  • python39Packages.fonttools
  • python39Packages.glymur
  • python39Packages.holoviews
  • python39Packages.hvplot
  • python39Packages.image-match
  • python39Packages.imagecorruptions
  • python39Packages.ipydatawidgets
  • python39Packages.ipympl
  • python39Packages.kmapper
  • python39Packages.lightning
  • python39Packages.markdown-it-py
  • python39Packages.matplotlib
  • python39Packages.mdit-py-plugins
  • python39Packages.memory_profiler
  • python39Packages.mplfinance
  • python39Packages.mutatormath
  • python39Packages.nbsmoke
  • python39Packages.nbval
  • python39Packages.osmpythontools
  • python39Packages.perfplot
  • python39Packages.persim
  • python39Packages.pims
  • python39Packages.pint
  • python39Packages.psautohint
  • python39Packages.pyalgotrade
  • python39Packages.pycm
  • python39Packages.pycocotools
  • python39Packages.pymatgen
  • python39Packages.pyregion
  • python39Packages.pyspice
  • python39Packages.pytest-astropy
  • python39Packages.pytest-astropy-header
  • python39Packages.pytest-mpl
  • python39Packages.pytest-regressions
  • python39Packages.python-igraph
  • python39Packages.pytomlpp
  • python39Packages.qcelemental
  • python39Packages.regional
  • python39Packages.ripser
  • python39Packages.scapy
  • python39Packages.scikit-bio
  • python39Packages.scikit-fuzzy
  • python39Packages.scikitimage
  • python39Packages.seaborn
  • python39Packages.showit
  • python39Packages.skidl
  • python39Packages.slicedimage
  • python39Packages.spyder-kernels_0_5
  • python39Packages.spyder_3
  • python39Packages.tadasets
  • python39Packages.ufoprocessor
  • python39Packages.vidstab
  • python39Packages.wordcloud
  • python39Packages.xgboost
  • scfbuild
  • tartube
  • visidata
  • xits-math
6 suggestions:
  • warning: unused-argument

    Unused argument: nose.
    Near pkgs/development/python-modules/matplotlib/default.nix:2:28:

      |
    2 | , which, cycler, dateutil, nose, numpy, pyparsing, sphinx, tornado, kiwisolver
      |                            ^
    
  • warning: license-missing

    Package is missing a license.

    Near pkgs/development/python-modules/matplotlib/default.nix:81:3:

       |
    81 |   meta = with lib; {
       |   ^
    
  • warning: unused-argument

    Unused argument: pythonOlder.
    Near pkgs/development/python-modules/matplotlib/default.nix:13:3:

       |
    13 | , pythonOlder
       |   ^
    
  • warning: unused-argument

    Unused argument: python.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:38:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                      ^
    
  • warning: unused-argument

    Unused argument: backports_functools_lru_cache.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:83:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                                                                   ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/matplotlib/default.nix:82:0:

       |
    82 |     description = "Python plotting library, making publication quality plots";
       | ^
    

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.


Result of nixpkgs-review pr 124439 at fa2fa9db run on aarch64-linux 1

83 packages marked as broken and skipped:
  • belle-sip
  • cassandra_2_1
  • cassandra_2_2
  • cvc4
  • deeptools
  • herwig
  • hydrus
  • impressive
  • liblinphone
  • lime
  • ...
4 packages failed to build:
278 packages skipped due to time constraints:
  • DisnixWebService
  • R
  • abcl
  • adapta-gtk-theme
  • adoptopenjdk-icedtea-web
  • alloy (alloy5)
  • alloy4
  • antlr
  • antlr3 (antlr3_5)
  • antlr3_4
  • ...
157 packages built successfully:
  • anki
  • ankisyncd
  • asciidoc-full-with-plugins
  • corrscope
  • datasette (python38Packages.datasette)
  • evillimiter
  • fdroidserver
  • flatcam
  • fop
  • graph-cli
  • igraph
  • image-analyzer
  • jre8 (jre8_headless)
  • mavproxy
  • mnemosyne
  • pitivi
  • psrecord
  • pyspread
  • python38Packages.Pweave
  • python38Packages.androguard
  • python38Packages.ase
  • python38Packages.basemap
  • python38Packages.bayespy
  • python38Packages.booleanoperations
  • python38Packages.castepxbin
  • python38Packages.colorcet
  • python38Packages.deap
  • python38Packages.defcon
  • python38Packages.dufte
  • python38Packages.fontmath
  • python38Packages.fontparts
  • python38Packages.fontpens
  • python38Packages.fonttools
  • python38Packages.glymur
  • python38Packages.gpy
  • python38Packages.gpyopt
  • python38Packages.holoviews
  • python38Packages.hvplot
  • python38Packages.image-match
  • python38Packages.imagecorruptions
  • python38Packages.ipydatawidgets
  • python38Packages.ipympl
  • python38Packages.kmapper
  • python38Packages.librosa
  • python38Packages.lightning
  • python38Packages.markdown-it-py
  • python38Packages.matplotlib
  • python38Packages.mdit-py-plugins
  • python38Packages.memory_profiler
  • python38Packages.mplfinance
  • python38Packages.mutatormath
  • python38Packages.nbsmoke
  • python38Packages.nbval
  • python38Packages.osmpythontools
  • python38Packages.persim
  • python38Packages.phik
  • python38Packages.pims
  • python38Packages.pint
  • python38Packages.psautohint
  • python38Packages.pyalgotrade
  • python38Packages.pycm
  • python38Packages.pycocotools
  • python38Packages.pymatgen
  • python38Packages.pyregion
  • python38Packages.pyspice
  • python38Packages.pytest-astropy
  • python38Packages.pytest-astropy-header
  • python38Packages.pytest-mpl
  • python38Packages.pytest-regressions
  • python38Packages.python-igraph
  • python38Packages.pytomlpp
  • python38Packages.qcelemental
  • python38Packages.regional
  • python38Packages.ripser
  • python38Packages.scapy
  • python38Packages.scikit-tda
  • python38Packages.scikitimage
  • python38Packages.seaborn
  • python38Packages.showit
  • python38Packages.skidl
  • python38Packages.slicedimage
  • spyder (python38Packages.spyder)
  • python38Packages.spyder-kernels_0_5
  • python38Packages.spyder_3
  • python38Packages.stytra
  • python38Packages.tadasets
  • python38Packages.ufoprocessor
  • python38Packages.vidstab
  • python38Packages.wordcloud
  • python39Packages.Pweave
  • python39Packages.androguard
  • python39Packages.ase
  • python39Packages.bayespy
  • python39Packages.booleanoperations
  • python39Packages.castepxbin
  • python39Packages.colorcet
  • python39Packages.deap
  • python39Packages.defcon
  • python39Packages.dufte
  • python39Packages.fontmath
  • python39Packages.fontparts
  • python39Packages.fontpens
  • python39Packages.fonttools
  • python39Packages.glymur
  • python39Packages.holoviews
  • python39Packages.hvplot
  • python39Packages.image-match
  • python39Packages.imagecorruptions
  • python39Packages.ipydatawidgets
  • python39Packages.ipympl
  • python39Packages.kmapper
  • python39Packages.lightning
  • python39Packages.livelossplot
  • python39Packages.markdown-it-py
  • python39Packages.matplotlib
  • python39Packages.mdit-py-plugins
  • python39Packages.memory_profiler
  • python39Packages.mplfinance
  • python39Packages.mutatormath
  • python39Packages.nbsmoke
  • python39Packages.nbval
  • python39Packages.osmpythontools
  • python39Packages.perfplot
  • python39Packages.persim
  • python39Packages.pims
  • python39Packages.pint
  • python39Packages.psautohint
  • python39Packages.pycm
  • python39Packages.pycocotools
  • python39Packages.pymatgen
  • python39Packages.pyregion
  • python39Packages.pyspice
  • python39Packages.pytest-astropy
  • python39Packages.pytest-astropy-header
  • python39Packages.pytest-mpl
  • python39Packages.pytest-regressions
  • python39Packages.python-igraph
  • python39Packages.pytomlpp
  • python39Packages.qcelemental
  • python39Packages.regional
  • python39Packages.ripser
  • python39Packages.scapy
  • python39Packages.scikit-fuzzy
  • python39Packages.scikitimage
  • python39Packages.seaborn
  • python39Packages.showit
  • python39Packages.skidl
  • python39Packages.slicedimage
  • python39Packages.spyder-kernels_0_5
  • python39Packages.tadasets
  • python39Packages.ufoprocessor
  • python39Packages.vidstab
  • python39Packages.wordcloud
  • scfbuild
  • tartube
  • visidata
  • xits-math
6 suggestions:
  • warning: unused-argument

    Unused argument: nose.
    Near pkgs/development/python-modules/matplotlib/default.nix:2:28:

      |
    2 | , which, cycler, dateutil, nose, numpy, pyparsing, sphinx, tornado, kiwisolver
      |                            ^
    
  • warning: unused-argument

    Unused argument: pythonOlder.
    Near pkgs/development/python-modules/matplotlib/default.nix:13:3:

       |
    13 | , pythonOlder
       |   ^
    
  • warning: python-include-tests

    Consider adding a checkPhase for tests, or if not feasible, pythonImportsCheck.

    Near pkgs/development/python-modules/matplotlib/default.nix:82:0:

       |
    82 |     description = "Python plotting library, making publication quality plots";
       | ^
    
  • warning: unused-argument

    Unused argument: backports_functools_lru_cache.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:83:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                                                                   ^
    
  • warning: license-missing

    Package is missing a license.

    Near pkgs/development/python-modules/matplotlib/default.nix:81:3:

       |
    81 |   meta = with lib; {
       |   ^
    
  • warning: unused-argument

    Unused argument: python.
    Near pkgs/development/python-modules/matplotlib/default.nix:1:38:

      |
    1 | { lib, stdenv, fetchPypi, writeText, python, buildPythonPackage, isPy3k, pycairo, backports_functools_lru_cache
      |                                      ^
    

Note that build failures may predate this PR, and could be nondeterministic or hardware dependent.
Please exercise your independent judgement. Does something look off? Please file an issue or reach out on IRC.

@SuperSandro2000
Copy link
Member

We probably should rebase this against staging.

@CMCDragonkai
Copy link
Member

This just bit me. The release 21.05 has this problem. It should be backported to release 21.05.

@veprbl veprbl added the 9.needs: port to stable A PR needs a backport to the stable release. label May 26, 2021
The default backend is chosen based on the content of the $DISPLAY
variable *and* a successfull call to libX11, loaded via dlopen().
The test fails because dlopen looks in /usr/lib and /lib, so matplotlib
falls back to a headless backend.

To reproduce try running:

    $ nix-shell -I nixpkgs=$PWD -p \
      'python3.withPackages (p: [ p.matplotlib ])' --run python
    >>> import matplotlib.pyplot as plt
    >>> assert plt.get_backend() == "TkAgg"
@ofborg ofborg bot requested a review from veprbl May 26, 2021 06:05
Copy link
Member

@veprbl veprbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ofborg ofborg bot requested a review from veprbl May 26, 2021 06:41
@veprbl
Copy link
Member

veprbl commented May 26, 2021

@GrahamcOfBorg build python3Packages.matplotlib

@veprbl veprbl merged commit 66e7fcf into NixOS:master May 26, 2021
@rnhmjoj rnhmjoj added the 8.has: port to stable A PR already has a backport to the stable release. label May 26, 2021
@CMCDragonkai
Copy link
Member

CMCDragonkai commented May 27, 2021

How to port this to 21.05? #122042

@veprbl veprbl removed the 9.needs: port to stable A PR needs a backport to the stable release. label May 27, 2021
@veprbl
Copy link
Member

veprbl commented May 27, 2021

How to port this to 21.05? #122042

Already done in #124488

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: python 8.has: port to stable A PR already has a backport to the stable release. 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 501-1000 10.rebuild-linux: 501+
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants