PyInstaller executable which packages xarray throws an error when executable is run

[canol]

This might be a problem related to PyInstaller, but we are packaging a lot of packages like numpy, pandas, matplotlib, scipy, PySide2; and xarray is the only one that causes an error, so I wanted to first open the ticket here.

What happened:

When I run an executable that PyInstaller creates out of a script that uses xarray >= 0.14, I get this error (using xarray 0.13 does not cause this error):

  File "example.py", line 2, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\__init__.py", line 3, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\testing.py", line 9, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\core\dataarray.py", line 24, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\plot\__init__.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\plot\dataset_plot.py", line 6, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\core\alignment.py", line 13, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\core\variable.py", line 26, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\core\common.py", line 22, in <module>
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 623, in exec_module
    exec(bytecode, module.__dict__)
  File "lib\site-packages\xarray\core\formatting_html.py", line 11, in <module>
  File "lib\site-packages\pkg_resources\__init__.py", line 1156, in resource_string
  File "lib\site-packages\pkg_resources\__init__.py", line 1401, in get_resource_string
  File "lib\site-packages\pkg_resources\__init__.py", line 1570, in _get
  File "c:\users\can\pycharmprojects\pythonproject\venv4\lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 471, in get_data
    with open(path, 'rb') as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\can\\PycharmProjects\\pythonProject\\dist\\example\\xarray\\static\\css\\style.css'
[2172] Failed to execute script example

Minimal Complete Verifiable Example:

Make sure xarray and PyInstaller is installed on your environment.

Create example.py:

import xarray as xr

a = xr.DataArray(np.zeros(10))
print(a)

on console run this command:

pyinstaller example.py

It will create an executable example.exe in a new dist folder. Run that executable in a command prompt to be able to see the error message.

Anything else we need to know?:

The error does not happen with xarray version 0.13.

Environment:

xr.show_versions() output:

python: 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 03:37:03) [MSC v.1900 64 bit (AMD64)]
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: AMD64 Family 23 Model 1 Stepping 1, AuthenticAMD
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
libhdf5: None
libnetcdf: None
xarray: 0.16.0
pandas: 1.1.0
numpy: 1.19.1
scipy: None
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: None
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.2.0
pip: 20.2
conda: None
pytest: None
IPython: None
sphinx: None

PyInstaller version: 3.6

[shoyer]

Thanks for the report!

I don't know much about how pyinstaller works, but I suspect this may be related to our specification of package_data in the newer setup.cfg rather than setup.py.

CC @crusaderky

[crusaderky]

I am getting the same error plus another:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/crusaderky/PycharmProjects/tmp/dist/t1/distributed/distributed.yaml'
[5

Both xarray and distributed work fine with pip install, python setup.py sdist, and python setup.py bdist. Please open a ticket on the pyinstaller board.
I tried re-adding the static files in setup.py but it doesn't fix the issue.

I'm opening a PR to stop loading the resource files if you're not running on jupyter notebook. This will allow pure xarray (without dask) to work with pyinstaller as long as you don't display an xarray object in jupyter.