Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xarray.open_mfdataset() example in Xee README fails #95

Closed
tylere opened this issue Nov 6, 2023 · 1 comment
Closed

xarray.open_mfdataset() example in Xee README fails #95

tylere opened this issue Nov 6, 2023 · 1 comment
Assignees
Labels
bug Something isn't working documentation Improvements or additions to documentation P2

Comments

@tylere
Copy link
Collaborator

tylere commented Nov 6, 2023

Is dask required for the open_mfdataset() example to work?

To reproduce

mamba create -n xee-test -c conda-forge python=3.11 xee 

Open a python shell and run the open_mfdataset example:

(xee-test) tylere@Tylers-MBP xee-test % python
Python 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:37:07) [Clang 15.0.7 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ee
>>> import xarray
>>> ee.Initialize(opt_url='https://earthengine-highvolume.googleapis.com')
>>> ds = xarray.open_mfdataset(['ee://ECMWF/ERA5_LAND/HOURLY', 'ee://NASA/GDDP-CMIP6'],
...                            engine='ee', crs='EPSG:4326', scale=0.25)

Traceback

/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xee/ext.py:565: UserWarning: Converting non-nanosecond precision datetime values to nanosecond precision. This behavior can eventually be relaxed in xarray, as it is an artifact from pandas which is now beginning to support non-nanosecond precision values. This warning is caused by passing non-nanosecond np.datetime64 or np.timedelta64 values to the DataArray or Variable constructor; it can be silenced by converting the values to nanosecond precision ahead of time.
  xarray.Variable(self.primary_dim_name, primary_coord),
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/backends/api.py", line 1027, in open_mfdataset
    datasets = [open_(p, **open_kwargs) for p in paths]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/backends/api.py", line 1027, in <listcomp>
    datasets = [open_(p, **open_kwargs) for p in paths]
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/backends/api.py", line 579, in open_dataset
    ds = _dataset_from_backend_dataset(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/backends/api.py", line 371, in _dataset_from_backend_dataset
    ds = _chunk_ds(
         ^^^^^^^^^^
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/backends/api.py", line 319, in _chunk_ds
    chunkmanager = guess_chunkmanager(chunked_array_type)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tylere/mambaforge/envs/xee-test/lib/python3.11/site-packages/xarray/core/parallelcompat.py", line 93, in guess_chunkmanager
    raise ValueError(
ValueError: unrecognized chunk manager dask - must be one of: []
>>>

Environment:

Pip package versions:

earthengine-api               0.1.377
xarray                        2023.10.1
xee                           0.0.2
pip list Package Version ------------------------ ------------ affine 2.4.0 aiohttp 3.8.6 aiosignal 1.3.1 async-timeout 4.0.3 attrs 23.1.0 Brotli 1.1.0 cachetools 5.3.2 certifi 2023.7.22 cffi 1.16.0 charset-normalizer 3.3.2 cryptography 41.0.5 earthengine-api 0.1.377 frozenlist 1.4.0 google-api-core 2.12.0 google-api-python-client 2.106.0 google-auth 2.23.4 google-auth-httplib2 0.1.1 google-cloud-core 2.3.3 google-cloud-storage 2.13.0 google-crc32c 1.1.2 google-resumable-media 2.6.0 googleapis-common-protos 1.61.0 grpcio 1.59.2 httplib2 0.22.0 idna 3.4 multidict 6.0.4 numpy 1.26.0 packaging 23.2 pandas 2.1.2 pip 23.3.1 protobuf 4.24.4 pyasn1 0.5.0 pyasn1-modules 0.3.0 pycparser 2.21 pyOpenSSL 23.3.0 pyparsing 3.1.1 pyproj 3.6.1 PySocks 1.7.1 python-dateutil 2.8.2 pytz 2023.3.post1 pyu2f 0.1.5 requests 2.31.0 rsa 4.9 setuptools 68.2.2 six 1.16.0 typing_extensions 4.8.0 tzdata 2023.3 uritemplate 4.1.1 urllib3 2.0.7 wheel 0.41.3 xarray 2023.10.1 xee 0.0.2 yarl 1.9.2
conda list # packages in environment at /Users/tylere/mambaforge/envs/xee-test: # # Name Version Build Channel affine 2.4.0 pyhd8ed1ab_0 conda-forge aiohttp 3.8.6 py311h05b510d_1 conda-forge aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.3 pyhd8ed1ab_0 conda-forge attrs 23.1.0 pyh71513ae_1 conda-forge brotli-python 1.1.0 py311ha891d26_1 conda-forge bzip2 1.0.8 h93a5062_5 conda-forge c-ares 1.21.0 h93a5062_0 conda-forge ca-certificates 2023.7.22 hf0a4a13_0 conda-forge cachetools 5.3.2 pyhd8ed1ab_0 conda-forge certifi 2023.7.22 pyhd8ed1ab_0 conda-forge cffi 1.16.0 py311h4a08483_0 conda-forge charset-normalizer 3.3.2 pyhd8ed1ab_0 conda-forge cryptography 41.0.5 py311h71175c2_0 conda-forge earthengine-api 0.1.377 pyhd8ed1ab_0 conda-forge frozenlist 1.4.0 py311heffc1b2_1 conda-forge google-api-core 2.12.0 pyhd8ed1ab_0 conda-forge google-api-python-client 2.106.0 pyhd8ed1ab_0 conda-forge google-auth 2.23.4 pyhca7485f_0 conda-forge google-auth-httplib2 0.1.1 pyhd8ed1ab_0 conda-forge google-cloud-core 2.3.3 pyhd8ed1ab_0 conda-forge google-cloud-storage 2.13.0 pyhca7485f_0 conda-forge google-crc32c 1.1.2 py311h533d1a3_5 conda-forge google-resumable-media 2.6.0 pyhd8ed1ab_0 conda-forge googleapis-common-protos 1.61.0 pyhd8ed1ab_0 conda-forge grpcio 1.59.2 py311h79dd126_0 conda-forge httplib2 0.22.0 pyhd8ed1ab_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge krb5 1.21.2 h92f50d5_0 conda-forge lerc 4.0.0 h9a09cb3_0 conda-forge libabseil 20230802.1 cxx17_h13dd4ca_0 conda-forge libblas 3.9.0 19_osxarm64_openblas conda-forge libcblas 3.9.0 19_osxarm64_openblas conda-forge libcrc32c 1.1.2 hbdafb3b_0 conda-forge libcurl 8.4.0 h2d989ff_0 conda-forge libcxx 16.0.6 h4653b0c_0 conda-forge libdeflate 1.19 hb547adb_0 conda-forge libedit 3.1.20191231 hc8eb9b7_2 conda-forge libev 4.33 h642e427_1 conda-forge libexpat 2.5.0 hb7217d7_1 conda-forge libffi 3.4.2 h3422bc3_5 conda-forge libgfortran 5.0.0 13_2_0_hd922786_1 conda-forge libgfortran5 13.2.0 hf226fd6_1 conda-forge libgrpc 1.59.2 hbcf6334_0 conda-forge libjpeg-turbo 3.0.0 hb547adb_1 conda-forge liblapack 3.9.0 19_osxarm64_openblas conda-forge libnghttp2 1.55.1 h2b02ca0_0 conda-forge libopenblas 0.3.24 openmp_hd76b1f2_0 conda-forge libprotobuf 4.24.4 hc9861d8_0 conda-forge libre2-11 2023.06.02 h1753957_0 conda-forge libsqlite 3.44.0 h091b4b1_0 conda-forge libssh2 1.11.0 h7a5bd25_0 conda-forge libtiff 4.6.0 ha8a6c65_2 conda-forge libwebp-base 1.3.2 hb547adb_0 conda-forge libzlib 1.2.13 h53f4e23_5 conda-forge llvm-openmp 17.0.4 hcd81f8e_0 conda-forge multidict 6.0.4 py311he2be06e_1 conda-forge ncurses 6.4 h463b476_2 conda-forge numpy 1.26.0 py311hb8f3215_0 conda-forge openssl 3.1.4 h0d3ecfb_0 conda-forge packaging 23.2 pyhd8ed1ab_0 conda-forge pandas 2.1.2 py311h6e08293_0 conda-forge pip 23.3.1 pyhd8ed1ab_0 conda-forge proj 9.3.0 h52fb9d0_2 conda-forge protobuf 4.24.4 py311h4d1eceb_0 conda-forge pyasn1 0.5.0 pyhd8ed1ab_0 conda-forge pyasn1-modules 0.3.0 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pyopenssl 23.3.0 pyhd8ed1ab_0 conda-forge pyparsing 3.1.1 pyhd8ed1ab_0 conda-forge pyproj 3.6.1 py311h20a9b75_4 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.11.6 h47c9636_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-tzdata 2023.3 pyhd8ed1ab_0 conda-forge python_abi 3.11 4_cp311 conda-forge pytz 2023.3.post1 pyhd8ed1ab_0 conda-forge pyu2f 0.1.5 pyhd8ed1ab_0 conda-forge re2 2023.06.02 h6135d0a_0 conda-forge readline 8.2 h92ec313_1 conda-forge requests 2.31.0 pyhd8ed1ab_0 conda-forge rsa 4.9 pyhd8ed1ab_0 conda-forge setuptools 68.2.2 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.44.0 hf2abe2d_0 conda-forge tk 8.6.13 h5083fa2_1 conda-forge typing-extensions 4.8.0 hd8ed1ab_0 conda-forge typing_extensions 4.8.0 pyha770c72_0 conda-forge tzdata 2023c h71feb2d_0 conda-forge uritemplate 4.1.1 pyhd8ed1ab_0 conda-forge urllib3 2.0.7 pyhd8ed1ab_0 conda-forge wheel 0.41.3 pyhd8ed1ab_0 conda-forge xarray 2023.10.1 pyhd8ed1ab_0 conda-forge xee 0.0.2 pyhd8ed1ab_0 conda-forge xz 5.2.6 h57fd34a_0 conda-forge yarl 1.9.2 py311h05b510d_1 conda-forge zstd 1.5.5 h4f39d0f_0 conda-forge
@naschmitz naschmitz self-assigned this Dec 20, 2023
@naschmitz naschmitz added bug Something isn't working documentation Improvements or additions to documentation P2 labels Dec 20, 2023
@naschmitz
Copy link
Collaborator

Closing this as fixed. I haven't mirrored your setup directly, but I fixed this dependency issue for integration tests in #94.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation P2
Projects
None yet
Development

No branches or pull requests

2 participants