Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: 0.2.0 appears to be latest conda-forge release installed on arm64/windows #315

Closed
durack1 opened this issue Aug 16, 2022 · 42 comments · Fixed by conda-forge/xcdat-feedstock#14
Assignees
Labels
type: bug Inconsistencies or issues which will cause an issue or problem for users or implementors.

Comments

@durack1
Copy link
Collaborator

durack1 commented Aug 16, 2022

What happened?

(xcdat031spy532) ml-9953350:ESGFReports duro$ mamba create --dry-run -n xc030spy532mat352 -c conda-forge xcdat

              __    __    __    __
             /  \  /  \  /  \  /  \
            /    \/    \/    \/    \

███████████████/ /██/ /██/ /██/ /████████████████████████
/ / \ / \ / \ / \ ____
/ / _/ _/ _/ \ o _,
/ / _
__/ `
|/
███╗ ███╗ █████╗ ███╗ ███╗██████╗ █████╗
████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
██╔████╔██║███████║██╔████╔██║██████╔╝███████║
██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
██║ ╚═╝ ██║██║ ██║██║ ╚═╝ ██║██████╔╝██║ ██║
╚═╝ ╚═╝╚═╝ ╚═╝╚═╝ ╚═╝╚═════╝ ╚═╝ ╚═╝
mamba (0.24.0) supported by @QuantStack
GitHub: https://github.com/mamba-org/mamba
Twitter: https://twitter.com/QuantStack
█████████████████████████████████████████████████████████████
Looking for: ['xcdat']
conda-forge/osx-arm64 Using cache
conda-forge/noarch Using cache
Transaction
Prefix: ~/mambaforge/envs/xc030spy532mat352
Updating specs:

  • xcdat
    Package Version Build Channel Size
    ────────────────────────────────────────────────────────────────────────────────────────────────
    Install:
    ────────────────────────────────────────────────────────────────────────────────────────────────
  • bokeh 2.4.3 pyhd8ed1ab_3 conda-forge/noarch 14MB
    ...
  • cf_xarray 0.7.4 pyhd8ed1ab_0 conda-forge/noarch Cached
    ...
  • dask 2022.8.0 pyhd8ed1ab_1 conda-forge/noarch 5kB
  • dask-core 2022.8.0 pyhd8ed1ab_0 conda-forge/noarch Cached
    ...
  • hdf4 4.2.15 hc683e77_4 conda-forge/osx-arm64 928kB
  • hdf5 1.12.2 nompi_h8968d4b_100 conda-forge/osx-arm64 Cached
    ...
  • libgfortran 5.0.0.dev0 11_0_1_hf114ba7_23 conda-forge/osx-arm64 Cached
  • libgfortran5 11.0.1.dev0 hf114ba7_23 conda-forge/osx-arm64 Cached
  • liblapack 3.9.0 16_osxarm64_openblas conda-forge/osx-arm64 13kB
  • libnetcdf 4.8.1 nompi_h996a5af_104 conda-forge/osx-arm64 1MB
    ...
  • netcdf4 1.6.0 nompi_py310h9d42ac0_101 conda-forge/osx-arm64 Cached
  • numpy 1.23.1 py310h0a343b5_0 conda-forge/osx-arm64 Cached
    ...
  • python 3.10.5 h71ab1a4_0_cpython conda-forge/osx-arm64 Cached
    ...
  • xarray 2022.6.0 pyhd8ed1ab_1 conda-forge/noarch Cached
    + xcdat 0.2.0 pyhd8ed1ab_0 conda-forge/noarch Cached
  • xorg-libxau 1.0.9 h27ca646_0 conda-forge/osx-arm64 Cached
    ...

What did you expect to happen?

I had expected 0.3.0 to be installed, as per availability through the xcdat conda-forge channel

Minimal Complete Verifiable Example

$ conda create --dry-run -n xc030 -c conda-forge xcdat

  xcdat              conda-forge/noarch::xcdat-0.2.0-pyhd8ed1ab_0

Relevant log output

See above

Anything else we need to know?

Nil

Environment

Nil

@durack1 durack1 added the type: bug Inconsistencies or issues which will cause an issue or problem for users or implementors. label Aug 16, 2022
@tomvothecoder tomvothecoder moved this to Todo in v0.3.1 Aug 16, 2022
@tomvothecoder
Copy link
Collaborator

@durack1 Thanks for reporting this.

This seems to be a problem that is plauging conda/mamba in general.

Also, your log outputs show that the local cached version of xcdat is being used (0.2.0). Maybe try running conda clean -p xcdat first, then conda create --dry-run -n xc030 -c conda-forge xcdat.

@tomvothecoder
Copy link
Collaborator

@durack1 Thanks for reporting this.

This seems to be a problem that is plauging conda/mamba in general (I think with the M1 arm64).

Also, your log outputs show that the local cached version of xcdat is being used (0.2.0). Maybe try running conda clean -p xcdat first, then conda create --dry-run -n xc030 -c conda-forge xcdat.

@durack1
Copy link
Collaborator Author

durack1 commented Aug 16, 2022

@tomvothecoder thanks for the tips. I have been trying a number of conda/mamba variants with clean -a, -i, -p, -t and can't seem to shake the cached xcdat 0.2.0 pyhd8ed1ab_0 conda-forge/noarch Cached package.

I may have to restart my install all over.

As an aside, it seems there are some distutils/setuptools issues that are running around the conda-sphere at the moment too:

@durack1
Copy link
Collaborator Author

durack1 commented Aug 17, 2022

@tomvothecoder apologies that took a while.

Ok after a fresh install of mambaforge 4.13.0-1 and a patch up-to-date with conda, I still get the same issue - full log output below:

(base) $ mamba create --dry-run -n xcd020spy532mat353 -c conda-forge xcdat spyder matplotlib
                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝
        mamba (0.24.0) supported by @QuantStack
        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack
█████████████████████████████████████████████████████████████
Looking for: ['xcdat', 'spyder', 'matplotlib']

conda-forge/osx-arm64                                3.1MB @ 510.2kB/s  6.1s
conda-forge/noarch                                   9.0MB @   1.2MB/s  7.6s
Transaction

  Prefix: ~/mambaforge/envs/xcd020spy532mat353

  Updating specs:

   - xcdat
   - spyder
   - matplotlib

  Package                               Version  Build                    Channel                     Size
────────────────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
────────────────────────────────────────────────────────────────────────────────────────────────────────────

  + alabaster                            0.7.12  py_0                     conda-forge/noarch          15kB
  + applaunchservices                     0.3.0  pyhd8ed1ab_2             conda-forge/noarch           9kB
  + appnope                               0.1.3  pyhd8ed1ab_0             conda-forge/noarch           8kB
  + arrow                                 1.2.2  pyhd8ed1ab_0             conda-forge/noarch          90kB
  + astroid                              2.11.6  py310hbe9552e_0          conda-forge/osx-arm64      366kB
  + atomicwrites                          1.4.1  pyhd8ed1ab_0             conda-forge/noarch          12kB
  + attrs                                22.1.0  pyh71513ae_1             conda-forge/noarch          49kB
  + autopep8                              1.6.0  pyhd8ed1ab_1             conda-forge/noarch          42kB
  + babel                                2.10.3  pyhd8ed1ab_0             conda-forge/noarch           7MB
  + backcall                              0.2.0  pyh9f0ad1d_0             conda-forge/noarch          14kB
  + backports                               1.0  py_2                     conda-forge/noarch           4kB
  + backports.functools_lru_cache         1.6.4  pyhd8ed1ab_0             conda-forge/noarch           9kB
  + beautifulsoup4                       4.11.1  pyha770c72_0             conda-forge/noarch          98kB
  + binaryornot                           0.4.4  py_1                     conda-forge/noarch         378kB
  + black                                22.6.0  pyhd8ed1ab_0             conda-forge/noarch         125kB
  + bleach                                5.0.1  pyhd8ed1ab_0             conda-forge/noarch         127kB
  + bokeh                                 2.4.3  pyhd8ed1ab_3             conda-forge/noarch          14MB
  + brotli                                1.0.9  h1c322ee_7               conda-forge/osx-arm64       18kB
  + brotli-bin                            1.0.9  h1c322ee_7               conda-forge/osx-arm64       18kB
  + brotlipy                              0.7.0  py310hf8d0d8f_1004       conda-forge/osx-arm64      349kB
  + bzip2                                 1.0.8  h3422bc3_4               conda-forge/osx-arm64     Cached
  + c-ares                               1.18.1  h3422bc3_0               conda-forge/osx-arm64     Cached
  + ca-certificates                   2022.6.15  h4653dfc_0               conda-forge/osx-arm64     Cached
  + certifi                           2022.6.15  py310hbe9552e_0          conda-forge/osx-arm64      159kB
  + cf_xarray                             0.7.4  pyhd8ed1ab_0             conda-forge/noarch         253kB
  + cffi                                 1.15.1  py310he00a5c5_0          conda-forge/osx-arm64      227kB
  + cftime                                1.6.1  py310h611a7d1_0          conda-forge/osx-arm64      196kB
  + chardet                               5.0.0  py310hbe9552e_0          conda-forge/osx-arm64      262kB
  + charset-normalizer                    2.1.0  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + click                                 8.1.3  py310hbe9552e_0          conda-forge/osx-arm64      152kB
  + cloudpickle                           2.1.0  pyhd8ed1ab_0             conda-forge/noarch          25kB
  + colorama                              0.4.5  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + cookiecutter                          2.1.1  pyh6c4a22f_0             conda-forge/noarch          93kB
  + cryptography                         37.0.4  py310h94bb23d_0          conda-forge/osx-arm64        1MB
  + curl                                 7.83.1  h2fcd78c_0               conda-forge/osx-arm64      148kB
  + cycler                               0.11.0  pyhd8ed1ab_0             conda-forge/noarch          10kB
  + cytoolz                              0.12.0  py310h02f21da_0          conda-forge/osx-arm64      355kB
  + dask                               2022.8.0  pyhd8ed1ab_1             conda-forge/noarch           5kB
  + dask-core                          2022.8.0  pyhd8ed1ab_0             conda-forge/noarch         864kB
  + dataclasses                             0.8  pyhc8e2a94_3             conda-forge/noarch          10kB
  + debugpy                               1.6.3  py310hc6dc59f_0          conda-forge/osx-arm64        2MB
  + decorator                             5.1.1  pyhd8ed1ab_0             conda-forge/noarch          12kB
  + defusedxml                            0.7.1  pyhd8ed1ab_0             conda-forge/noarch          24kB
  + diff-match-patch                   20200713  pyh9f0ad1d_0             conda-forge/noarch          39kB
  + dill                                0.3.5.1  pyhd8ed1ab_0             conda-forge/noarch          73kB
  + distributed                        2022.8.0  pyhd8ed1ab_1             conda-forge/noarch         760kB
  + docutils                               0.19  py310hbe9552e_0          conda-forge/osx-arm64      780kB
  + entrypoints                             0.4  pyhd8ed1ab_0             conda-forge/noarch           9kB
  + flake8                                4.0.1  pyhd8ed1ab_2             conda-forge/noarch          85kB
  + fonttools                            4.35.0  py310h8c01e39_0          conda-forge/osx-arm64        2MB
  + freetype                             2.12.1  hd633e50_0               conda-forge/osx-arm64      873kB
  + fsspec                             2022.7.1  pyhd8ed1ab_0             conda-forge/noarch          98kB
  + gettext                            0.19.8.1  h049c9fb_1008            conda-forge/osx-arm64        3MB
  + giflib                                5.2.1  h27ca646_2               conda-forge/osx-arm64       74kB
  + glib                                 2.72.1  h332123e_0               conda-forge/osx-arm64      442kB
  + glib-tools                           2.72.1  h332123e_0               conda-forge/osx-arm64       96kB
  + gst-plugins-base                     1.20.3  hbf05cfb_0               conda-forge/osx-arm64        2MB
  + gstreamer                            1.20.3  h3d5cedb_0               conda-forge/osx-arm64        1MB
  + hdf4                                 4.2.15  hc683e77_4               conda-forge/osx-arm64      928kB
  + hdf5                                 1.12.2  nompi_h8968d4b_100       conda-forge/osx-arm64        3MB
  + heapdict                              1.0.1  py_0                     conda-forge/noarch           7kB
  + icu                                    70.1  h6b3803e_0               conda-forge/osx-arm64     Cached
  + idna                                    3.3  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + imagesize                             1.4.1  pyhd8ed1ab_0             conda-forge/noarch          10kB
  + importlib-metadata                   4.11.4  py310hbe9552e_0          conda-forge/osx-arm64       35kB
  + importlib_resources                   5.9.0  pyhd8ed1ab_0             conda-forge/noarch          28kB
  + inflection                            0.5.1  pyh9f0ad1d_0             conda-forge/noarch           9kB
  + intervaltree                          3.0.2  py_0                     conda-forge/noarch          24kB
  + ipykernel                            6.15.1  pyh736e0ef_0             conda-forge/noarch          98kB
  + ipython                              7.33.0  py310hbe9552e_0          conda-forge/osx-arm64        1MB
  + ipython_genutils                      0.2.0  py_1                     conda-forge/noarch          22kB
  + isort                                5.10.1  pyhd8ed1ab_0             conda-forge/noarch          81kB
  + jedi                                 0.18.1  pyhd8ed1ab_2             conda-forge/noarch         818kB
  + jellyfish                             0.9.0  py310hf8d0d8f_1          conda-forge/osx-arm64       39kB
  + jinja2                                3.1.2  pyhd8ed1ab_1             conda-forge/noarch         101kB
  + jinja2-time                           0.2.0  pyhd8ed1ab_3             conda-forge/noarch           9kB
  + jpeg                                     9e  he4db4b2_2               conda-forge/osx-arm64      266kB
  + jsonschema                            4.9.1  pyhd8ed1ab_0             conda-forge/noarch          64kB
  + jupyter_client                        7.3.4  pyhd8ed1ab_0             conda-forge/noarch          93kB
  + jupyter_core                         4.11.1  py310hbe9552e_0          conda-forge/osx-arm64       84kB
  + jupyterlab_pygments                   0.2.2  pyhd8ed1ab_0             conda-forge/noarch          17kB
  + keyring                              23.8.2  py310hbe9552e_0          conda-forge/osx-arm64       54kB
  + kiwisolver                            1.4.4  py310hd23d0e8_0          conda-forge/osx-arm64       63kB
  + krb5                                 1.19.3  hf9b2bbe_0               conda-forge/osx-arm64     Cached
  + lazy-object-proxy                     1.7.1  py310hf8d0d8f_1          conda-forge/osx-arm64       32kB
  + lcms2                                  2.12  had6a04f_0               conda-forge/osx-arm64      378kB
  + lerc                                  4.0.0  h9a09cb3_0               conda-forge/osx-arm64      216kB
  + libblas                               3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libbrotlicommon                       1.0.9  h1c322ee_7               conda-forge/osx-arm64       66kB
  + libbrotlidec                          1.0.9  h1c322ee_7               conda-forge/osx-arm64       30kB
  + libbrotlienc                          1.0.9  h1c322ee_7               conda-forge/osx-arm64      294kB
  + libcblas                              3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libclang                             14.0.6  default_h81a5282_0       conda-forge/osx-arm64      130kB
  + libclang13                           14.0.6  default_he70ec0d_0       conda-forge/osx-arm64        8MB
  + libcurl                              7.83.1  h2fcd78c_0               conda-forge/osx-arm64     Cached
  + libcxx                               14.0.6  h04bba0f_0               conda-forge/osx-arm64     Cached
  + libdeflate                             1.13  h57fd34a_0               conda-forge/osx-arm64       61kB
  + libedit                        3.1.20191231  hc8eb9b7_2               conda-forge/osx-arm64     Cached
  + libev                                  4.33  h642e427_1               conda-forge/osx-arm64     Cached
  + libffi                                3.4.2  h3422bc3_5               conda-forge/osx-arm64     Cached
  + libgfortran                      5.0.0.dev0  11_0_1_hf114ba7_23       conda-forge/osx-arm64       20kB
  + libgfortran5                    11.0.1.dev0  hf114ba7_23              conda-forge/osx-arm64      923kB
  + libglib                              2.72.1  ha1047ec_0               conda-forge/osx-arm64        3MB
  + libiconv                               1.16  h642e427_0               conda-forge/osx-arm64     Cached
  + liblapack                             3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libllvm14                            14.0.6  h37c5ba8_0               conda-forge/osx-arm64       26MB
  + libnetcdf                             4.8.1  nompi_h996a5af_104       conda-forge/osx-arm64        1MB
  + libnghttp2                           1.47.0  h232270b_1               conda-forge/osx-arm64     Cached
  + libogg                                1.3.4  h27ca646_1               conda-forge/osx-arm64      207kB
  + libopenblas                          0.3.21  openmp_hcb59c3b_1        conda-forge/osx-arm64        4MB
  + libopus                               1.3.1  h27ca646_1               conda-forge/osx-arm64      253kB
  + libpng                               1.6.37  h9b2ccdc_4               conda-forge/osx-arm64      324kB
  + libpq                                  14.5  hb2ab832_0               conda-forge/osx-arm64        3MB
  + libsodium                            1.0.18  h27ca646_1               conda-forge/osx-arm64      325kB
  + libspatialindex                       1.9.3  hbdafb3b_4               conda-forge/osx-arm64      385kB
  + libsqlite                            3.39.2  h2c9beb0_1               conda-forge/osx-arm64     Cached
  + libssh2                              1.10.0  hb80f160_3               conda-forge/osx-arm64     Cached
  + libtiff                               4.4.0  hd6ca8d7_3               conda-forge/osx-arm64      601kB
  + libvorbis                             1.3.7  h9f76cd9_0               conda-forge/osx-arm64      255kB
  + libwebp                               1.2.4  h328b37c_0               conda-forge/osx-arm64       86kB
  + libwebp-base                          1.2.4  h57fd34a_0               conda-forge/osx-arm64      324kB
  + libxcb                                 1.13  h9b22ae9_1004            conda-forge/osx-arm64      353kB
  + libzip                                1.9.2  h96606af_1               conda-forge/osx-arm64      117kB
  + libzlib                              1.2.12  ha287fd2_2               conda-forge/osx-arm64     Cached
  + llvm-openmp                          14.0.4  hd125106_0               conda-forge/osx-arm64      304kB
  + locket                                1.0.0  pyhd8ed1ab_0             conda-forge/noarch           8kB
  + lz4                                   4.0.0  py310h810453e_2          conda-forge/osx-arm64      133kB
  + lz4-c                                 1.9.3  hbdafb3b_1               conda-forge/osx-arm64     Cached
  + markupsafe                            2.1.1  py310hf8d0d8f_1          conda-forge/osx-arm64       22kB
  + matplotlib                            3.5.3  py310hb6292c7_1          conda-forge/osx-arm64        7kB
  + matplotlib-base                       3.5.3  py310h5b2abfc_1          conda-forge/osx-arm64        8MB
  + matplotlib-inline                     0.1.3  pyhd8ed1ab_0             conda-forge/noarch          11kB
  + mccabe                                0.6.1  py_1                     conda-forge/noarch           9kB
  + mistune                               0.8.4  py310he2143c4_1005       conda-forge/osx-arm64       56kB
  + msgpack-python                        1.0.4  py310hd23d0e8_0          conda-forge/osx-arm64       83kB
  + munkres                               1.1.4  pyh9f0ad1d_0             conda-forge/noarch          12kB
  + mypy_extensions                       0.4.3  py310hbe9552e_5          conda-forge/osx-arm64       11kB
  + mysql-common                         8.0.30  haa90c7b_0               conda-forge/osx-arm64        2MB
  + mysql-libs                           8.0.30  h0a1dba2_0               conda-forge/osx-arm64        2MB
  + nbclient                             0.5.13  pyhd8ed1ab_0             conda-forge/noarch          65kB
  + nbconvert                             6.4.5  py310hbe9552e_0          conda-forge/osx-arm64      556kB
  + nbformat                              5.4.0  pyhd8ed1ab_0             conda-forge/noarch         106kB
  + ncurses                                 6.3  h07bb92c_1               conda-forge/osx-arm64     Cached
  + nest-asyncio                          1.5.5  pyhd8ed1ab_0             conda-forge/noarch           9kB
  + netcdf4                               1.6.0  nompi_py310h9d42ac0_101  conda-forge/osx-arm64      446kB
  + nspr                                   4.32  hbdafb3b_1               conda-forge/osx-arm64      244kB
  + nss                                    3.78  h1483a63_0               conda-forge/osx-arm64        2MB
  + numpy                                1.23.2  py310h127c7cf_0          conda-forge/osx-arm64        6MB
  + numpydoc                              1.4.0  pyhd8ed1ab_1             conda-forge/noarch          46kB
  + openjpeg                              2.5.0  h5d4e404_1               conda-forge/osx-arm64      402kB
  + openssl                              1.1.1q  ha287fd2_0               conda-forge/osx-arm64     Cached
  + packaging                              21.3  pyhd8ed1ab_0             conda-forge/noarch          36kB
  + pandas                                1.4.3  py310ha6a5cd6_0          conda-forge/osx-arm64       12MB
  + pandocfilters                         1.5.0  pyhd8ed1ab_0             conda-forge/noarch          12kB
  + parso                                 0.8.3  pyhd8ed1ab_0             conda-forge/noarch          71kB
  + partd                                 1.3.0  pyhd8ed1ab_0             conda-forge/noarch          19kB
  + pathspec                              0.9.0  pyhd8ed1ab_0             conda-forge/noarch          31kB
  + pcre                                   8.45  hbdafb3b_0               conda-forge/osx-arm64      236kB
  + pexpect                               4.8.0  pyh9f0ad1d_2             conda-forge/noarch          48kB
  + pickleshare                           0.7.5  py_1003                  conda-forge/noarch           9kB
  + pillow                                9.2.0  py310h173adc3_2          conda-forge/osx-arm64       47MB
  + pip                                  22.2.2  pyhd8ed1ab_0             conda-forge/noarch           2MB
  + pkgutil-resolve-name                 1.3.10  pyhd8ed1ab_0             conda-forge/noarch           9kB
  + platformdirs                          2.5.2  pyhd8ed1ab_1             conda-forge/noarch          16kB
  + pluggy                                1.0.0  py310hbe9552e_3          conda-forge/osx-arm64       26kB
  + ply                                    3.11  py_1                     conda-forge/noarch          45kB
  + prompt-toolkit                       3.0.30  pyha770c72_0             conda-forge/noarch         260kB
  + psutil                                5.9.1  py310h02f21da_0          conda-forge/osx-arm64      368kB
  + pthread-stubs                           0.4  h27ca646_1001            conda-forge/osx-arm64        6kB
  + ptyprocess                            0.7.0  pyhd3deb0d_0             conda-forge/noarch          17kB
  + pycodestyle                           2.8.0  pyhd8ed1ab_0             conda-forge/noarch          40kB
  + pycparser                              2.21  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + pydocstyle                            6.1.1  pyhd8ed1ab_0             conda-forge/noarch          35kB
  + pyflakes                              2.4.0  pyhd8ed1ab_0             conda-forge/noarch          59kB
  + pygments                             2.13.0  pyhd8ed1ab_0             conda-forge/noarch         840kB
  + pylint                               2.14.5  pyhd8ed1ab_0             conda-forge/noarch         319kB
  + pyls-spyder                           0.4.0  pyhd8ed1ab_0             conda-forge/noarch          10kB
  + pyobjc-core                             8.5  py310hbbccc5d_0          conda-forge/osx-arm64      358kB
  + pyobjc-framework-cocoa                  8.5  py310hbbccc5d_0          conda-forge/osx-arm64      384kB
  + pyobjc-framework-coreservices           8.5  py310h02f21da_1          conda-forge/osx-arm64       42kB
  + pyobjc-framework-fsevents               8.5  py310hbe9552e_1          conda-forge/osx-arm64       15kB
  + pyopenssl                            22.0.0  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + pyparsing                             3.0.9  pyhd8ed1ab_0             conda-forge/noarch          81kB
  + pyqt                                 5.15.7  py310h8775411_0          conda-forge/osx-arm64        5MB
  + pyqt5-sip                           12.11.0  py310hb07a4bc_0          conda-forge/osx-arm64       74kB
  + pyqtwebengine                        5.15.7  py310h8775411_0          conda-forge/osx-arm64      144kB
  + pyrsistent                           0.18.1  py310hf8d0d8f_1          conda-forge/osx-arm64       92kB
  + pysocks                               1.7.1  py310hbe9552e_5          conda-forge/osx-arm64       29kB
  + python                               3.10.5  h71ab1a4_0_cpython       conda-forge/osx-arm64       13MB
  + python-dateutil                       2.8.2  pyhd8ed1ab_0             conda-forge/noarch         246kB
  + python-fastjsonschema                2.16.1  pyhd8ed1ab_0             conda-forge/noarch         249kB
  + python-lsp-black                      1.2.1  pyhd8ed1ab_0             conda-forge/noarch          10kB
  + python-lsp-jsonrpc                    1.0.0  pyhd8ed1ab_0             conda-forge/noarch          11kB
  + python-lsp-server                     1.5.0  pyhd8ed1ab_0             conda-forge/noarch          47kB
  + python-slugify                        6.1.2  pyhd8ed1ab_0             conda-forge/noarch          16kB
  + python.app                              1.4  py310hd86e3df_0          conda-forge/osx-arm64       13kB
  + python_abi                             3.10  2_cp310                  conda-forge/osx-arm64        4kB
  + pytoolconfig                          1.2.2  pyhd8ed1ab_0             conda-forge/noarch          18kB
  + pytz                               2022.2.1  pyhd8ed1ab_0             conda-forge/noarch         229kB
  + pyyaml                                  6.0  py310hf8d0d8f_4          conda-forge/osx-arm64      171kB
  + pyzmq                                23.2.1  py310h666594b_0          conda-forge/osx-arm64      473kB
  + qdarkstyle                            3.0.3  pyhd8ed1ab_0             conda-forge/noarch         320kB
  + qstylizer                             0.2.1  pyhd8ed1ab_0             conda-forge/noarch          16kB
  + qt-main                              5.15.4  haf604a7_2               conda-forge/osx-arm64       68MB
  + qt-webengine                         5.15.4  h43c6231_3               conda-forge/osx-arm64       52MB
  + qtawesome                             1.1.1  pyhd8ed1ab_0             conda-forge/noarch           2MB
  + qtconsole                             5.3.1  pyhd8ed1ab_0             conda-forge/noarch           5kB
  + qtconsole-base                        5.3.1  pyha770c72_0             conda-forge/noarch          93kB
  + qtpy                                  2.2.0  pyhd8ed1ab_0             conda-forge/noarch          50kB
  + readline                              8.1.2  h46ed386_0               conda-forge/osx-arm64     Cached
  + requests                             2.28.1  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + rope                                  1.3.0  pyhd8ed1ab_0             conda-forge/noarch         135kB
  + rtree                                 1.0.0  py310ha3239f5_1          conda-forge/osx-arm64       51kB
  + setuptools                           65.0.1  py310hbe9552e_0          conda-forge/osx-arm64        1MB
  + sip                                   6.6.2  py310hb07a4bc_0          conda-forge/osx-arm64      495kB
  + six                                  1.16.0  pyh6c4a22f_0             conda-forge/noarch        Cached
  + snowballstemmer                       2.2.0  pyhd8ed1ab_0             conda-forge/noarch          59kB
  + sortedcontainers                      2.4.0  pyhd8ed1ab_0             conda-forge/noarch          26kB
  + soupsieve                       2.3.2.post1  pyhd8ed1ab_0             conda-forge/noarch          35kB
  + sphinx                                5.1.1  pyhd8ed1ab_1             conda-forge/noarch           2MB
  + sphinxcontrib-applehelp               1.0.2  py_0                     conda-forge/noarch          29kB
  + sphinxcontrib-devhelp                 1.0.2  py_0                     conda-forge/noarch          23kB
  + sphinxcontrib-htmlhelp                2.0.0  pyhd8ed1ab_0             conda-forge/noarch          32kB
  + sphinxcontrib-jsmath                  1.0.1  py_0                     conda-forge/noarch           7kB
  + sphinxcontrib-qthelp                  1.0.3  py_0                     conda-forge/noarch          26kB
  + sphinxcontrib-serializinghtml         1.1.5  pyhd8ed1ab_2             conda-forge/noarch          28kB
  + spyder                                5.3.2  py310hbe9552e_0          conda-forge/osx-arm64       13MB
  + spyder-kernels                        2.3.2  py310hbe9552e_0          conda-forge/osx-arm64      112kB
  + sqlite                               3.39.2  h40dfcc0_1               conda-forge/osx-arm64     Cached
  + tblib                                 1.7.0  pyhd8ed1ab_0             conda-forge/noarch          15kB
  + testpath                              0.6.0  pyhd8ed1ab_0             conda-forge/noarch          87kB
  + text-unidecode                          1.3  py_0                     conda-forge/noarch          70kB
  + textdistance                          4.3.0  pyhd8ed1ab_0             conda-forge/noarch          27kB
  + three-merge                           0.1.1  pyh9f0ad1d_0             conda-forge/noarch           9kB
  + tinycss2                              1.1.1  pyhd8ed1ab_0             conda-forge/noarch          23kB
  + tk                                   8.6.12  he1e0b03_0               conda-forge/osx-arm64     Cached
  + toml                                 0.10.2  pyhd8ed1ab_0             conda-forge/noarch          18kB
  + tomli                                 2.0.1  pyhd8ed1ab_0             conda-forge/noarch          16kB
  + tomlkit                              0.11.4  pyha770c72_0             conda-forge/noarch          33kB
  + toolz                                0.12.0  pyhd8ed1ab_0             conda-forge/noarch          49kB
  + tornado                                 6.1  py310hf8d0d8f_3          conda-forge/osx-arm64      674kB
  + traitlets                             5.3.0  pyhd8ed1ab_0             conda-forge/noarch          87kB
  + typed-ast                             1.5.4  py310h02f21da_0          conda-forge/osx-arm64      185kB
  + typing                             3.10.0.0  pyhd8ed1ab_0             conda-forge/noarch           7kB
  + typing-extensions                     4.3.0  hd8ed1ab_0               conda-forge/noarch           8kB
  + typing_extensions                     4.3.0  pyha770c72_0             conda-forge/noarch          29kB
  + tzdata                                2022c  h191b570_0               conda-forge/noarch        Cached
  + ujson                                 5.3.0  py310hb07a4bc_0          conda-forge/osx-arm64       49kB
  + unicodedata2                         14.0.0  py310hf8d0d8f_1          conda-forge/osx-arm64      509kB
  + unidecode                             1.3.4  pyhd8ed1ab_0             conda-forge/noarch         159kB
  + urllib3                             1.26.11  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + watchdog                              2.1.9  py310h02f21da_0          conda-forge/osx-arm64      111kB
  + wcwidth                               0.2.5  pyh9f0ad1d_2             conda-forge/noarch          34kB
  + webencodings                          0.5.1  py_1                     conda-forge/noarch          12kB
  + whatthepatch                          1.0.2  pyhd8ed1ab_0             conda-forge/noarch          15kB
  + wheel                                0.37.1  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + wrapt                                1.14.1  py310h02f21da_0          conda-forge/osx-arm64       49kB
  + wurlitzer                             3.0.2  pyhd8ed1ab_2             conda-forge/noarch          11kB
  + xarray                             2022.6.0  pyhd8ed1ab_1             conda-forge/noarch         714kB
  + xcdat                                 0.2.0  pyhd8ed1ab_0             conda-forge/noarch          39kB
  + xorg-libxau                           1.0.9  h27ca646_0               conda-forge/osx-arm64       12kB
  + xorg-libxdmcp                         1.1.3  h27ca646_0               conda-forge/osx-arm64       18kB
  + xz                                    5.2.6  h57fd34a_0               conda-forge/osx-arm64     Cached
  + yaml                                  0.2.5  h3422bc3_2               conda-forge/osx-arm64     Cached
  + yapf                                 0.32.0  pyhd8ed1ab_0             conda-forge/noarch         132kB
  + zeromq                                4.3.4  hbdafb3b_1               conda-forge/osx-arm64      318kB
  + zict                                  2.2.0  pyhd8ed1ab_0             conda-forge/noarch          20kB
  + zipp                                  3.8.1  pyhd8ed1ab_0             conda-forge/noarch          13kB
  + zstd                                  1.5.2  h68b4297_4               conda-forge/osx-arm64     Cached

  Summary:

  Install: 269 packages

  Total download: 348MB

@durack1
Copy link
Collaborator Author

durack1 commented Aug 17, 2022

And identical if I reduce the additionals, leaving just a vanilla xcdat install:

(base) $ mamba create --dry-run -n xcd -c conda-forge xcdat

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.24.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['xcdat']

conda-forge/osx-arm64                                       Using cache
conda-forge/noarch                                          Using cache
Transaction

  Prefix: ~/mambaforge/envs/xcd

  Updating specs:

   - xcdat


  Package                   Version  Build                    Channel                     Size
────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
────────────────────────────────────────────────────────────────────────────────────────────────

  + bokeh                     2.4.3  pyhd8ed1ab_3             conda-forge/noarch          14MB
  + brotlipy                  0.7.0  py310hf8d0d8f_1004       conda-forge/osx-arm64      349kB
  + bzip2                     1.0.8  h3422bc3_4               conda-forge/osx-arm64     Cached
  + c-ares                   1.18.1  h3422bc3_0               conda-forge/osx-arm64     Cached
  + ca-certificates       2022.6.15  h4653dfc_0               conda-forge/osx-arm64     Cached
  + certifi               2022.6.15  py310hbe9552e_0          conda-forge/osx-arm64      159kB
  + cf_xarray                 0.7.4  pyhd8ed1ab_0             conda-forge/noarch         253kB
  + cffi                     1.15.1  py310he00a5c5_0          conda-forge/osx-arm64      227kB
  + cftime                    1.6.1  py310h611a7d1_0          conda-forge/osx-arm64      196kB
  + click                     8.1.3  py310hbe9552e_0          conda-forge/osx-arm64      152kB
  + cloudpickle               2.1.0  pyhd8ed1ab_0             conda-forge/noarch          25kB
  + cryptography             37.0.4  py310h94bb23d_0          conda-forge/osx-arm64        1MB
  + curl                     7.83.1  h2fcd78c_0               conda-forge/osx-arm64      148kB
  + cytoolz                  0.12.0  py310h02f21da_0          conda-forge/osx-arm64      355kB
  + dask                   2022.8.0  pyhd8ed1ab_1             conda-forge/noarch           5kB
  + dask-core              2022.8.0  pyhd8ed1ab_0             conda-forge/noarch         864kB
  + distributed            2022.8.0  pyhd8ed1ab_1             conda-forge/noarch         760kB
  + freetype                 2.12.1  hd633e50_0               conda-forge/osx-arm64      873kB
  + fsspec                 2022.7.1  pyhd8ed1ab_0             conda-forge/noarch          98kB
  + hdf4                     4.2.15  hc683e77_4               conda-forge/osx-arm64      928kB
  + hdf5                     1.12.2  nompi_h8968d4b_100       conda-forge/osx-arm64        3MB
  + heapdict                  1.0.1  py_0                     conda-forge/noarch           7kB
  + idna                        3.3  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + jinja2                    3.1.2  pyhd8ed1ab_1             conda-forge/noarch         101kB
  + jpeg                         9e  he4db4b2_2               conda-forge/osx-arm64      266kB
  + krb5                     1.19.3  hf9b2bbe_0               conda-forge/osx-arm64     Cached
  + lcms2                      2.12  had6a04f_0               conda-forge/osx-arm64      378kB
  + lerc                      4.0.0  h9a09cb3_0               conda-forge/osx-arm64      216kB
  + libblas                   3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libcblas                  3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libcurl                  7.83.1  h2fcd78c_0               conda-forge/osx-arm64     Cached
  + libcxx                   14.0.6  h04bba0f_0               conda-forge/osx-arm64     Cached
  + libdeflate                 1.13  h57fd34a_0               conda-forge/osx-arm64       61kB
  + libedit            3.1.20191231  hc8eb9b7_2               conda-forge/osx-arm64     Cached
  + libev                      4.33  h642e427_1               conda-forge/osx-arm64     Cached
  + libffi                    3.4.2  h3422bc3_5               conda-forge/osx-arm64     Cached
  + libgfortran          5.0.0.dev0  11_0_1_hf114ba7_23       conda-forge/osx-arm64       20kB
  + libgfortran5        11.0.1.dev0  hf114ba7_23              conda-forge/osx-arm64      923kB
  + liblapack                 3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libnetcdf                 4.8.1  nompi_h996a5af_104       conda-forge/osx-arm64        1MB
  + libnghttp2               1.47.0  h232270b_1               conda-forge/osx-arm64     Cached
  + libopenblas              0.3.21  openmp_hcb59c3b_1        conda-forge/osx-arm64        4MB
  + libpng                   1.6.37  h9b2ccdc_4               conda-forge/osx-arm64      324kB
  + libsqlite                3.39.2  h2c9beb0_1               conda-forge/osx-arm64     Cached
  + libssh2                  1.10.0  hb80f160_3               conda-forge/osx-arm64     Cached
  + libtiff                   4.4.0  hd6ca8d7_3               conda-forge/osx-arm64      601kB
  + libwebp-base              1.2.4  h57fd34a_0               conda-forge/osx-arm64      324kB
  + libxcb                     1.13  h9b22ae9_1004            conda-forge/osx-arm64      353kB
  + libzip                    1.9.2  h96606af_1               conda-forge/osx-arm64      117kB
  + libzlib                  1.2.12  ha287fd2_2               conda-forge/osx-arm64     Cached
  + llvm-openmp              14.0.4  hd125106_0               conda-forge/osx-arm64      304kB
  + locket                    1.0.0  pyhd8ed1ab_0             conda-forge/noarch           8kB
  + lz4                       4.0.0  py310h810453e_2          conda-forge/osx-arm64      133kB
  + lz4-c                     1.9.3  hbdafb3b_1               conda-forge/osx-arm64     Cached
  + markupsafe                2.1.1  py310hf8d0d8f_1          conda-forge/osx-arm64       22kB
  + msgpack-python            1.0.4  py310hd23d0e8_0          conda-forge/osx-arm64       83kB
  + ncurses                     6.3  h07bb92c_1               conda-forge/osx-arm64     Cached
  + netcdf4                   1.6.0  nompi_py310h9d42ac0_101  conda-forge/osx-arm64      446kB
  + numpy                    1.23.2  py310h127c7cf_0          conda-forge/osx-arm64        6MB
  + openjpeg                  2.5.0  h5d4e404_1               conda-forge/osx-arm64      402kB
  + openssl                  1.1.1q  ha287fd2_0               conda-forge/osx-arm64     Cached
  + packaging                  21.3  pyhd8ed1ab_0             conda-forge/noarch          36kB
  + pandas                    1.4.3  py310ha6a5cd6_0          conda-forge/osx-arm64       12MB
  + partd                     1.3.0  pyhd8ed1ab_0             conda-forge/noarch          19kB
  + pillow                    9.2.0  py310h173adc3_2          conda-forge/osx-arm64       47MB
  + pip                      22.2.2  pyhd8ed1ab_0             conda-forge/noarch           2MB
  + psutil                    5.9.1  py310h02f21da_0          conda-forge/osx-arm64      368kB
  + pthread-stubs               0.4  h27ca646_1001            conda-forge/osx-arm64        6kB
  + pycparser                  2.21  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + pyopenssl                22.0.0  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + pyparsing                 3.0.9  pyhd8ed1ab_0             conda-forge/noarch          81kB
  + pysocks                   1.7.1  py310hbe9552e_5          conda-forge/osx-arm64       29kB
  + python                   3.10.5  h71ab1a4_0_cpython       conda-forge/osx-arm64       13MB
  + python-dateutil           2.8.2  pyhd8ed1ab_0             conda-forge/noarch         246kB
  + python_abi                 3.10  2_cp310                  conda-forge/osx-arm64        4kB
  + pytz                   2022.2.1  pyhd8ed1ab_0             conda-forge/noarch         229kB
  + pyyaml                      6.0  py310hf8d0d8f_4          conda-forge/osx-arm64      171kB
  + readline                  8.1.2  h46ed386_0               conda-forge/osx-arm64     Cached
  + setuptools               65.0.1  py310hbe9552e_0          conda-forge/osx-arm64        1MB
  + six                      1.16.0  pyh6c4a22f_0             conda-forge/noarch        Cached
  + sortedcontainers          2.4.0  pyhd8ed1ab_0             conda-forge/noarch          26kB
  + sqlite                   3.39.2  h40dfcc0_1               conda-forge/osx-arm64     Cached
  + tblib                     1.7.0  pyhd8ed1ab_0             conda-forge/noarch          15kB
  + tk                       8.6.12  he1e0b03_0               conda-forge/osx-arm64     Cached
  + toolz                    0.12.0  pyhd8ed1ab_0             conda-forge/noarch          49kB
  + tornado                     6.1  py310hf8d0d8f_3          conda-forge/osx-arm64      674kB
  + typing_extensions         4.3.0  pyha770c72_0             conda-forge/noarch          29kB
  + tzdata                    2022c  h191b570_0               conda-forge/noarch        Cached
  + urllib3                 1.26.11  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + wheel                    0.37.1  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + xarray                 2022.6.0  pyhd8ed1ab_1             conda-forge/noarch         714kB
  + xcdat                     0.2.0  pyhd8ed1ab_0             conda-forge/noarch          39kB
  + xorg-libxau               1.0.9  h27ca646_0               conda-forge/osx-arm64       12kB
  + xorg-libxdmcp             1.1.3  h27ca646_0               conda-forge/osx-arm64       18kB
  + xz                        5.2.6  h57fd34a_0               conda-forge/osx-arm64     Cached
  + yaml                      0.2.5  h3422bc3_2               conda-forge/osx-arm64     Cached
  + zict                      2.2.0  pyhd8ed1ab_0             conda-forge/noarch          20kB
  + zstd                      1.5.2  h68b4297_4               conda-forge/osx-arm64     Cached

  Summary:

  Install: 98 packages

  Total download: 119MB

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Aug 17, 2022

Hmm, not sure exactly why xcdat 0.2.0 is getting pulled, even after clean and fresh runs.

Let's try this again after 0.3.1 is released by the end of this week.

If we find that 0.2.0 is installing on osx-arm64, while 0.3.1 is installing on other platforms, then it might be rooted in how conda/mamba is resolving the environment with osx-arm64 (the issues I previously cited). If this is the case, then we can update the docs for osx-arm64 users to specify a version such as xcdat=0.3.0 as a temporary workaround.

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Aug 17, 2022

then it might be rooted in how conda/mamba is resolving the environment with osx-arm64 (the issues I previously cited)

Here's a useful comment that might give us clues:
napari/napari#4891 (comment)

I think it has to do with the ongoing openssl transition at conda-forge. See how installing napari forces a downgrade from 3.x to 1.x:

openssl                                  3.0.5-h7aea29f_1 --> 1.1.1q-ha287fd2_0

Sounds very similar to these:

* [Old version of a package installed instead of the latest conda/conda-libmamba-solver#64 (comment)](https://github.com/conda/conda-libmamba-solver/issues/64)

* [Drastically different version picked when order changed mamba-org/mamba#1581 (comment)](https://github.com/mamba-org/mamba/issues/1581#issuecomment-1068845806)

... and it all revolves to this PR:

* [Rebuild for openssl3 conda-forge/cryptography-feedstock#98](https://github.com/conda-forge/cryptography-feedstock/pull/98)

I know, I know, not very useful for the end-user 😬 My only advice now is to tell the solver what you want more explicitly:

$ conda create -n napari-env python=3.9 napari=0.4.16

This will even solve faster!

I noticed openssl is an implicit sub-dependency for xcdat:
+ openssl 1.1.1q ha287fd2_0 conda-forge/osx-arm64 Cached

@tomvothecoder tomvothecoder removed this from v0.3.1 Aug 17, 2022
@tomvothecoder tomvothecoder moved this to Todo in v0.3.2 Aug 17, 2022
@durack1
Copy link
Collaborator Author

durack1 commented Aug 17, 2022

@tomvothecoder, this is a real example of why minimizing your dependencies is a very good idea. Will await for 0.3.1 to land and try again.

Attempting to forcing openssl=3.0.5 doesn't seem to solve it, so some other package must be causing some issues - it would be useful to ascertain which package is well off being the latest release

$ mamba create --dry-run -n xcd -c conda-forge xcdat openssl=3.0.5

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.24.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['xcdat', 'openssl=3.0.5']

conda-forge/osx-arm64                                3.1MB @   3.1MB/s  1.1s
conda-forge/noarch                                   9.0MB @   3.8MB/s  2.5s
Transaction

  Prefix: ~/mambaforge/envs/xcd

  Updating specs:

   - xcdat
   - openssl=3.0.5


  Package                   Version  Build                    Channel                     Size
────────────────────────────────────────────────────────────────────────────────────────────────
  Install:
────────────────────────────────────────────────────────────────────────────────────────────────

  + bokeh                     2.4.3  pyhd8ed1ab_3             conda-forge/noarch          14MB
  + bzip2                     1.0.8  h3422bc3_4               conda-forge/osx-arm64     Cached
  + c-ares                   1.18.1  h3422bc3_0               conda-forge/osx-arm64     Cached
  + ca-certificates       2022.6.15  h4653dfc_0               conda-forge/osx-arm64     Cached
  + cf_xarray                 0.7.4  pyhd8ed1ab_0             conda-forge/noarch         253kB
  + cftime                    1.6.1  py310h611a7d1_0          conda-forge/osx-arm64      196kB
  + click                     8.0.4  py310hbe9552e_0          conda-forge/osx-arm64      153kB
  + cloudpickle               2.1.0  pyhd8ed1ab_0             conda-forge/noarch          25kB
  + curl                     7.83.1  h7965298_0               conda-forge/osx-arm64      148kB
  + cytoolz                  0.12.0  py310h02f21da_0          conda-forge/osx-arm64      355kB
  + dask                   2022.3.0  pyhd8ed1ab_1             conda-forge/noarch           5kB
  + dask-core              2022.3.0  pyhd8ed1ab_0             conda-forge/noarch         832kB
  + distributed            2022.3.0  pyhd8ed1ab_0             conda-forge/noarch         730kB
  + freetype                 2.12.1  hd633e50_0               conda-forge/osx-arm64      873kB
  + fsspec                 2022.7.1  pyhd8ed1ab_0             conda-forge/noarch          98kB
  + hdf4                     4.2.15  hc683e77_4               conda-forge/osx-arm64      928kB
  + hdf5                     1.12.2  nompi_h33dac16_100       conda-forge/osx-arm64        3MB
  + heapdict                  1.0.1  py_0                     conda-forge/noarch           7kB
  + jinja2                    3.1.2  pyhd8ed1ab_1             conda-forge/noarch         101kB
  + jpeg                         9e  he4db4b2_2               conda-forge/osx-arm64      266kB
  + krb5                     1.19.3  he492e65_0               conda-forge/osx-arm64        1MB
  + lcms2                      2.12  had6a04f_0               conda-forge/osx-arm64      378kB
  + lerc                      4.0.0  h9a09cb3_0               conda-forge/osx-arm64      216kB
  + libblas                   3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libcblas                  3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libcurl                  7.83.1  h7965298_0               conda-forge/osx-arm64      309kB
  + libcxx                   14.0.6  h04bba0f_0               conda-forge/osx-arm64     Cached
  + libdeflate                 1.13  h57fd34a_0               conda-forge/osx-arm64       61kB
  + libedit            3.1.20191231  hc8eb9b7_2               conda-forge/osx-arm64     Cached
  + libev                      4.33  h642e427_1               conda-forge/osx-arm64     Cached
  + libffi                    3.4.2  h3422bc3_5               conda-forge/osx-arm64     Cached
  + libgfortran          5.0.0.dev0  11_0_1_hf114ba7_23       conda-forge/osx-arm64       20kB
  + libgfortran5        11.0.1.dev0  hf114ba7_23              conda-forge/osx-arm64      923kB
  + liblapack                 3.9.0  16_osxarm64_openblas     conda-forge/osx-arm64       13kB
  + libnetcdf                 4.8.1  nompi_h996a5af_104       conda-forge/osx-arm64        1MB
  + libnghttp2               1.47.0  h519802c_1               conda-forge/osx-arm64      828kB
  + libopenblas              0.3.21  openmp_hcb59c3b_1        conda-forge/osx-arm64        4MB
  + libpng                   1.6.37  h9b2ccdc_4               conda-forge/osx-arm64      324kB
  + libsqlite                3.39.2  h2c9beb0_1               conda-forge/osx-arm64     Cached
  + libssh2                  1.10.0  h7a5bd25_3               conda-forge/osx-arm64      223kB
  + libtiff                   4.4.0  hd6ca8d7_3               conda-forge/osx-arm64      601kB
  + libwebp-base              1.2.4  h57fd34a_0               conda-forge/osx-arm64      324kB
  + libxcb                     1.13  h9b22ae9_1004            conda-forge/osx-arm64      353kB
  + libzip                    1.9.2  h76ab92c_1               conda-forge/osx-arm64      117kB
  + libzlib                  1.2.12  ha287fd2_2               conda-forge/osx-arm64     Cached
  + llvm-openmp              14.0.4  hd125106_0               conda-forge/osx-arm64      304kB
  + locket                    1.0.0  pyhd8ed1ab_0             conda-forge/noarch           8kB
  + lz4                       4.0.0  py310h810453e_2          conda-forge/osx-arm64      133kB
  + lz4-c                     1.9.3  hbdafb3b_1               conda-forge/osx-arm64     Cached
  + markupsafe                2.1.1  py310hf8d0d8f_1          conda-forge/osx-arm64       22kB
  + msgpack-python            1.0.4  py310hd23d0e8_0          conda-forge/osx-arm64       83kB
  + ncurses                     6.3  h07bb92c_1               conda-forge/osx-arm64     Cached
  + netcdf4                   1.6.0  nompi_py310h9d42ac0_101  conda-forge/osx-arm64      446kB
  + numpy                    1.23.2  py310h127c7cf_0          conda-forge/osx-arm64        6MB
  + openjpeg                  2.5.0  h5d4e404_1               conda-forge/osx-arm64      402kB
  + openssl                   3.0.5  h7aea29f_1               conda-forge/osx-arm64        2MB
  + packaging                  21.3  pyhd8ed1ab_0             conda-forge/noarch          36kB
  + pandas                    1.4.3  py310ha6a5cd6_0          conda-forge/osx-arm64       12MB
  + partd                     1.3.0  pyhd8ed1ab_0             conda-forge/noarch          19kB
  + pillow                    9.2.0  py310h173adc3_2          conda-forge/osx-arm64       47MB
  + pip                      22.2.2  pyhd8ed1ab_0             conda-forge/noarch           2MB
  + psutil                    5.9.1  py310h02f21da_0          conda-forge/osx-arm64      368kB
  + pthread-stubs               0.4  h27ca646_1001            conda-forge/osx-arm64        6kB
  + pyparsing                 3.0.9  pyhd8ed1ab_0             conda-forge/noarch          81kB
  + python                   3.10.5  h4eee789_0_cpython       conda-forge/osx-arm64       13MB
  + python-dateutil           2.8.2  pyhd8ed1ab_0             conda-forge/noarch         246kB
  + python_abi                 3.10  2_cp310                  conda-forge/osx-arm64        4kB
  + pytz                   2022.2.1  pyhd8ed1ab_0             conda-forge/noarch         229kB
  + pyyaml                      6.0  py310hf8d0d8f_4          conda-forge/osx-arm64      171kB
  + readline                  8.1.2  h46ed386_0               conda-forge/osx-arm64     Cached
  + setuptools               65.0.2  py310hbe9552e_0          conda-forge/osx-arm64        1MB
  + six                      1.16.0  pyh6c4a22f_0             conda-forge/noarch        Cached
  + sortedcontainers          2.4.0  pyhd8ed1ab_0             conda-forge/noarch          26kB
  + sqlite                   3.39.2  h40dfcc0_1               conda-forge/osx-arm64     Cached
  + tblib                     1.7.0  pyhd8ed1ab_0             conda-forge/noarch          15kB
  + tk                       8.6.12  he1e0b03_0               conda-forge/osx-arm64     Cached
  + toolz                    0.12.0  pyhd8ed1ab_0             conda-forge/noarch          49kB
  + tornado                     6.1  py310hf8d0d8f_3          conda-forge/osx-arm64      674kB
  + typing_extensions         4.3.0  pyha770c72_0             conda-forge/noarch          29kB
  + tzdata                    2022c  h191b570_0               conda-forge/noarch        Cached
  + wheel                    0.37.1  pyhd8ed1ab_0             conda-forge/noarch        Cached
  + xarray                 2022.6.0  pyhd8ed1ab_1             conda-forge/noarch         714kB
  + xcdat                     0.2.0  pyhd8ed1ab_0             conda-forge/noarch          39kB
  + xorg-libxau               1.0.9  h27ca646_0               conda-forge/osx-arm64       12kB
  + xorg-libxdmcp             1.1.3  h27ca646_0               conda-forge/osx-arm64       18kB
  + xz                        5.2.6  h57fd34a_0               conda-forge/osx-arm64     Cached
  + yaml                      0.2.5  h3422bc3_2               conda-forge/osx-arm64     Cached
  + zict                      2.2.0  pyhd8ed1ab_0             conda-forge/noarch          20kB
  + zstd                      1.5.2  h8128057_4               conda-forge/osx-arm64      387kB

  Summary:

  Install: 89 packages

  Total download: 122MB

@durack1
Copy link
Collaborator Author

durack1 commented Aug 17, 2022

Looks like I may have found it, it seems the esmpy 8.2.0 is not available for osx-arm64 and therefore, causes problems. I suppose you didn't define the dependencies for 0.2.0 as tightly as the 0.3.x release?

$ mamba create --dry-run -n xcd -c conda-forge xcdat=0.3.0 openssl=3.0.5

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.24.0) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['xcdat=0.3.0', 'openssl=3.0.5']

conda-forge/osx-arm64                                       Using cache
conda-forge/noarch                                          Using cache
Encountered problems while solving:
  - nothing provides esmpy needed by xcdat-0.3.0-pyhd8ed1ab_0

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Aug 18, 2022

@tomvothecoder, this is a real example of why minimizing your dependencies is a very good idea. Will await for 0.3.1 to land and try again.

Totally agree. We're focusing on keeping dependencies directly used by xcdat in the conda recipe (e.g., not including matplotlib) and only introducing dependencies that are absolutely needed if the Python standard library doesn't suffice.

Looks like I may have found it, it seems the esmpy 8.2.0 is not available for osx-arm64 and therefore, causes problems. I suppose you didn't define the dependencies for 0.2.0 as tightly as the 0.3.x release?

Good job finding out the exact problem. Mamba is much better than conda for these specific reasons.

0.2.0 doesn't have horizontal regridding, so esmpy, xesmf, and numba weren't included: https://github.com/conda-forge/xcdat-feedstock/blob/5071957bd0d9c3a80ca20f73924722c4c94d877a/recipe/meta.yaml#L17-L29

0.3.x has these dependencies: https://github.com/conda-forge/xcdat-feedstock/blob/701b3fca311be2a170c56afc4c37e2e9bb1f3c68/recipe/meta.yaml#L17-L33

@xylar mentioned issues with esmpy with E3SM-unified in the past. I noticed esmpy doesn't offer a noarch build on their anaconda page: https://anaconda.org/conda-forge/esmpy.

@jasonb5 might have an idea on how to handle this? The solution I currently see is for esmpy to release noarch, which means we have to open up a GitHub issue there to bug them.

@tomvothecoder
Copy link
Collaborator

Well well, there's already an issue open here and @xylar is apart of the discussion: conda-forge/esmpy-feedstock#55

@xylar
Copy link

xylar commented Aug 18, 2022

@jasonb5 and @tomvothecoder, I've had no luck getting ESMF to build for OSX ARM and haven't got much help on that:
conda-forge/esmf-feedstock#75
It's not really a high priority for anything I'm working directly on so I haven't really dug into it but building ESMF is generally pretty painful. If either of you or people you know have the necessary expertise, xarray folks would also appreciate this!

esmpy needs esmf to be working so that's at least part of what's holding up that package. esmpy definitely can't be noarch since it's not a pure python package but even if it were, esmf would still be the problem.

@tomvothecoder tomvothecoder moved this from Todo to In Progress in v0.3.2 Aug 18, 2022
@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Aug 19, 2022

I appreciate the follow up @xylar. I'm glad to see there's an issue and PR open for ESMF to add OSX ARM.

Unfortunately, I don't think anyone on the xCDAT team has expertise for building on OSX ARM.

Generally speaking, I expect most folks to work on other platforms. We'll have to update our docs to mention that OSX ARM is not currently supported due to ESMF for the time being.

@pochedls
Copy link
Collaborator

pochedls commented Aug 19, 2022

@tomvothecoder - I think the action to update the docs makes sense. One possible path forward is to make xesmf an optional dependency and to exclude .regridding if xesmf isn't in the environment. I'm not sure how complicated this is (or if .regridding functions are used in other xcdat modules).

@tomvothecoder
Copy link
Collaborator

tomvothecoder commented Aug 19, 2022

@pochedls I thought of that as an alternative option too.

I think the typical convention (which I think xarray does with their optional dependencies) is something like:

regridder.py

try:
	import xesmf
except:
	raise ImportError('`xesmf` is not installed in this environment')

If we want to be platform specific instead, we can do something like

if platform == "osx-arm64"
  raise ImportError('xesmf' is not supported on osx-arm64 at this time, which is required for regridding')
else:
   import xesmf

Having this implemented alongside conda/conda#326 makes sense.

@xylar
Copy link

xylar commented Aug 19, 2022

@tomvothecoder, I would suggest the former over the latter. If you have something about a specific platform, you have to modify both the xcdat code and the conda-forge recipe (i.e. a new release) to add support for ESMF under OSX ARM. If you just try xesmf and raise and exception, when OSX ARM support becomes available under any platform that doesn't currently support it, all you have to do is make a new build of the conda-forge recipe that now includes xesmf as a dependency on that platform.

@pochedls
Copy link
Collaborator

This sounds good, though shouldn't we log a warning rather than throw an exception. I think the exception would prevent import xcdat.

@durack1
Copy link
Collaborator Author

durack1 commented Aug 19, 2022

@tomvothecoder as an M1 guinea pig, happy to test out some PRs here

@pochedls
Copy link
Collaborator

@durack1 – would you be willing to attempt to fix this (based on the above conversation) and create a PR?

@durack1
Copy link
Collaborator Author

durack1 commented Aug 19, 2022

@pochedls this makes sense, I'm the only one on M1 at the moment (is that right?). For a complete interim solution, we'll also need to solve the conda dependency issue, that will require esmpy to be defined as an optional dependency to be defined, right @tomvothecoder?

@durack1
Copy link
Collaborator Author

durack1 commented Aug 31, 2022

@pochedls I had tried variations including run_constrained which gives me:

$ mamba env create --dry-run -f conda-env/dev.yml 

EnvironmentSectionNotValid: The following section on '~/sync/git/xcdat/conda-env/dev.yml' is invalid and will be ignored:
 - run_constrained

But it seems to offer to generate an env, so I'll give this a try, whilst noting the packageVersions.txt

@pochedls
Copy link
Collaborator

I don't see dependencies in the conda-forge recipe documentation. I wonder if we need a section like this:

requirements:
  run_constrained:
    - esmpy=8.2.0

@durack1
Copy link
Collaborator Author

durack1 commented Aug 31, 2022

@pochedls that just moves the error

EnvironmentSectionNotValid: The following section on '~/sync/git/xcdat/conda-env/dev.yml' is invalid and will be ignored:
 - requirements

I've got an env free of esmpy, xesmf and pandoc so can continue, with cleanup after the other changes are in place (if there is a way to resolve the problem across platforms)

@durack1
Copy link
Collaborator Author

durack1 commented Sep 1, 2022

@jaimergp I noted you provided some guidance upstream at conda-forge/esmf-feedstock#75 (review), I'm wondering if you can provide some guidance here. I have the following in a conda.yml:

name: xcdat_dev_noEsmpy1
channels:
    - conda-forge
    - defaults
dependencies:
    - python=3.9.13
    - pip=22.2.2
    - cf_xarray=0.7.4
    - cftime=1.6.1
    - dask=2022.8.0
    #- esmpy=8.2.0  # [build_platform != 'osx-arm64']
    - esmpy=8.2.0  # [osx-arm64]
...

What I am trying to do, is create the conda env WITHOUT esmpy if the host is osx-arm64, but keep this as a dependency for the other platforms that have support linux-64, osx-64, nothing that I have tried seems to work - is there some magic syntax you can share - or better point me to some doc that defines this?

I have tried adding the block:

requirements:
    # per https://docs.conda.io/projects/conda-build/en/latest/resources/define-metadata.html#run-constrained
    run_constrained:
        - esmpy=8.2.0 # https://github.com/conda-forge/esmpy-feedstock/issues/55
        ...

This solves me for osx-arm64, but also ensures that for the platforms that are supported, esmpy is also not installed

@jaimergp
Copy link

jaimergp commented Sep 1, 2022

You could publish an empty osx-arm64 shim with the just the metadata to a custom channel with higher priority.

Or force remove esmpy after the solve :)

@durack1
Copy link
Collaborator Author

durack1 commented Sep 1, 2022

You could publish an empty osx-arm64 shim with the just the metadata to a custom channel with higher priority.

Ouch, this is sounding like real work - I was hoping I could deal with this trivially using an additional arg in the *.yml to note it's an optional dependency. I'm guessing that such functionality, while discussed numerous places before e.g. conda/conda#3299, conda/conda#7502, conda/conda#8089, etc, it's presently vapor ware.

Or force remove esmpy after the solve :)

Never thought this was an option, is there an example you could point me to?

@jaimergp
Copy link

jaimergp commented Sep 1, 2022

Let's say you created your env with:

$ conda env create -f environment.yml -n my_env
$ conda activate my_env

You can do:

$ conda remove --force esmpy

That's it. I don't think selectors or run_constrained work in env YAMLs and it might be being silently ignored. These are meta.yaml features only known to conda-build.


Sometimes it's just easier to maintain separate env YAMLs, with some scripting on top if you want to keep things tidy (e.g. have a Jinja template you pre-process before feeding the "rendered" YAML to conda-env).

@jaimergp
Copy link

jaimergp commented Sep 1, 2022

I misunderstood and I thought you needed a solution without esmfpy, but the problem is that the solution requires esmfpy and it's not available for the platform. In these cases, mocking up the package with an empty one is the hacky but "easy" solution - but for individual cases only, I'd say!

The other solution is at the packaging level, where esmf(py) is deemed an "optional" dependency (with soft imports, or something), using run_constrained to mark what versions are compatible in the event the user ends up installing it (potentially following the advice given by the soft-import). For example:

Let's say xcdat changes the esmfpy import to:

try:
  import esmfpy
  HAS_ESMFPY = True
except ImportError as exc:
  HAS_ESMFPY = False
  print(f"Could not import 'esmfpy': {exc}\n. Consider installing it with 'conda install -c conda-forge::esmfpy'")

xcdat's recipe meta.yaml would have change their requirements from:

...
requirements:
  ...
  run:
    ...
    - esmfpy >=0.8

To:

...
requirements:
  ...
  run:
    ...
  run_constrained:
    - esmfpy >=0.8

But you cannot do this at user-install level. It has to be done during packaging, in the conda-forge feedstock.

@xylar
Copy link

xylar commented Sep 1, 2022

@durack1, referring to #315 (comment), I think I misunderstood what the purpose of your YAML file was. I thought it was a meta.yaml that defines a conda recipe. These have selectors like I suggested.

You have a yaml file for the environment. In that case, I think you're going to either have to have a yaml file per platform or you're going to have to do the templating yourself using jinja. I don't think all this run_constrained stuff applies to the type of yaml file you're creating. That's also specifically for a conda recipe (meta.yaml) as I understand it.

@durack1
Copy link
Collaborator Author

durack1 commented Sep 1, 2022

@jaimergp thanks for the tips!

You have a yaml file for the environment. In that case, I think you're going to either have to have a yaml file per platform or you're going to have to do the templating yourself using jinja. I don't think all this run_constrained stuff applies to the type of yaml file you're creating. That's also specifically for a conda recipe (meta.yaml) as I understand it.

@xylar thanks, this is where I think I was ending up. It does seem that progress is being made with esmpy 8.4.0 appears to be in beta and so maybe it's just a matter of waiting a short while and the osx-arm64 not package available problem will disappear.

In my internet travels, I also found an interesting case of multiple conda installs side-by-side. Following this path would allow for an osx-64 and osx-arm64 install to co-exist: https://betterprogramming.pub/switching-between-multiple-conda-distributions-on-macos-b78b6b21720

@tomvothecoder
Copy link
Collaborator

Thank you @jaimergp and @xylar for the help!

@durack1 I'm happy to see you now have a path forward. I'll respond to your comment here.

@durack1
Copy link
Collaborator Author

durack1 commented Sep 12, 2022

This issue is also applicable to windows which will not have esmpy support any time soon conda/conda#342

@durack1 durack1 changed the title [Bug]: 0.2.0 appears to be latest conda-forge release installed on arm64 [Bug]: 0.2.0 appears to be latest conda-forge release installed on arm64/windows Sep 12, 2022
@tomvothecoder tomvothecoder moved this to Todo in v0.3.2 Sep 13, 2022
@tomvothecoder tomvothecoder moved this from Todo to In Progress in v0.3.2 Sep 13, 2022
@tomvothecoder tomvothecoder moved this from In Progress to Review In Progress in v0.3.2 Sep 14, 2022
@tomvothecoder tomvothecoder moved this from Review In Progress to Done in v0.3.2 Sep 16, 2022
@tomvothecoder tomvothecoder moved this from Done to In Progress in v0.3.2 Sep 16, 2022
Repository owner moved this from In Progress to Done in v0.3.2 Sep 16, 2022
@durack1
Copy link
Collaborator Author

durack1 commented Oct 5, 2022

It appears that osx-arm64 support is still a while off for ESMPy anyway if esmf-org/esmf#60 is anything to go by, so aspirationally ESMF 8.4.0 might be a target for platform support

With the changes in esmf-org/esmf#79 ESMF now builds and runs nearly all tests cleanly on osx-arm64 with gfortran + clang. We plan to merge this PR within the next few days and then get it into the upcoming ESMF 8.4.0 release.

It wasn't totally obvious to me where to drop this, but it seems that osx-arm64 support is landing in the upcoming ESMF 8.4.0 - conda-forge/esmpy-feedstock#56 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Inconsistencies or issues which will cause an issue or problem for users or implementors.
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

6 participants