Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Is cmor compatible with 2.12 (or what is 2.12 compatible with?)? #230

Closed
jypeter opened this issue Sep 7, 2017 · 11 comments
Closed

Is cmor compatible with 2.12 (or what is 2.12 compatible with?)? #230

jypeter opened this issue Sep 7, 2017 · 11 comments
Assignees
@doutriaux1 @dnadeau4
Milestone
3.2.7

Comments

@jypeter
Copy link

jypeter commented Sep 7, 2017

@dnadeau4 @doutriaux1 is cmor compatible with uvcdat 2.12? I have just spent quite some time checking if I could install my standard extra packages in a clone of uvcdat-2.12 and I always get conflicts. I have tried to remove packages, reorder a bit the channels, but in the end (at the bottom), I get conflicts just when I want to install cmor...

On the other hand, if I remove cmor from my big extra installation line, there is not conflict, but the installation wants to downgrade numpy from 1.13 to 1.11, which triggers lots of changes. See the end of this issue

Mixing most of these packages in 2.10 only required a harmless downgrade of zlib. See my detailed notes

(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c uvcdat -c pcmdi -c scitools -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy iris rpy2 cmocean iris gdal vacumm > /home/scratch01/jypeter/lsce-cdatm16-extra_01_install_170907.txt


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - vacumm
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c uvcdat -c pcmdi -c scitools -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy iris rpy2 cmocean iris gdal > /home/scratch01/jypeter/lsce-cdatm16-extra_01_install_170907.txt


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - iris
  - pcmdi_metrics
Use "conda info <package>" to see the dependencies for each package.

(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c uvcdat -c pcmdi pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2
Fetching package metadata ...............
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - netcdf4
  - pcmdi_metrics
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c pcmdi -c uvcdat pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter cmor shapely cartopy rpy2
Fetching package metadata ...............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - netcdf4
  - pcmdi_metrics
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c pcmdi -c uvcdat netcdf4 pcmdi_metrics cmor
Fetching package metadata ...............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - netcdf4
  - pcmdi_metrics
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c pcmdi -c uvcdat pcmdi_metrics cmor
Fetching package metadata ...............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - uvcdat
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c pcmdi -c uvcdat cmor
Fetching package metadata ...............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - uvcdat
Use "conda info <package>" to see the dependencies for each package.


(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c pcmdi cmor

Fetching package metadata .............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - dv3d
Use "conda info <package>" to see the dependencies for each package.

Without cmor

(cdatm16) bash-4.1$ conda install --dry-run -n cdatm16 -c conda-forge -c uvcdat -c pcmdi -c scitools -c vacumm pillow pandas statsmodels seaborn scikit-image seawater gsw netcdf4 pyferret basemap-data-hires pcmdi_metrics xlsxwriter shapely cartopy iris rpy2 cmocean iris gdal vacumm                                                                   
Fetching package metadata ...................
Solving package specifications: .
Warning: 4 possible package resolutions (only showing differing packages):
  - conda-forge::gdal-2.2.1-py27_0, conda-forge::statsmodels-0.8.0-py27_0
  - conda-forge::gdal-2.2.1-py27_0, conda-forge::statsmodels-0.8.0-np111py27_0
  - conda-forge::gdal-2.2.1-np111py27_0, conda-forge::statsmodels-0.8.0-py27_0
  - conda-forge::gdal-2.2.1-np111py27_0, conda-forge::statsmodels-0.8.0-np111py27_0

Package plan for installation in environment /home/share/unix_files/cdat/miniconda2/envs/cdatm16:

The following NEW packages will be INSTALLED:

    basemap-data-hires: 1.1.0-0                   conda-forge
    bcrypt:             3.1.3-py27_0                         
    biggus:             0.15.0-py27_0             conda-forge
    cairo:              1.14.6-4                  conda-forge
    cf_units:           1.1.3-py27_1              conda-forge
    cmocean:            1.1-py27_0                conda-forge
    configobj:          5.0.6-py27_0                         
    ecmwf_grib:         1.23.1-0                  conda-forge
    freexl:             1.0.2-2                   conda-forge
    gdal:               2.2.1-py27_0              conda-forge
    gettext:            0.19.7-1                  conda-forge
    giflib:             5.1.4-0                   conda-forge
    glib:               2.51.4-0                  conda-forge
    graphite2:          1.3.9-0                   conda-forge
    gsl:                2.1-2                     conda-forge
    gsw:                3.0.6-py27_0              conda-forge
    harfbuzz:           1.4.3-0                   conda-forge
    imageio:            2.2.0-py27_0              conda-forge
    iris:               1.13.0-py27_1             conda-forge
    iris-grib:          0.10.1-py27_0             conda-forge
    json-c:             0.12.1-0                  conda-forge
    kealib:             1.4.7-2                   conda-forge
    libdap4:            3.18.3-2                  conda-forge
    libgdal:            2.2.1-0                   conda-forge
    libmo_unpack:       3.0-1                     conda-forge
    libpq:              9.6.3-0                   conda-forge
    libspatialite:      4.3.0a-15                 conda-forge
    mo_pack:            0.2.0-py27_1              conda-forge
    nc_time_axis:       1.0.2-py27_0              conda-forge
    netcdf4:            1.2.9-py27_1              conda-forge
    networkx:           1.11-py27_0               conda-forge
    openjpeg:           2.2.0-0                   conda-forge
    pango:              1.40.4-0                  conda-forge
    paramiko:           2.2.1-py27_0              conda-forge
    patsy:              0.4.1-py27_0              conda-forge
    pcmdi_metrics:      v1.1.1-PMP                pcmdi      
    pcre:               8.39-0                    conda-forge
    pixman:             0.34.0-0                  conda-forge
    pyasn1:             0.3.3-py27_0              conda-forge
    pyferret:           7.1.0-py27_0              conda-forge
    pyke:               1.1.1-py27_0              conda-forge
    pynacl:             1.1.2-py27_0              conda-forge
    python-ecmwf_grib:  1.23.1-py27_0             conda-forge
    pywavelets:         0.5.2-np111py27_0         conda-forge
    r-base:             3.3.2-5                   conda-forge
    rpy2:               2.8.5-py27r3.3.2_0        conda-forge
    scikit-image:       0.13.0-py27_1             conda-forge
    seaborn:            0.8.1-py27_0              conda-forge
    seawater:           3.3.4-py27_0              conda-forge
    statsmodels:        0.8.0-py27_0              conda-forge
    unidata:            2.8-np111py27_2           uvcdat     
    util-linux:         2.21-0                               
    vacumm:             3.4.1-1                   vacumm     
    xerces-c:           3.1.4-3                   conda-forge
    xlrd:               1.0.0-py27_1              conda-forge
    xlsxwriter:         0.9.8-py27_0              conda-forge
    xlutils:            2.0.0-py27_0                         
    xlwt:               1.2.0-py27_0              conda-forge

The following packages will be UPDATED:

    binaryio:           2.12-np113py27h8073811_0  uvcdat      --> 2.12-np111py27h044819a_0 uvcdat     
    cdms2:              2.12-np113py27_0          conda-forge --> 2.12-np111py27_0         conda-forge
    cdtime:             2.12-np113py27_2          conda-forge --> 2.12-np111py27_2         conda-forge
    cssgrid:            2.12-np113py27h3898552_0  uvcdat      --> 2.12-np111py27h314ff11_0 uvcdat     
    dsgrid:             2.12-np113py27h9d524be_0  uvcdat      --> 2.12-np111py27hc815344_0 uvcdat     
    genutil:            2.12-np113py27_0          conda-forge --> 2.12-np111py27_0         conda-forge
    lmoments:           2.12-np113py27h5574120_0  uvcdat      --> 2.12-np111py27h58b1cd0_0 uvcdat     
    matplotlib:         1.5.3-np113py27_8         conda-forge --> 2.0.0-np111py27_2        conda-forge
    natgrid:            2.12-np113py27h481ab8b_0  uvcdat      --> 2.12-np111py27h4f51b5e_0 uvcdat     
    ort:                2.12-np113py27hd89025a_0  uvcdat      --> 2.12-np111py27hfd1f69f_0 uvcdat     
    pillow:             4.2.1-py27_0              conda-forge --> 4.2.1-py27_1             conda-forge
    regridpack:         2.12-np113py27hf1cacc0_0  uvcdat      --> 2.12-np111py27hf311d9d_0 uvcdat     
    shgrid:             2.12-np113py27h2504d4d_0  uvcdat      --> 2.12-np111py27hf037da4_0 uvcdat     
    trends:             2.12-np113py27hedb240a_0  uvcdat      --> 2.12-np111py27hdb0eb65_0 uvcdat     
    uvcmetrics:         2.2.1-np113py27h151233a_0 uvcdat      --> 2.2.1-np111py27_0        uvcdat     
    zonalmeans:         2.12-np113py27h96f9f80_0  uvcdat      --> 2.12-np111py27h28606d6_0 uvcdat     

The following packages will be DOWNGRADED due to dependency conflicts:

    numpy:              1.13.1-py27_0                         --> 1.11.3-py27_0                       
    scikit-learn:       0.19.0-np113py27_0                    --> 0.18.1-np111py27_1                  
    scipy:              0.19.1-np113py27_0                    --> 0.19.0-np111py27_0                  
    zlib:               1.2.11-0                  conda-forge --> 1.2.8-3                  conda-forge
@doutriaux1 doutriaux1 added this to the 3.2.7 milestone Sep 7, 2017
@jypeter
Copy link
Author

jypeter commented Sep 12, 2017

For reference, some answers to this issue are provided in CDAT/genutil#14 (comment)

@jypeter
Copy link
Author

jypeter commented Sep 12, 2017

So, not sure I got @doutriaux1 instructions right, but my cdatm16 is a clone of the official uvcdat-2.12 and I have tried to downgrade numpy to a version where I can install cmor (and other stuff), but it's not quite working as expected

All I'm trying to achieve, if possible, is uvcdat 2.12 and cmor 3.2.6 and some other packages

Downgrade numpy

I'm getting a numpy+blas instead of just a downgraded numpy, but why not (blas can be useful)

bash-4.1$ conda install -n cdatm16 -c conda-forge -c pcmdi -c uvcdat numpy=1.12
Fetching package metadata ...............
Solving package specifications: .

Package plan for installation in environment /home/share/unix_files/cdat/miniconda2/envs/cdatm16:

The following NEW packages will be INSTALLED:

    blas:         1.1-openblas              conda-forge
    openblas:     0.2.19-2                  conda-forge

The following packages will be UPDATED:

    binaryio:     2.12-np113py27h8073811_0  uvcdat      --> 2.12-np112py27h5813a3b_0      uvcdat     
    cdms2:        2.12-np113py27_0          conda-forge --> 2.12-np112py27_0              conda-forge
    cdtime:       2.12-np113py27_2          conda-forge --> 2.12-np112py27_2              conda-forge
    cssgrid:      2.12-np113py27h3898552_0  uvcdat      --> 2.12-np112py27hf1c8260_0      uvcdat     
    dsgrid:       2.12-np113py27h9d524be_0  uvcdat      --> 2.12-np112py27h822527b_0      uvcdat     
    genutil:      2.12-np113py27_0          conda-forge --> 2.12-np112py27_0              conda-forge
    lmoments:     2.12-np113py27h5574120_0  uvcdat      --> 2.12-np112py27h665d648_0      uvcdat     
    matplotlib:   1.5.3-np113py27_8         conda-forge --> 2.0.0-np112py27_2             conda-forge
    natgrid:      2.12-np113py27h481ab8b_0  uvcdat      --> 2.12-np112py27h569c149_0      uvcdat     
    ort:          2.12-np113py27hd89025a_0  uvcdat      --> 2.12-np112py27h964b06a_0      uvcdat     
    regridpack:   2.12-np113py27hf1cacc0_0  uvcdat      --> 2.12-np112py27h6a169cd_0      uvcdat     
    scikit-learn: 0.19.0-np113py27_0                    --> 0.19.0-py27_blas_openblas_201 conda-forge [blas_openblas]
    scipy:        0.19.1-np113py27_0                    --> 0.19.1-py27_blas_openblas_202 conda-forge [blas_openblas]
    shgrid:       2.12-np113py27h2504d4d_0  uvcdat      --> 2.12-np112py27h6f2dc27_0      uvcdat     
    trends:       2.12-np113py27hedb240a_0  uvcdat      --> 2.12-np112py27he3ac0c2_0      uvcdat     
    uvcmetrics:   2.2.1-np113py27h151233a_0 uvcdat      --> 2.2.1-np112py27_0             uvcdat     
    zonalmeans:   2.12-np113py27h96f9f80_0  uvcdat      --> 2.12-np112py27hbce4e2c_0      uvcdat     

The following packages will be SUPERCEDED by a higher-priority channel:

    numpy:        1.13.1-py27_0                         --> 1.12.1-py27_blas_openblas_200 conda-forge [blas_openblas]

Proceed ([y]/n)? y

Try to install cmor

Does not work this way

bash-4.1$ conda install -n cdatm16 -c conda-forge -c pcmdi -c uvcdat cmor
Fetching package metadata ...............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - uvcdat
Use "conda info <package>" to see the dependencies for each package.

I get another dependency error message if I remove the uvcdat channel

bash-4.1$ conda install -n cdatm16 -c conda-forge -c pcmdi cmor
Fetching package metadata .............
Solving package specifications: .


UnsatisfiableError: The following specifications were found to be in conflict:
  - cmor
  - uvcmetrics
Use "conda info <package>" to see the dependencies for each package.

What next?

Any ideas? Did I do something wrong? I have listed below the conflicting packages versions

I don't have too much time to work on this, with our PMIP conference in less than 2 weeks and nothing ready so I will maybe stick to 2.10, rather than trying the bleeding edge 2.12

Just thought I'd give this a try and share my findings

bash-4.1$ conda list -n cdatm16 numpy                  
# packages in environment at /home/share/unix_files/cdat/miniconda2/envs/cdatm16:
#
numpy                     1.12.1          py27_blas_openblas_200  [blas_openblas]  conda-forge
numpydoc                  0.7.0                    py27_0    conda-forge
bash-4.1$ conda list -n cdatm16 uvcdat 
# packages in environment at /home/share/unix_files/cdat/miniconda2/envs/cdatm16:
#
uvcdat                    2.12                 h5103ec1_0    uvcdat
bash-4.1$ conda list -n cdatm16 uvcmetrics
# packages in environment at /home/share/unix_files/cdat/miniconda2/envs/cdatm16:
#
uvcmetrics                2.2.1               np112py27_0    uvcdat

@doutriaux1
Copy link
Collaborator

@jypeter I need to push an hdf5 cmor onto the uvcdat channel

@doutriaux1
Copy link
Collaborator

@dnadeau4 where is the recipe?

@doutriaux1
Copy link
Collaborator

@jypeter pushed a cmor 3.2.6 on uvcdat channel, should be able to bring it into your linux 2.12 now

conda install -c conda-forge -c uvcdat cmor

@durack1
Copy link
Contributor

durack1 commented Sep 12, 2017

@doutriaux1 these once off hack jobs make it really confusing for a user.. It would have been better to wait for @dnadeau4 to push out the 3.2.7 release.. Now when obs4MIPs, CMIP6 users are trying to get cmor the ordering of their channels is going to matter ALOT more

@jypeter
Copy link
Author

jypeter commented Sep 12, 2017

It worked indeed! But it is utterly confusing for the user (me!) knowing which channels to use, when, and in which order! And in that case I should not even use pcmdi

(cdatm16) bash-4.1$ conda install -n cdatm16 -c conda-forge -c uvcdat cmor
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment /home/share/unix_files/cdat/miniconda2/envs/cdatm16:

The following NEW packages will be INSTALLED:

    cmor: 3.2.6-np112py27h8434654_0 uvcdat

Proceed ([y]/n)? y

@doutriaux1
Copy link
Collaborator

nope it's 3.2.6 official so order does not matter, it's the same version. Once @dnadeau4 releases 3.2.7 I will either take these off or label them.

@durack1
Copy link
Contributor

durack1 commented Sep 12, 2017

@dnadeau4 did you want to push the 3.2.7 release just with the hdf fix today, and then relabel the outstanding issues for 3.2.8? It'd be great to clean this up sooner rather than later as it's just going to confuse people

@dnadeau4
Copy link
Collaborator

I don't wan to push 3.2.7, there is nothing new and the users can "source activate" their cmor environment. In 3.2.7 I'll make sure I have the same dependencies as UV-CDAT 2.12. UV-CDAT updated HDF5 in their release last week. @jypeter can still use UV-CDAT 2.10 with CMOR 3.2.6.

@dnadeau4
Copy link
Collaborator

Fix 3.2.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@doutriaux1 doutriaux1

@dnadeau4 dnadeau4

Labels
None yet
Projects
None yet
Milestone
3.2.7
Development

No branches or pull requests
4 participants
@doutriaux1 @jypeter @durack1 @dnadeau4