Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use --satisfied-skip-solve on conda install #5631

Closed
wants to merge 1 commit into from

Conversation

humitos
Copy link
Member

@humitos humitos commented Apr 25, 2019

Skip (do not upgrade) already satisfied dependencies that were
installed by the user when creating the environment, when installing
our own dependencies.

I'm building the images locally to do some QA before merging it.

Closes #3829
Related readthedocs/readthedocs-docker-images#102

@humitos
Copy link
Member Author

humitos commented Apr 25, 2019

This feature was introduced on conda 4.6.0 and we are running < 4.6 in production.

https://github.com/conda/conda/blob/master/CHANGELOG.md#improvements-6

So, we can't merge and deploy this PR until we upgrade all the images first. On the other hand, this can be added under a feature flag, but I think it's better to make a new deploy of the Docker images at this point.

@humitos humitos added the Status: blocked Issue is blocked on another issue label Apr 25, 2019
Skip (do not upgrade) already satisfied dependencies that were
installed by the user when creating the environment, when installing
our own dependencies.
@humitos humitos force-pushed the humitos/satisfied-dependencies-conda branch from 78e7146 to c63efba Compare April 25, 2019 11:47
@humitos
Copy link
Member Author

humitos commented Apr 25, 2019

OK. I found that this is still a problem for our use case :/

I did some tests with this repo (https://github.com/rtfd/test-builds/tree/conda-satisfied-skip-solve)

$ conda env create -f environment.yml
...

$ conda list | grep sphinx                                                                  
sphinx                    1.8.1                 py37_1000    conda-forge
sphinx-autorun            1.1.0                    pypi_0    pypi
sphinx_rtd_theme          0.4.3                      py_0    conda-forge
sphinxcontrib-websupport  1.1.0                      py_1    conda-forge
(condabuild) (miniconda3-latest) 

$ conda install --quiet --satisfied-skip-solve sphinx                                 

# All requested packages already installed.

Up to here, it works as I expected. Although, when there is at least one package that needs to be upgraded, it upgrades all the packages and we don't want that:

$ conda install --quiet --satisfied-skip-solve sphinx mock
Collecting package metadata: ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/humitos/.pyenv/versions/miniconda3-latest/envs/condabuild

  added / updated specs:
    - mock
    - sphinx


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    certifi-2019.3.9           |           py37_0         155 KB
    mock-2.0.0                 |           py37_0         104 KB
    pbr-5.1.3                  |             py_0          74 KB


    sphinx-2.0.1               |             py_0         1.4 MB


    sphinxcontrib-applehelp-1.0.1|             py_0          29 KB
    sphinxcontrib-devhelp-1.0.1|             py_0          23 KB
    sphinxcontrib-htmlhelp-1.0.2|             py_0          28 KB
    sphinxcontrib-jsmath-1.0.1 |             py_0           8 KB
    sphinxcontrib-qthelp-1.0.2 |             py_0          26 KB
    sphinxcontrib-serializinghtml-1.1.3|             py_0          24 KB
    ------------------------------------------------------------
                                           Total:         1.9 MB

The following NEW packages will be INSTALLED:

  mock               pkgs/main/linux-64::mock-2.0.0-py37_0
  pbr                pkgs/main/noarch::pbr-5.1.3-py_0
  sphinxcontrib-app~ pkgs/main/noarch::sphinxcontrib-applehelp-1.0.1-py_0
  sphinxcontrib-dev~ pkgs/main/noarch::sphinxcontrib-devhelp-1.0.1-py_0
  sphinxcontrib-htm~ pkgs/main/noarch::sphinxcontrib-htmlhelp-1.0.2-py_0
  sphinxcontrib-jsm~ pkgs/main/noarch::sphinxcontrib-jsmath-1.0.1-py_0
  sphinxcontrib-qth~ pkgs/main/noarch::sphinxcontrib-qthelp-1.0.2-py_0
  sphinxcontrib-ser~ pkgs/main/noarch::sphinxcontrib-serializinghtml-1.1.3-py_0

The following packages will be UPDATED:

  sphinx             conda-forge/linux-64::sphinx-1.8.1-py~ --> pkgs/main/noarch::sphinx-2.0.1-py_0

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates    conda-forge::ca-certificates-2019.3.9~ --> pkgs/main::ca-certificates-2019.1.23-0
  certifi                                       conda-forge --> pkgs/main
  openssl            conda-forge::openssl-1.1.1b-h14c3975_1 --> pkgs/main::openssl-1.1.1b-h7b6447c_1


Proceed ([y]/n)? 

It will update sphinx to 2.0.1.

@humitos
Copy link
Member Author

humitos commented Jun 12, 2019

I just tried this with miniconda 4.6.14 and this still happens.

humitos added a commit that referenced this pull request Jul 18, 2019
We run 3 steps when a project depends on conda.

1. create the whole environment based on user's YAML file
2. run `conda install` with our own dependencies
3. run `pip install` with some of our dependencies that are not
published on conda repositories.

This commit changes this to make it in just one step (at environment
creation). To do this, it appends our own requirements to the
`environment.yml` file under `dependencies` and `dependencies.pip`
config (see https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#create-env-file-manually)

It also shows the resulting `environment.yml` in the build output.

This behavior is added behind a feature flag so we can test it first.

This allow users to be able to pin their dependencies as they want
without us upgrading them in the second step. Also, this should
improve the build time, since dependencies resolution is done just
once.

Related to

* #3829
* #5631
@humitos
Copy link
Member Author

humitos commented Aug 7, 2019

I'm closing this PR for now since we have a feature flag that helps us here. We can revisit in the future when conda behaves as we need.

@humitos humitos closed this Aug 7, 2019
@stsewd stsewd deleted the humitos/satisfied-dependencies-conda branch January 21, 2021 21:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: blocked Issue is blocked on another issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ability to pin Sphinx and friends in a conda environment
1 participant