Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docker recipe to install from source #1651

Merged
merged 34 commits into from
Aug 3, 2020
Merged

Conversation

jvegreg
Copy link
Contributor

@jvegreg jvegreg commented May 7, 2020

Before you start, read CONTRIBUTING.md and the guide for diagnostic developers.

Please discuss your idea with the development team before getting started, to avoid disappointment later. The way to do this is to open a new issue on GitHub. If you are planning to modify an existing functionality, please discuss it with the original author(s) by tagging them in the issue.


Tasks

  • Create an issue to discuss what you are going to do, if you haven't done so already (and add the link at the bottom)
  • Give this pull request a descriptive title that can be used as a one line summary in a changelog
  • Make sure your code is composed of functions of no more than 50 lines and uses meaningful names for variables
  • Circle/CI tests pass. Status can be seen below your pull request. If the tests are failing, click the link to find out why.
  • Preferably Codacy code quality checks pass, however a few remaining hard to solve Codacy issues are still acceptable. Status can be seen below your pull request. If there is an error, click the link to find out why. If you suspect Codacy may be wrong, please ask by commenting.
  • Please use yamllint to check that your YAML files do not contain mistakes

If you need help with any of the tasks above, please do not hesitate to ask by commenting in the issue or pull request.


Closes #1579

@jvegreg
Copy link
Contributor Author

jvegreg commented May 7, 2020

Can not be merge until ESMValGroup/ESMValCore#603 is merged

@bouweandela
Copy link
Member

ESMValGroup/ESMValCore#603 has now been merged.

@jvegreg
Copy link
Contributor Author

jvegreg commented Jul 8, 2020

I am having the same issue @ledm describes here #1698

I have removed esmvalcore from the environment and it works well.

I also had trouble with Julia (it made the dependencies incompatible)

docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile.dev Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile.dev Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
environment.yml Outdated Show resolved Hide resolved
@bouweandela
Copy link
Member

I am having the same issue @ledm describes here #1698

I'm not able to reproduce that problem. Would you be able to provide a bash script that I can run to reproduce the issue?

docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile.dev Outdated Show resolved Hide resolved

RUN conda update -y conda pip && conda env update --name base --file environment.yml && conda clean --all -y
RUN pip install . && pip cache purge
RUN apt install julia -y && julia esmvaltool/install/Julia/setup.jl
Copy link
Member

@bouweandela bouweandela Jul 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix the julia diagnostics, I think you need to run the installed version of the setup.jl script here, not the one from the source code. Or install the tool in editable mode with pip -e.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you test again, please?

docker/Dockerfile.dev Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
@jvegreg
Copy link
Contributor Author

jvegreg commented Jul 28, 2020

The key to reduce the image size is to properly set the .dockerignore file, but I'm struggling with it

@jvegreg
Copy link
Contributor Author

jvegreg commented Jul 28, 2020

It works with -e option...

docker/Dockerfile.exp Outdated Show resolved Hide resolved
docker/Dockerfile.exp Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
@bouweandela bouweandela merged commit 346841b into master Aug 3, 2020
@bouweandela bouweandela deleted the update_docker branch August 3, 2020 09:17
bouweandela added a commit that referenced this pull request Aug 3, 2020
* Update docker files

* Add esmvaltool install command

* Upload coverage for regular unit tests

Co-authored-by: Bouwe Andela <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Container (docker/singularity) outdated
3 participants