Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build and cache conda environments with lockfiles #2011

Merged
merged 13 commits into from
May 16, 2022

Conversation

epassaro
Copy link
Member

@epassaro epassaro commented May 4, 2022

Description

This pull request implement important changes in the test and build-docs workflows:

  • Uses the lockfiles to build the environment according to TEP013 bringing more speed and stability to the developers.
    • Every TARDIS commit is identified by a single, reproducible and always working environment. Developers can be sure they are running the same environment as the testing pipeline and other users with the same OS.
    • Faster build times.
  • Caches the conda environment with the workflow I submitted to the GitHub Actions Hackathon 2021. My workflow is already used as example in the setup-miniconda action documentation.
    • Faster build times, see the table below.

      OS Current Lockfile Lockfile + Cache
      linux-64 10m 52s 8m 59s 7m 4s
      osx-64 20m 51s 17m 46s 15m 39s

Motivation and context

How has this been tested?

  • Testing pipeline
  • Other

Examples

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • None of the above

Checklist

  • I have updated the documentation according to my changes
  • I have built the documentation by applying the build_docs label to this pull request (if you don't have enough privileges a reviewer will do it for you)
  • I have requested two reviewers for this pull request

@epassaro epassaro marked this pull request as draft May 4, 2022 21:53
@codecov
Copy link

codecov bot commented May 4, 2022

Codecov Report

Merging #2011 (d2c1f38) into master (9976a20) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2011   +/-   ##
=======================================
  Coverage   60.01%   60.01%           
=======================================
  Files          70       70           
  Lines        8115     8115           
=======================================
  Hits         4870     4870           
  Misses       3245     3245           

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@epassaro epassaro changed the title Build with lockfiles on CI Build and cache conda environments with lockfiles May 4, 2022
@tardis-bot
Copy link
Contributor

*beep* *bop*

Hi, human.

The build-docs workflow has succeeded ✔️

Click here to see your results.

@epassaro epassaro marked this pull request as ready for review May 5, 2022 14:32
Copy link
Contributor

@andrewfullard andrewfullard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How long are environments cached for? Just a specific release cycle?

@epassaro
Copy link
Member Author

epassaro commented May 6, 2022

How long are environments cached for? Just a specific release cycle?

The cache is scoped to the key and branch, the cache of the default branch is also available to other branches (reference).

The key is formed like this: conda-{platform}-{hash of the lockfile}-{manual cache number}

Then, the cache will not change unless the lockfiles are updated (a release cycle) or manually. We can also add a date to key to reset every 24hs (seems not necessary for us but can be done).

@andrewfullard andrewfullard self-requested a review May 16, 2022 15:15
@andrewfullard andrewfullard merged commit 0205add into tardis-sn:master May 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants