GitHub Actions #1806

valeriupredoi · 2020-09-04T11:03:08Z

Summary

Implementation of cross-platform, cross-Python version testing:

Linux(Ubuntu) and OSX, both latest
Python 3.6-7-8
Installation from Conda and source
Tests
Conda package build for latest python and Linux only (as backup for the Circle test that fails coz of exceeding memory limit)

Notes

all tests pass for Linux and all Python versions
we have issues with OSX and R-packages and Python 3.8, see Installation and tests on OSX #1807
working tests for OSX:
- source install bar Python 3.8
- Conda installation for esmvaltool-python and esmvaltool-ncl
- tests in not installation mode
I have added an installation decorator for the r-lint test so it's skipped when there is no working r-linter

Issues to be looked at after merge

The problems with OSX and Python 3.8 and also the R-packages

valeriupredoi · 2020-09-04T11:09:53Z

@bouweandela @nielsdrost would you reckon the conda_build test should be done only on Linux and only with latest Python?

valeriupredoi · 2020-09-04T11:38:47Z

test_r_lint fails both in the GA testing environment and on my machine -- I thought we'd fixed that with either skip or run? @bouweandela

EDIT am being a dummy and I forgot to install the R packages - done now but R is being a turkey for OSX

valeriupredoi · 2020-09-07T15:57:22Z

OSX related issues in separate issue: #1807

valeriupredoi · 2020-09-08T12:41:43Z

OK this one's ready for merge 👍

nielsdrost · 2020-09-14T15:00:18Z

.circleci/config.yml

@@ -169,9 +164,10 @@ jobs:
    working_directory: /esmvaltool
    docker:
      - image: continuumio/miniconda3
+    resource_class: medium+


This breaks the circle-ci build for me :-(

https://app.circleci.com/pipelines/github/ESMValGroup/ESMValTool/3128/workflows/237313fa-aac4-41da-bbe5-c40f961c40de/jobs/36447

This job has been blocked because resource-class:medium+ is not available on your plan. Please upgrade to continue building.

I removed that a while ago man, why are you still picking it up?

meh proves out I didnt - sorry about that @nielsdrost - that test failes in Circle coz 137 (memory), that's why we now have a GA test that does that

bouweandela · 2020-09-14T15:54:06Z

@valeriupredoi Nice work! It looks like you branched off from a branch in a now merged pull request. Could you merge master into this branch to minimize the changes shown in the diff tab?

bouweandela · 2020-09-14T15:58:47Z

@bouweandela @nielsdrost would you reckon the conda_build test should be done only on Linux and only with latest Python?

Yes, that would be great.

valeriupredoi · 2020-09-14T16:01:24Z

@bouweandela @nielsdrost would you reckon the conda_build test should be done only on Linux and only with latest Python?

Yes, that would be great.

did it last week, man, and it passes nicely all the time

nielsdrost

Looks good to go to me, thanks @valeriupredoi

nielsdrost · 2020-09-15T15:10:53Z

.github/workflows/action-conda-build.yml

+
+# runs on a push on master and at the end of every day
+on:
+  # triggering on push without branch name will run tests everytime


this comment could perhaps be removed.

nielsdrost · 2020-09-15T15:11:11Z

.github/workflows/action-conda-build.yml

+  push:
+    branches:
+    - master
+    - github-actions2


perhaps add a note that this is a test branch

nielsdrost · 2020-09-15T15:13:17Z

.github/workflows/action-conda-build.yml

+          miniconda-version: "latest"
+          channels: conda-forge
+      - shell: bash -l {0}
+        run: mkdir -p condabuild_artifacts_python_${{ matrix.python-version }}


shame there is no way (that I could find) to auto-save all logs as artifacts.

there is none yet (apparently, after I too have dug quite a bit); but the output is identical to the one you see on the test screen so it's prob not that useful now

valeriupredoi · 2020-09-17T10:54:47Z

thanks very much for the review @nielsdrost - I'll go ahead and push these tests now but we'll have to revisit the OSX bits since a couple improvements from the core env ESMValGroup/ESMValCore#786 and ESMValGroup/ESMValCore#780 will have improved some aspects; there still is the R issues with OSX that I'll look into too 🍺

bouweandela and others added 7 commits September 2, 2020 15:17

Use pytest-xdist, avoid setup.py test and update CircleCI config

0572f90

Try to use codacy coverage orb

739f439

Remove test alias

2ecabd6

Require minimal version of pytest-cov

400def4

Require minimum version of pytest-flake8

da25e63

added extra requirement for conda build memory

df6f1ac

added actions workflows

763c4e1

valeriupredoi requested review from nielsdrost, bouweandela and jvegreg September 4, 2020 11:03

valeriupredoi mentioned this pull request Sep 4, 2020

Speed up tests on CircleCI and use pytest to run them #1804

Merged

5 tasks

added conda build workflow

1721590

valeriupredoi added 9 commits September 4, 2020 13:15

comment out julis symlink for now

d6b0c19

use 2020 pip solver

bd0584e

gaahhh

02cd81f

force ncurses to be from conda forge

b6b5e29

wrong OS for ncurses fixed

b1c554b

removed py 38 from OSX conda install

21ca0ea

try to install R and Julia first

90b7190

force ncurses from conda forge

0ad1cbd

removed channel activation and forcing ncurses

4ca542c

valeriupredoi mentioned this pull request Sep 7, 2020

Installation and tests on OSX #1807

Closed

valeriupredoi added 5 commits September 7, 2020 17:15

force and keep ancurses from conda forge

c9d7ea0

force ncurses via uninstall

fed5f4c

gave up installing full package and revert to only python and ncl

3a56553

remove tweaking ncurses

08c18e3

made the r-lint as install test to be skipped

906de1a

valeriupredoi added 3 commits September 8, 2020 13:03

removed py 3.8 for osx testing

177168b

removed py 3.8 for osx testing

709607b

removed py 3.8 for osx testing

60c5612

Merge branch 'master' of https://github.com/ESMValGroup/ESMValTool

5ae149a

nielsdrost reviewed Sep 14, 2020

View reviewed changes

Merge branch 'master' of https://github.com/ESMValGroup/ESMValTool

d235467

Merge branch 'master' into github-actions2

4750dd4

removed requirement - weird - I thought I removed this ages ago

5c2d55e

nielsdrost approved these changes Sep 15, 2020

View reviewed changes

removed some comments

09dc52f

valeriupredoi merged commit 1125c36 into master Sep 17, 2020

valeriupredoi deleted the github-actions2 branch September 17, 2020 11:19

valeriupredoi restored the github-actions2 branch September 17, 2020 11:19

bouweandela mentioned this pull request Sep 21, 2020

Improve CircleCI setup #1113

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Actions #1806

GitHub Actions #1806

valeriupredoi commented Sep 4, 2020 •

edited

Loading

valeriupredoi commented Sep 4, 2020

valeriupredoi commented Sep 4, 2020 •

edited

Loading

valeriupredoi commented Sep 7, 2020

valeriupredoi commented Sep 8, 2020

nielsdrost Sep 14, 2020

nielsdrost Sep 14, 2020

valeriupredoi Sep 14, 2020

valeriupredoi Sep 14, 2020

bouweandela commented Sep 14, 2020

bouweandela commented Sep 14, 2020

valeriupredoi commented Sep 14, 2020

nielsdrost left a comment

nielsdrost Sep 15, 2020

nielsdrost Sep 15, 2020

valeriupredoi Sep 17, 2020

nielsdrost Sep 15, 2020

valeriupredoi Sep 17, 2020

valeriupredoi commented Sep 17, 2020

GitHub Actions #1806

GitHub Actions #1806

Conversation

valeriupredoi commented Sep 4, 2020 • edited Loading

valeriupredoi commented Sep 4, 2020

valeriupredoi commented Sep 4, 2020 • edited Loading

valeriupredoi commented Sep 7, 2020

valeriupredoi commented Sep 8, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bouweandela commented Sep 14, 2020

bouweandela commented Sep 14, 2020

valeriupredoi commented Sep 14, 2020

nielsdrost left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valeriupredoi commented Sep 17, 2020

valeriupredoi commented Sep 4, 2020 •

edited

Loading

valeriupredoi commented Sep 4, 2020 •

edited

Loading