From 1597572eca77bc6e498d1bd4f3ddf9e026f99c7c Mon Sep 17 00:00:00 2001 From: Samuel Nichols Date: Fri, 15 Mar 2024 10:42:53 -0600 Subject: [PATCH] GitHub actions integration tests (#48) * GitHub actions clean (#40) * Create pytest.yml * Create pylint.yml * Create .pylintrc * Create test_env.yml * Full path * Remove conda install * Replace path * Pytest tests * pip -e * Create integration_tests.yml * Simplify name * CRISPRESSO2_DIR environment variable * Up one dir * ls workspace * Install CRISPResso and ydiff * Clone repo instead of checkout * submodule * ls * CRISPResso2_copy * ls * Update env * Simplify * Pull from githubactions branch * Pull githubactions repo * Checkout githubactions * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman * Run tests individually * Pin plotly version * Run all tests even if one fails * Test on another branch * Switch branch with token * Update integration_tests.yml * Introduce pandas sorting in CRISPRessoCompare (#47) * New makefile commands * Fix interleaved fastq input in CRISPRessoPooled and suppress CRISPRessoWGS params (#42) * Extract out split_interleaved_fastq function to CRISPRessoShared * Implement splitting interleaved fastq files in CRISPRessoPooled * Suppress split_interleaved_input from CRISPRessoWGS parameters * Suppress other parameters in CRISPRessoWGS * Move where interleaved fastq files are split to be trimmed properly * Bug Fix - 367 (#35) * - Fixed references to ref_names_for_pe * removed extra tabs * trying to match empty line, no tabs * - changed references to ref_names[0] * Mckay/pd warnings (#45) * refactor errors='ignore' to try except * refactored integer slice to iloc[] * moved to_numeric try except to function * Refactor to_numeric_ignore_errors to to_numeric_ignore_columns This change is slightly cleaner because it addresses the root issue that some columns are strings (and can therefore not be converted to numeric types). Now if an error does occur when converting the dfs to numeric types it won't be swallowed up. * Add documentation to to_numeric_ignore_columns --------- Co-authored-by: Cole Lyman --------- Co-authored-by: Cole Lyman * On push no branches * On push no branches * All in one file * Fix yml errors * Rename jobs * Remove old workflow files * Remove paths * Run jobs in parallel --------- Co-authored-by: mbowcut2 <55161542+mbowcut2@users.noreply.github.com> Co-authored-by: Cole Lyman --- .github/envs/test_env.yml | 2 +- .github/workflows/integration_tests.yml | 80 +++++++++++++++++++++++++ .github/workflows/pylint.yml | 2 - .github/workflows/pytest.yml | 7 +-- 4 files changed, 82 insertions(+), 9 deletions(-) create mode 100644 .github/workflows/integration_tests.yml diff --git a/.github/envs/test_env.yml b/.github/envs/test_env.yml index c5ef4954..20b3a1bf 100644 --- a/.github/envs/test_env.yml +++ b/.github/envs/test_env.yml @@ -15,4 +15,4 @@ dependencies: - scipy - matplotlib - pandas - - plotly + - plotly=5.18.0 diff --git a/.github/workflows/integration_tests.yml b/.github/workflows/integration_tests.yml new file mode 100644 index 00000000..fae5a96b --- /dev/null +++ b/.github/workflows/integration_tests.yml @@ -0,0 +1,80 @@ +name: Run Integration Tests + +on: + push: + +jobs: + build: + runs-on: ubuntu-latest + defaults: + run: + shell: bash -l {0} + env: + CRISPRESSO2_DIR: ${GITHUB_WORKSPACE}/../CRISPResso2_copy + + strategy: + fail-fast: false + + steps: + - uses: actions/checkout@v3 + + - name: Set up Conda + uses: conda-incubator/setup-miniconda@v2 + with: + mamba-version: "*" + channels: conda-forge,bioconda,defaults + auto-activate-base: false + activate-environment: test_env + environment-file: .github/envs/test_env.yml + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y gcc g++ bowtie2 samtools libsys-hostname-long-perl + pip install ydiff + + - name: Create directory for files + run: | + mkdir ../CRISPResso2_copy + cp -r * ../CRISPResso2_copy + + - name: Copy C2_tests repo + uses: actions/checkout@master + with: + repository: edilytics/CRISPResso2_tests + token: ${{ secrets.ACCESS_CRISPRESSO2_TESTS }} + # ref: '' // Use this to specify a branch other than master + + - name: Run Basic + run: | + make basic test + + - name: Run Params + if: success() || failure() + run: | + make params test + + - name: Run Prime Editor + if: success() || failure() + run: | + make prime-editor test + + - name: Run Batch + if: success() || failure() + run: | + make batch test + + - name: Run Pooled + if: success() || failure() + run: | + make pooled test + + - name: Run WGS + if: success() || failure() + run: | + make wgs test + + - name: Run Compare + if: success() || failure() + run: | + make compare test diff --git a/.github/workflows/pylint.yml b/.github/workflows/pylint.yml index f174beda..67b57b6f 100644 --- a/.github/workflows/pylint.yml +++ b/.github/workflows/pylint.yml @@ -2,8 +2,6 @@ name: Pylint on: push: - branches: - - '*' jobs: build: diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml index 81052d89..fb2a86dc 100644 --- a/.github/workflows/pytest.yml +++ b/.github/workflows/pytest.yml @@ -1,12 +1,7 @@ -name: Run Pytest on Push +name: Run Pytest on: push: - paths: - - '.github/workflows/pytest.yml' - - '.github/envs/test_env.yml' - branches: - - '*' jobs: build: