Skip to content
This repository has been archived by the owner on May 20, 2020. It is now read-only.

Improve conda #244

Closed
wants to merge 36 commits into from
Closed
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
c8f6c31
fix install command and publish only once
fdiblen May 19, 2020
a579b6c
Merge branch 'alignment' into improve-conda
fdiblen May 19, 2020
ad56171
merge anaconda and python build workflows
fdiblen May 19, 2020
57856f9
remove python build badge
fdiblen May 19, 2020
aa4b27c
bump to 0.3.2
fdiblen May 19, 2020
f3aff59
update the changelog
fdiblen May 19, 2020
eda8b12
handle data in setup.py
fdiblen May 20, 2020
a6a39a3
change conda info step while testing conda package build
fdiblen May 20, 2020
7fe99d2
automatically activate conda environments
fdiblen May 20, 2020
13d2293
test generated conda package by installing it
fdiblen May 20, 2020
8859b04
remove conda/conda_build_config.yaml
fdiblen May 20, 2020
a4635fa
add BUILD_FOLDER variable
fdiblen May 20, 2020
a9d02d5
Update setup.py
fdiblen May 20, 2020
59001cf
list conda environments for debugging
fdiblen May 20, 2020
eaed848
Merge branch 'improve-conda' of github.com:matchms/matchms into impro…
fdiblen May 20, 2020
57f7704
remove the build step
fdiblen May 20, 2020
eb30213
deactivate previous environment before creating a new environment
fdiblen May 20, 2020
b712c28
Merge branch 'master' into improve-conda
fdiblen May 20, 2020
d2fc329
data is added by setup.py, build scripts ware removed
fdiblen May 20, 2020
0e97ac7
Merge branch 'improve-conda' of github.com:matchms/matchms into impro…
fdiblen May 20, 2020
9abd42a
remove condarc
fdiblen May 20, 2020
27467bd
use defaults for conda ction
fdiblen May 20, 2020
aa2f9a0
add Python to action names
fdiblen May 20, 2020
bb75b60
split workflows and test the conda package on windows/unix separately
fdiblen May 20, 2020
d1cc37d
update temp folder for windows
fdiblen May 20, 2020
7f9112c
update CHANGELOG.md
fdiblen May 20, 2020
49e2b5b
add strategy section to build workflow
fdiblen May 20, 2020
81ef3cc
add strategy section to publish workflow
fdiblen May 20, 2020
2062f7a
make distinction for workflow jobs using different names
fdiblen May 20, 2020
2399b7f
updated conda build files
jspaaks May 20, 2020
c023bd2
change strategy - needs order in build
fdiblen May 20, 2020
4faa138
Merge branch 'improve-conda' of github.com:matchms/matchms into impro…
fdiblen May 20, 2020
61fdfab
test build only on ubuntu
fdiblen May 20, 2020
3cde0e7
remove deactivate step
fdiblen May 20, 2020
41f2852
test the build workflow as a single job
fdiblen May 20, 2020
b18026d
remove extra checkouts
fdiblen May 20, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 0 additions & 82 deletions .github/workflows/build.yml

This file was deleted.

91 changes: 83 additions & 8 deletions .github/workflows/conda_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,92 @@ jobs:
python-version: ['3.7']
steps:
- uses: actions/checkout@v2
with:
fetch-depth: '0'
- uses: goanpeca/setup-miniconda@v1
with:
auto-update-conda: false
python-version: ${{ matrix.python-version }}
activate-environment: matchms-dev
environment-file: conda/environment-dev.yml
condarc-file: conda/condarc.yml
jspaaks marked this conversation as resolved.
Show resolved Hide resolved
auto-activate-base: true
jspaaks marked this conversation as resolved.
Show resolved Hide resolved
- name: Conda info
shell: bash -l {0}
run: conda info
- name: Python info
shell: bash -l {0}
run: |
which python
python --version
- name: Conda list
shell: bash -l {0}
run: conda list
- name: Check style against standards using prospector
shell: bash -l {0}
run: prospector -o grouped -o pylint:pylint-report.txt
- name: Check whether import statements are used consistently
shell: bash -l {0}
run: isort --check-only --diff --conda-env matchms-dev --recursive --wrap-length 79 --lines-after-imports 2 --force-single-line --no-lines-before FUTURE --no-lines-before STDLIB --no-lines-before THIRDPARTY --no-lines-before FIRSTPARTY --no-lines-before LOCALFOLDER matchms/ tests/ integration-tests/
- name: Build
shell: bash -l {0}
run: |
python setup.py build
- name: Build documentation
jspaaks marked this conversation as resolved.
Show resolved Hide resolved
shell: bash -l {0}
run: |
make coverage html
working-directory: readthedocs/
env:
SPHINXOPTS: '-n' # enable nit-picky mode
- name: Check documentation coverage threshold
if: matrix.os == 'ubuntu-latest'
shell: bash -l {0}
run: |
cat readthedocs/_build/coverage/python.txt
UNCOVERED_MEMBERS=$(grep '*' readthedocs/_build/coverage/python.txt | wc -l)
UNCOVERED_MEMBERS_ALLOWED=25
if (( $UNCOVERED_MEMBERS > $UNCOVERED_MEMBERS_ALLOWED )) ; then echo "There are currently ${UNCOVERED_MEMBERS} uncovered members in the documentation, which is more than the ${UNCOVERED_MEMBERS_ALLOWED} allowed."; exit 1;fi
echo "The code is sufficiently documented with ${UNCOVERED_MEMBERS} uncovered members out of ${UNCOVERED_MEMBERS_ALLOWED} allowed.";
- name: Test
if: matrix.os != 'ubuntu-latest'
shell: bash -l {0}
run: |
python setup.py test
- name: Test with coverage
if: matrix.os == 'ubuntu-latest'
shell: bash -l {0}
run: |
pytest --cov --cov-report term --cov-report xml --junitxml=xunit-result.xml
- name: Correct coverage paths
if: matrix.os == 'ubuntu-latest'
run: sed -i "s+$PWD/++g" coverage.xml
- name: SonarCloud Scan
if: matrix.os == 'ubuntu-latest'
uses: sonarsource/sonarcloud-github-action@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
- name: Create matchms-build environment
uses: goanpeca/setup-miniconda@v1
with:
activate-environment: matchms-build
auto-activate-base: false
auto-update-conda: true
auto-activate-base: true
auto-update-conda: false
environment-file: conda/environment-build.yml
miniconda-version: 'latest'
python-version: ${{ matrix.python-version }}
- name: Show conda config
- name: Conda info
shell: bash -l {0}
run: |
conda info
conda list
conda config --show-sources
conda config --show
run: conda info
- name: Python info
shell: bash -l {0}
run: |
which python
python --version
- name: Conda list
shell: bash -l {0}
run: conda list
- name: Build the conda package
shell: bash -l {0}
run: |
Expand All @@ -44,3 +110,12 @@ jobs:
--channel bioconda --channel conda-forge \
--croot $BUILD_FOLDER \
./conda
- name: Test local conda package
shell: bash -l {0}
run: |
export BUILD_FOLDER=/tmp/matchms/_build
conda install \
--channel bioconda \
--channel conda-forge \
--channel file://${BUILD_FOLDER}/noarch/ \
matchms
4 changes: 2 additions & 2 deletions .github/workflows/conda_publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest', 'windows-latest']
os: ['ubuntu-latest']
python-version: ['3.7']
steps:
- uses: actions/checkout@v2
Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:
- name: Install the conda package from anaconda repository
shell: bash -l {0}
run: |
conda install conda build -c bioconda -c conda-forge -c nlesc matchms
conda install -c bioconda -c conda-forge -c nlesc matchms
- name: Install development dependencies
shell: bash -l {0}
run: python -m pip install -r conda/requirements-dev.txt
Expand Down
19 changes: 16 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.3.2] - 2020-05-20

### Changed

- improve conda workflow [#244](https://github.com/matchms/matchms/pull/244)
- merge anaconda and python build workflows
- fix conda package install command in build workflow
- publish only on ubuntu machine

### Removed

- remove python build badge [#244](https://github.com/matchms/matchms/pull/244)

## [0.3.1] - 2020-05-19

### Added

- improve conda package https://github.com/matchms/matchms/pull/225
- improve conda package [#225](https://github.com/matchms/matchms/pull/225)
- Build scripts for Windows and Unix(MacOS and Linux) systems
- verify conda package after uploading to anaconda repository by installing it
- conda package also includes `matchms/data` folder
Expand All @@ -29,9 +42,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- skip builds for Python 2.X
- more flexible package versions
- add deployment requirements to meta.yml
- verify conda package https://github.com/matchms/matchms/pull/225
- verify conda package [#225](https://github.com/matchms/matchms/pull/225)
- use conda/environment.yml when building the package
- split anaconda workflow https://github.com/matchms/matchms/pull/225
- split anaconda workflow [#225](https://github.com/matchms/matchms/pull/225)
- conda build: tests conda packages on every push and pull request
- conda publish: publish and test conda package on release
- update the developer instructions
Expand Down
8 changes: 2 additions & 6 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Vector representation and similarity measure for mass spectrometry data.
* - **Other best practices**
-
* - Continuous integration
- |Python Build| |Anaconda Build| |Anaconda Publish|
- |Anaconda Build| |Anaconda Publish|
* - Documentation
- |ReadTheDocs Badge|
* - Code Quality
Expand Down Expand Up @@ -75,10 +75,6 @@ Vector representation and similarity measure for mass spectrometry data.
:target: https://sonarcloud.io/component_measures?id=matchms_matchms&metric=Coverage&view=list
:alt: Sonarcloud Coverage

.. |Python Build| image:: https://github.com/matchms/matchms/workflows/Python%20Build/badge.svg
:target: https://github.com/matchms/matchms/actions?query=workflow%3A%22Python%20Build%22
:alt: Python Build

.. |Anaconda Build| image:: https://github.com/matchms/matchms/workflows/Anaconda%20Build/badge.svg
:target: https://github.com/matchms/matchms/actions?query=workflow%3A%22Anaconda%20Build%22
:alt: Anaconda Build
Expand Down Expand Up @@ -191,7 +187,7 @@ installation of this conda package works with:
conda install \
--channel bioconda \
--channel conda-forge \
--channel file://${CONDA_PREFIX}/output/noarch/ \
--channel file://${CONDA_PREFIX}/noarch/ \
matchms

To publish the package on anaconda cloud, do:
Expand Down
6 changes: 3 additions & 3 deletions conda/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ env

$PYTHON -m pip install --no-deps --ignore-installed . -vv
jspaaks marked this conversation as resolved.
Show resolved Hide resolved

# Copy data folder
mkdir -p $PREFIX/site-packages/matchms/data
cp matchms/data/* $PREFIX/site-packages/matchms/data/
# # Copy data folder
# mkdir -p $PREFIX/site-packages/matchms/data
# cp matchms/data/* $PREFIX/site-packages/matchms/data/
Empty file removed conda/conda_build_config.yaml
Empty file.
2 changes: 1 addition & 1 deletion conda/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{% set name = "matchms" %}
{% set version = "0.3.1" %}
{% set version = "0.3.2" %}

package:
name: {{ name|lower }}
Expand Down
2 changes: 1 addition & 1 deletion matchms/__version__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '0.3.1'
__version__ = '0.3.2'
6 changes: 2 additions & 4 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 0.3.1
current_version = 0.3.2

[bumpversion:file:conda/meta.yaml]
search = set version = "{current_version}"
Expand All @@ -13,8 +13,7 @@ replace = __version__ = '{new_version}'
description-file = README.rst

[aliases]
# Define `python setup.py test`
test=pytest
test = pytest

[coverage:run]
branch = True
Expand All @@ -25,7 +24,6 @@ testpaths = tests integration-tests
python_classes = *TestSuite
junit_family = xunit2

# Define `python setup.py build_sphinx`
[build_sphinx]
source-dir = docs
build-dir = docs/_build
Expand Down
3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,5 +55,6 @@
],
extras_require={
# see conda/environment.yml
}
},
package_data={'matchms': ['data/*.yaml']},
jspaaks marked this conversation as resolved.
Show resolved Hide resolved
)