From 8b65485421f5dbd0f18b42604f039a42c2b7bcc1 Mon Sep 17 00:00:00 2001 From: Jonathan Margoliash Date: Mon, 22 Aug 2022 08:20:46 -0700 Subject: [PATCH 1/2] Delete requirements.txt. Clarify bioconda build process in REQUIREMENTS.rst --- .readthedocs.yml | 2 +- ...onda_env.yml => .readthedocs_conda_env.yml | 0 PUBLISHING.rst | 52 +++++++++++++------ RELEASE_NOTES.rst | 7 +++ doc/requirements.txt | 10 ---- requirements.txt | 10 ---- 6 files changed, 44 insertions(+), 37 deletions(-) rename .trtools_conda_env.yml => .readthedocs_conda_env.yml (100%) delete mode 100644 doc/requirements.txt delete mode 100644 requirements.txt diff --git a/.readthedocs.yml b/.readthedocs.yml index 92f3df3c..a8601014 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -14,5 +14,5 @@ formats: - pdf conda: - environment: .trtools_conda_env.yml + environment: .readthedocs_conda_env.yml diff --git a/.trtools_conda_env.yml b/.readthedocs_conda_env.yml similarity index 100% rename from .trtools_conda_env.yml rename to .readthedocs_conda_env.yml diff --git a/PUBLISHING.rst b/PUBLISHING.rst index 0cc54801..1da56d19 100644 --- a/PUBLISHING.rst +++ b/PUBLISHING.rst @@ -6,27 +6,40 @@ If you are a community member and want to contribute new code, see the contribut If you are a community member and have already contributed new code and want us to publish it now, please contact us (our contact info is in the README) -This document explains how trtools maintainers should publish new changes. +This document explains how trtools maintainers should publish new changes. Maintainers should reach consensus before going ahead with publishing changes. -We use a simplified version of +We use a simplified version of `git flow `_ to maintain and publish trtools. We use the master branch as the default branch with the latest stable codebase. The builds from this branch are distributed to PyPI and conda. The develop branch contains new features that have yet to make their way into master. +New Dependencies +---------------- +If you've added dependencies to trtools or its tests, those dependencies should be listed in + + * setup.py + * the .readthedocs_conda_env.yml file in the root of the repositiory that's used for building + TRTool's Read The Docs webpage. + * the appropriate section of the bioconda recipe (see below) + + +Publishing Steps +---------------- + Once changes have been made to develop that are ready to be published, first choose the new version number. Then set up the environment you're going to publish TRTools from: #. Create a clean environment. #. Install setuptools with version >= 40.8.0 -#. Install all the requirements in requirements.txt #. Additionally, install ``pytest``, ``wheel`` and ``twine`` +#. Clone the `trtools repo `_ +#. Check out the develop branch +#. Run :code:`pip install --upgrade pip && pip install -e .` Then go through the steps of merging the changes into the master branch: -#. Clone the `trtools repo `_ -#. Check out the develop branch #. Run pytest and make sure all the tests pass #. Change the 'Unreleased Changes' section of :code:`RELEASE_NOTES.rst` to the new version number. #. Check if any changes have been made that have not yet been documented in the release notes. If so, document them. @@ -51,15 +64,25 @@ Then go through the steps of publishing the changed code to PyPI: 4. Run :code:`twine upload dist/*` to upload the build to PyPI -Lastly go through the following steps to publish the changed code to bioconda: (see `here `_ for official documentation) +Lastly, the change needs to be published to bioconda. + +A bioconda bot will automatically open a pull request (within a day?) updating the version number +and the PyPI reference. If there are no new dependencies, no changes the to the build, +and no new tests that need to be integrated into the build, and all we need to do is mark that PR as okay. +(Jonathan Margoliash is currently the bioconda recipe maintainer and will get pinged by this. Please notify him to look out for that ping. +If you'd like to be a bioconda recipe maintainer, let's set that up.) + +If there are new dependencies or build changes, then we'll need to close the automatic PR without accepting it and make our own. +To do that, go through the following steps to publish the changed code to bioconda: (see `here `_ for official documentation) 1. Run :code:`cd && openssl sha256 dist/trtools-.tar.gz` and save the generated hash code for later 2. Create a fork of the `bioconda recipes `_ repo and clone it. 3. Run ``git remote add upstream https://github.com/bioconda/bioconda-recipes`` so that you can pull from not only your fork but also upstream. 4. Create a new branch from master with your package's name 5. In `the recipe for trtools `_, update the version number to the latest version, and change the sha256 hashcode to the code you recorded in step 1 above -6. Run :code:`git pull upstream master` to make sure you're up to date with the central repo's master branch -7. Check the recipe by cd'ing to the bioconda-recipes project root and running: +6. Also update any other part of the yaml file as needed to account for the new dependencies/builds steps/etc. +7. Run :code:`git pull upstream master` to make sure you're up to date with the central repo's master branch +8. Check the recipe by cd'ing to the bioconda-recipes project root and running: .. code-block:: bash @@ -80,18 +103,15 @@ Lastly go through the following steps to publish the changed code to bioconda: ( Possible Issues: -* bioconda packages should not include large test data files. If the dist/trtools-.tar.gz file contains such files, you'll need to modify the MANIFEST.in file to exclude them, fix the test_trtools.sh script to download them manually and point pytest to them, confirm the tests run in a :code:`conda build` and then restart the publishing process. (This should not happen if new test files are just put in :code:`trtools/testsupport/sample_vcfs` or :code:`trtools/testsupport/sample_regions`) -* If you've added dependencies to trtools or its tests, those dependencies should be listed in - - * setup.py - * requirements.txt (list a specific version of the dependency that is up to date and that we know will work) - * the appropriate section of the bioconda recipe +* bioconda packages should not include large test data files. If the dist/trtools-.tar.gz file contains such files, you'll need to modify the MANIFEST.in file to exclude them, + fix the test_trtools.sh script to download them manually and point pytest to them, confirm the tests run in a :code:`conda build` and then restart the publishing process. + (This should not happen if new test files are just put in :code:`trtools/testsupport/sample_vcfs` or :code:`trtools/testsupport/sample_regions`) Git Tagging ----------- -Git tags are used to mark specific commits with certain names (i.e. v1.2.0). -Please note that tags are assigned to commits, not branches. +Git tags are used to mark specific commits with certain names (i.e. v1.2.0). +Please note that tags are assigned to commits, not branches. You can tag a commit in two different ways. #. Command line:: diff --git a/RELEASE_NOTES.rst b/RELEASE_NOTES.rst index f2f19a5d..8c9a024d 100644 --- a/RELEASE_NOTES.rst +++ b/RELEASE_NOTES.rst @@ -7,6 +7,13 @@ Bug fixes: a 'stuck in infinite loop' message and crashing * MergeSTR no longer crashes if run with the --verbose flag and the last position in each of the VCFs being merged isn't identical. +* StatSTR now errors out if any of the files listed in --samples contain no samples that are present + in the input VCF. + +Doc changes: + +* Clarified in PUBLISHING.rst how to handle dependencies and how to publish to bioconda. +* requirements.txt was unneeded, so delete it and remove the reference to it from PUBLISHING.rst 4.1.0 ----- diff --git a/doc/requirements.txt b/doc/requirements.txt deleted file mode 100644 index 08ad29e0..00000000 --- a/doc/requirements.txt +++ /dev/null @@ -1,10 +0,0 @@ -# the autodoc_typehints configuration option requires sphinx>=3 -# and sphinx 3.0.4 is the latest released version of sphinx according to -# https://www.sphinx-doc.org/en/master/changes.html -sphinx==3.0.4 - -# include all of trtools's runtime requirements as document -# building requirements, as autodoc imports all python modules -# in order to build their docs, and this will require the entire -# runtime environment to be available --r ../requirements.txt diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 6371f059..00000000 --- a/requirements.txt +++ /dev/null @@ -1,10 +0,0 @@ -pandas==1.0.5 -numpy==1.18.5 -pybedtools==0.8.1 -pytest==5.4.3 -scipy==1.4.1 -scikit-learn==0.23.1 -matplotlib==3.2.2 -pysam==0.16.0.1 -cython==0.29.20 -cyvcf2==0.30.1 \ No newline at end of file From fbabeee3d6c019b0d023d4db6d1c38da63a68a7a Mon Sep 17 00:00:00 2001 From: LiterallyUniqueLogin Date: Mon, 22 Aug 2022 09:20:56 -0700 Subject: [PATCH 2/2] Apply suggestions from code review Grammar fixes Co-authored-by: Arya Massarat <23412689+aryarm@users.noreply.github.com> --- PUBLISHING.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/PUBLISHING.rst b/PUBLISHING.rst index 1da56d19..33dc2c7d 100644 --- a/PUBLISHING.rst +++ b/PUBLISHING.rst @@ -21,7 +21,7 @@ New Dependencies If you've added dependencies to trtools or its tests, those dependencies should be listed in * setup.py - * the .readthedocs_conda_env.yml file in the root of the repositiory that's used for building + * the .readthedocs_conda_env.yml file in the root of the repository that's used for building TRTool's Read The Docs webpage. * the appropriate section of the bioconda recipe (see below) @@ -67,7 +67,7 @@ Then go through the steps of publishing the changed code to PyPI: Lastly, the change needs to be published to bioconda. A bioconda bot will automatically open a pull request (within a day?) updating the version number -and the PyPI reference. If there are no new dependencies, no changes the to the build, +and the PyPI reference. If there are no new dependencies, no changes to the build, and no new tests that need to be integrated into the build, and all we need to do is mark that PR as okay. (Jonathan Margoliash is currently the bioconda recipe maintainer and will get pinged by this. Please notify him to look out for that ping. If you'd like to be a bioconda recipe maintainer, let's set that up.)