Merge branch 'master' into ci/py313

.devcontainer.json

@@ -5,10 +5,7 @@
 	// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
 	"image": "mcr.microsoft.com/devcontainers/base:jammy",
 	"features": {
-		"ghcr.io/rocker-org/devcontainer-features/miniforge:1": {
-			"version": "latest",
-			"variant": "Mambaforge"
-		}
+		"ghcr.io/rocker-org/devcontainer-features/miniforge:2": {}
 	},
 
 	// Features to add to the dev container. More info: https://containers.dev/features.

.github/workflows/conventional-prs.yml

@@ -18,6 +18,6 @@ jobs:
       statuses: write  # for amannn/action-semantic-pull-request to mark status of analyzed PR
     runs-on: ubuntu-latest
     steps:
-      - uses: amannn/action-semantic-pull-request@v5.4.0
+      - uses: amannn/action-semantic-pull-request@v5
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/tests.yml

@@ -36,11 +36,10 @@ jobs:
       - name: Check out the repository
         uses: actions/checkout@v4
 
-      - name: Setup Mambaforge
+      - name: Setup Miniforge
         uses: conda-incubator/setup-miniconda@v3
         with:
           activate-environment: trtools
-          miniforge-variant: Mambaforge
           auto-activate-base: false
           miniforge-version: latest
           use-mamba: true
@@ -103,7 +102,7 @@ jobs:
         uses: actions/checkout@v4
 
       - name: Check for large files
-        uses: actionsdesk/lfs-warning@v3.2
+        uses: ppremk/lfs-warning@v3.3
         with:
           token: ${{ secrets.GITHUB_TOKEN }} # Optional
           filesizelimit: 500000b

CHANGELOG.md

@@ -1,5 +1,30 @@
 # Changelog
 
+## [6.1.0](https://github.com/gymrek-lab/TRTools/compare/v6.0.2...v6.1.0) (2024-11-06)
+
+
+### Features
+
+* Add longtr support ([#232](https://github.com/gymrek-lab/TRTools/issues/232)) ([93ff231](https://github.com/gymrek-lab/TRTools/commit/93ff231a74a8b0cd03ee957b8374abfc9451119d))
+* Add options to facilitate debugging annotaTR runs on large files ([#234](https://github.com/gymrek-lab/TRTools/issues/234)) ([0d5d96b](https://github.com/gymrek-lab/TRTools/commit/0d5d96b8699d043aa88dc62866ded0465e0ba713))
+* adding the annotaTR tool ([#226](https://github.com/gymrek-lab/TRTools/issues/226)) ([59dd532](https://github.com/gymrek-lab/TRTools/commit/59dd5323b6d58b38abe36836fbfe13f789ff4048))
+* support for Apple M1 silicon ([#227](https://github.com/gymrek-lab/TRTools/issues/227)) ([dfec9c2](https://github.com/gymrek-lab/TRTools/commit/dfec9c229955f2862fb796d3d3360d1044f9cdc8))
+
+
+### Bug Fixes
+
+* associatr test files overwritten during pytest ([#230](https://github.com/gymrek-lab/TRTools/issues/230)) ([2b955be](https://github.com/gymrek-lab/TRTools/commit/2b955bed4d7d81ad5ba25a392fdd81df90a534d9))
+* bug in beagle dosages with no alt alleles ([#231](https://github.com/gymrek-lab/TRTools/issues/231)) ([ae3ca15](https://github.com/gymrek-lab/TRTools/commit/ae3ca15e64617f8502cb56a71c839ef8a35f524e))
+* enable filter_hrun with beagle file in dumpSTR ([be63e79](https://github.com/gymrek-lab/TRTools/commit/be63e79bb6402276e82cd7aea85bb76942217f75))
+
+
+### Documentation
+
+* generate internal links dynamically ([#228](https://github.com/gymrek-lab/TRTools/issues/228)) ([9402042](https://github.com/gymrek-lab/TRTools/commit/94020423b8667caeb20ea2615008d7441dfe5ac4))
+* mention github codespaces and clarify dependency management ([#223](https://github.com/gymrek-lab/TRTools/issues/223)) ([45ccf28](https://github.com/gymrek-lab/TRTools/commit/45ccf28c306bcd98c6bc2b285d659571bf9afb6d))
+* update biorxiv citation for prancSTR ([#235](https://github.com/gymrek-lab/TRTools/issues/235)) ([1e45d23](https://github.com/gymrek-lab/TRTools/commit/1e45d233afbf23e8d738786546f7c89ab6fe6805))
+* warn about scipy deprecation in dumpSTR ([#221](https://github.com/gymrek-lab/TRTools/issues/221)) ([a1a3a39](https://github.com/gymrek-lab/TRTools/commit/a1a3a39b7b7fd2d2f694d3a41bfe7bcf72ff72c3))
+
 ## [6.0.2](https://github.com/gymrek-lab/TRTools/compare/v6.0.1...v6.0.2) (2024-06-24)
 
 

doc/index.rst

@@ -62,7 +62,11 @@ Note: if you will run or test :code:`simTR`, you will also need to install `ART
 From source
 ^^^^^^^^^^^
 
-To install from source (only recommended for development) clone the TRTools repository from `github <https://github.com/gymrek-lab/TRTools/>`_ and checkout the branch you're interested in::
+If you would like to develop or edit the TRTools source code, you will need to perform a "dev install" directly from the source.
+
+**Note:** Instead of performing the following steps, you can also just open `a GitHub codespace <https://docs.github.com/en/codespaces/overview>`_. Simply type a comma "," when viewing a branch on `our GitHub <https://github.com/gymrek-lab/TRTools>`_ to open an editor with our development setup pre-installed. You can then run :code:`conda activate trtools` and :code:`poetry shell` in the terminal there.
+
+You can clone the TRTools repository from `github <https://github.com/gymrek-lab/TRTools/>`_ and checkout the branch you're interested in::
 
         git clone -b master https://github.com/gymrek-lab/TRTools
         cd TRTools/
@@ -77,6 +81,11 @@ Now, whenever you'd like to run/import pytest or TRTools, you will first need to
         conda activate trtools
         poetry shell
 
+.. note::
+    There's no need to install TRTools this way if you aren't planning to develop or edit the source code! If you want the latest version from our master branch and just can't wait for us to release it, you only need to run::
+
+        pip install --upgrade --force-reinstall git+https://github.com/gymrek-lab/trtools.git@master
+
 With Docker
 ^^^^^^^^^^^
 
@@ -163,8 +172,10 @@ We appreciate contributions to TRTools. If you would like to contribute a fix or
 #. Make your changes. 
 #. If you need to add a dependency or update the version of a dependency, you can use the :code:`poetry add` command.
 
-    * You should specify a `version constraint <https://python-poetry.org/docs/master/dependency-specification#version-constraints>`_ when adding a dependency. Use the oldest version compatible with your code. Don't worry if you're not sure at first, since you can (and should!) always update it later. For example, to specify a version of :code:`numpy>=1.23.0`, you can run :code:`poetry add 'numpy>=1.23.0'`.
-    * Afterwards, double-check that the :code:`poetry.lock` file contains 1.23.0 in it. **All of our dependencies should be locked to their minimum versions at all times.** To downgrade to a specific version of :code:`numpy` in our lock file, you can explicitly add the version via :code:`poetry add 'numpy==1.23.0'`, manually edit the pyproject.toml file to use a :code:`>=` sign in front of the version number, and then run :code:`poetry lock --no-update`.
+    * You should specify a `version constraint <https://python-poetry.org/docs/master/dependency-specification#version-constraints>`_ when adding a dependency. Use the oldest version compatible with your code. For example, to specify a version of :code:`numpy>=1.23.0`, you can run :code:`poetry add 'numpy>=1.23.0'`. Note: For most cases, a version constraint operator of :code:`>=` `is better than <https://iscinumpy.dev/post/bound-version-constraints>`_ poetry's default of :code:`^`.
+    * Afterward, double-check that the :code:`poetry.lock` file contains 1.23.0 in it. **All of our dependencies should be locked to their minimum versions at all times.** To downgrade to a specific version of :code:`numpy` in our lock file, you can explicitly add the version via :code:`poetry add 'numpy==1.23.0'`, manually edit the pyproject.toml file to use a :code:`>=` sign in front of the version number, and then run :code:`poetry lock --no-update`. The :code:`--no-update` is important because otherwise, poetry will try to update other dependencies in the lock file.
+    * Only PyPI packages can be added to our pyproject.toml file. So if a dependency is only available on conda, then you can add it to our :code:`dev-env.yml` file instead. Please note that anyone who installs TRTools from PyPI will not be guaranteed to have your dependency installed, so you should design your code accordingly. 
+    * Any changes to our dependencies must also added to our bioconda recipe at the time of publication. See `PUBLISHING.rst <https://github.com/gymrek-lab/TRTools/blob/master/PUBLISHING.rst>`_ for more details.
 
 #. Document your changes.
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "trtools"
-version = "6.0.2"
+version = "6.1.0"
 authors = [
     "Melissa Gymrek <[email protected]>",
     "Gymrek Lab",

trtools/prancSTR/README.rst

@@ -45,12 +45,12 @@ Other general parameters:
 * :code:`--region <string>`: Restrict to the region chr:start-end. VCF file must be bgzipped and indexed to use this option.
 * :code:`--samples <string>`: Restrict to the given list of samples. Samples are comma separated.
 * :code:`--vcftype <string>`: Specify the tool which generated the vcf call file for STRs. Currently this will fail if using anything other than :code:`hipstr` VCFs.
-* :code:`--only-passing`: Filters out the VCF records with non-passing FILTER column
-* :code:`--output-all`: Force tool to output results for all loci. Overrides :code:``--only-passing``.
+* :code:`--only-passing`: Filters out the VCF records with non-passing FILTER column.
+* :code:`--output-all`: Force tool to output results for all loci. Overrides :code:`--only-passing`.
 * :code:`--readfield <string>`: Specify which VCF format field output by HipSTR to utilize for extracting read information. We recommend setting this to "MALLREADS". "ALLREADS" is also accepted but we have found that it produces unreliable results.
 * :code:`--debug`: Print helpful debug messages.
 * :code:`--quiet`: Restrict printing of any messages.
-* :code:`--version`: Print the version of the tool
+* :code:`--version`: Print the version of the tool.
 
 Notes:
 

trtools/statSTR/README.rst

@@ -30,6 +30,7 @@ Optional general parameters:
 * :code:`--sample-prefixes <string>`: The prefixes to name output for each samples group. By default uses 1, 2, 3 etc. Must be sample length as :code:`--samples`.
 * :code:`--region <string>`: Restrict to specific regions (chrom:start-end). Requires the input VCF to be bgzipped and tabix indexed.
 * :code:`--precision <int>`: How much precision to use when writing stats (default = 3)
+* :code:`--only-passing`: Filters out the VCF records with non-passing FILTER column.
 
 For specific statistics available, see below.
 

trtools/statSTR/statSTR.py

@@ -453,6 +453,8 @@ def getargs(): # pragma: no cover
     filter_group.add_argument("--region", help="Restrict to the region "
                               "chrom:start-end. Requires file to bgzipped and"
                               " tabix indexed.", type=str)
+    filter_group.add_argument("--only-passing", help="Only process records "
+                              " where FILTER==PASS", action="store_true")
     stat_group_name = "Stats group"
     stat_group = parser.add_argument_group(stat_group_name)
     stat_group.add_argument("--thresh", help="Output threshold field (max allele size, used for GangSTR strinfo).", action="store_true")
@@ -574,6 +576,10 @@ def main(args):
             nrecords += 1
 
             trrecord = trh.HarmonizeRecord(vcftype, record)
+
+            if args.only_passing and record.FILTER is not None:
+                continue
+
             if args.plot_afreq and num_plotted <= MAXPLOTS:
                 PlotAlleleFreqs(trrecord, args.out, sample_indexes=sample_indexes, sampleprefixes=sample_prefixes)
                 num_plotted += 1

trtools/statSTR/tests/test_statSTR.py

@@ -17,6 +17,7 @@ def args(tmpdir):
     args.sample_prefixes = None
     args.plot_afreq = False
     args.region = None
+    args.only_passing = False
     args.thresh = False
     args.afreq = False
     args.acount = False
@@ -49,6 +50,17 @@ def test_RightFile(args, vcfdir):
     retcode = main(args)
     assert retcode==0
 
+# Test rhe only passing option
+def test_OnlyPassing(args, vcfdir):
+    fname = os.path.join(vcfdir, "CEU_test.vcf.gz")
+    args.vcf = fname
+
+    # With only passing
+    args.only_passing = True
+    args.region = None
+    retcode = main(args)
+    assert retcode==0
+
 # Test all the statSTR options
 def test_Stats(args, vcfdir, capsys):
     fname = os.path.join(vcfdir, "few_samples_few_loci.vcf.gz")