Update documentation in preparation for release of v2.5.4

BOWTIE2_VERSION

@@ -1 +1 @@
-2.5.3
+2.5.4

MANUAL

@@ -153,28 +153,33 @@ wrapper will help determine the appropriate index type for uncompressed
 and gzipped inputs.
 
 To build bowtie2-build with libsais first make sure that the libsais
-submodule is available. This can be done in one of the following ways: *
-first time cloning bowtie2 --
-git clone --recursive https://github.com/BenLangmead/bowtie2.git *
-existing checkout of bowtie2 --
-git submodule init && git submodule update
+submodule is available. This can be done in one of the following ways:
 
-Issue the following command line to build libsais: * with OpenMP support
--- [g]make libsais USE_SAIS_OPENMP=1 * without OpenMP support --
-[g]make libsais USE_SAIS=1
+-   first time cloning bowtie2 --
+    git clone --recursive https://github.com/BenLangmead/bowtie2.git
+-   existing checkout of bowtie2 --
+    git submodule init && git submodule update
+
+Issue the following command line to build libsais:
+
+-   with OpenMP support -- [g]make libsais USE_SAIS_OPENMP=1
+-   without OpenMP support -- [g]make libsais USE_SAIS=1
 
 The choice of using OpenMP will determine whether or not the algorithm
 runs multithreaded. The [-p/--threads] argument to bowtie2-build will be
 ignored when libsais is compiled without OpenMP support.
 
-Finally, building the build executable: * with OpenMP support --
-[g]make bowtie2-build-s USE_SAIS_OPENMP=1 * without OpenMP support --
-[g]make bowtie2-build-s USE_SAIS=1
+Finally, building the bowtie2-build executable:
+
+-   with OpenMP support -- [g]make bowtie2-build-s USE_SAIS_OPENMP=1
+-   without OpenMP support -- [g]make bowtie2-build-s USE_SAIS=1
 
 Building with CMake
 
 To build Bowtie2 with SRA and libsais support issue the following
-command: * cmake . -D USE_SRA=1 -D USE_SAIS=1 && cmake --build .
+command:
+
+-   cmake . -D USE_SRA=1 -D USE_SAIS=1 && cmake --build .
 
 CMake will take care of building and linking against the specified
 dependencies.
@@ -816,7 +821,7 @@ alignment to be considered valid, and x is the read length.
 
 Usage
 
-    bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i> | --sra-acc <acc> | b <bam>} -S [<sam>]
+    bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i> | --sra-acc <acc> | -b <bam>} -S [<sam>]
 
 Main arguments
 
@@ -1489,6 +1494,17 @@ Use '='/'X', instead of 'M', to specify matches/mismatches in SAM record
 Append FASTA/FASTQ comment to SAM record, where a comment is everything
 after the first space in the read name.
 
+    --sam-opt-config <config>
+
+Use <config> to toggle SAM Optional Fields where <config> is a string of
+comma delimited, case-insensitive, two-letter tags. Tags prefixed with a
+"-" will be turned off and hence will not be included in the SAM output.
+The example below turns off the "MD" tag and enables the
+bowtie2-specific "YP" tag. The config is additive, so, any default OPT
+flags that need to be turned off will have to explicitly specified.
+
+    `bowtie2 ... --sam-opt-config "-md,yp"
+
 Performance options
 
     -o/--offrate <int>

MANUAL.markdown

@@ -158,24 +158,28 @@ will help determine the appropriate index type for uncompressed and gzipped inpu
 
 To build `bowtie2-build` with [libsais] first make sure that the libsais submodule
 is available. This can be done in one of the following ways:
+
 * first time cloning bowtie2 -- `git clone --recursive https://github.com/BenLangmead/bowtie2.git`
 * existing checkout of bowtie2 -- `git submodule init && git submodule update`
 
 Issue the following command line to build libsais:
+
 * with OpenMP support -- `[g]make libsais USE_SAIS_OPENMP=1`
 * without OpenMP support -- `[g]make libsais USE_SAIS=1`
 
 The choice of using OpenMP will determine whether or not the algorithm
 runs multithreaded. The [`-p/--threads`] argument to `bowtie2-build` will
 be ignored when libsais is compiled without OpenMP support.
 
-Finally, building the build executable:
+Finally, building the bowtie2-build executable:
+
 * with OpenMP support -- `[g]make bowtie2-build-s USE_SAIS_OPENMP=1`
 * without OpenMP support -- `[g]make bowtie2-build-s USE_SAIS=1`
 
 ### Building with CMake ###
 
 To build Bowtie2 with SRA and libsais support issue the following command:
+
 * `cmake . -D USE_SRA=1 -D USE_SAIS=1 && cmake --build .`
 
 CMake will take care of building and linking against the specified dependencies.
@@ -813,7 +817,7 @@ considered valid, and `x` is the read length.
 
 ### Usage
 
-    bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i> | --sra-acc <acc> | b <bam>} -S [<sam>]
+    bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i> | --sra-acc <acc> | -b <bam>} -S [<sam>]
 
 ### Main arguments
 
@@ -1908,6 +1912,23 @@ Use `'='/'X'`, instead of `'M'`, to specify matches/mismatches in SAM record
 Append FASTA/FASTQ comment to SAM record, where a comment is everything
 after the first space in the read name.
 
+</td></tr>
+<tr><td id="bowtie2-options-sam-opt-config">
+
+    --sam-opt-config <config>
+
+</td><td>
+
+Use `<config>` to toggle SAM Optional Fields where `<config>` is a
+string of comma delimited, case-insensitive, two-letter tags.  Tags
+prefixed with a "-" will be turned off and hence will not be included
+in the SAM output. The example below turns off the "MD" tag and
+enables the `bowtie2`-specific "YP" tag. The config is additive, so,
+any default OPT flags that need to be turned off will have to
+explicitly specified.
+
+    `bowtie2 ... --sam-opt-config "-md,yp"
+
 </td></tr>
 </table>
 
@@ -2931,6 +2952,8 @@ for more details and variations on this process.
 [`--very-sensitive-local`]:                           #bowtie2-options-very-sensitive-local
 [`--very-sensitive`]:                                 #bowtie2-options-very-sensitive
 [`--xeq`]:                                            #bowtie2-options-xeq
+[`--sam-append-comment`]:                             #bowtie2-options-sam-append-comment
+[`--sam-opt-config`]:                                 #bowtie2-options-sam-opt-config
 [`-1`]:                                               #bowtie2-options-1
 [`-2`]:                                               #bowtie2-options-2
 [`-3`/`--trim3`]:                                     #bowtie2-options-3

NEWS

@@ -4,7 +4,7 @@ Bowtie 2 NEWS
 Bowtie 2 is available for download from the project website,
 http://bowtie-bio.sf.net/bowtie2 and on Github,
 https://github.com/BenLangmead/bowtie2/releases.  2.0.0-beta1 is
-the first version released to the public and 2.5.3 is the latest
+the first version released to the public and 2.5.4 is the latest
 version.  Bowtie 2 is licensed under the GPLv3 license.  See `LICENSE'
 file for details.
 
@@ -19,6 +19,20 @@ Please report any issues to the Bowtie 2 Github page or using the Sourceforge bu
 Version Release History
 =======================
 
+## Version 2.5.4 - May 15, 2024 ##
+
+### bowtie2 ###
+   * Added `--sam-opt-config` command line option for toggling SAM Opt flags.
+     See MANUAL for details.
+   * Fixed an issue causing `bowtie2`'s memory usage to increase over time
+     when aligning BAM files.
+   * Changed bowtie2 to continue flushing output in the event of a partial write.
+   * Changed the behavior of `bowtie2-build` to throw an exception if it is unable
+     to write the BWT (.1.bt2, .1.rev.bt2). In prior versions `bowtie2-build`, would silently
+     ignore the error which has led some to report the absence of the BWT files in
+     a "completed" index build.
+   * Reverted the changes made in v2.5.0 that sometimes caused unique concordant alignments to be overcounted.
+
 ## Version 2.5.3 - Jan 16, 2024 ##
 
    * Fixed an issue causing `bowtie2`'s memory usage to increase over time.

README.md

@@ -2,7 +2,7 @@
 <!-- badges: start -->
 ![Random Tests](https://github.com/BenLangmead/bowtie2/actions/workflows/random-tests.yml/badge.svg)
 ![Simple Tests](https://github.com/BenLangmead/bowtie2/actions/workflows/simple-tests.yml/badge.svg)
-[![Version](https://img.shields.io/badge/version-2.5.3-green.svg)](https://shields.io/)
+[![Version](https://img.shields.io/badge/version-2.5.4-green.svg)](https://shields.io/)
 <!-- [![Build Status](https://travis-ci.org/BenLangmead/bowtie2.svg?branch=master)](https://travis-ci.org/BenLangmead/bowtie2) -->
 [![License: GPL v3](https://img.shields.io/badge/license-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
 <!--badges: end -->

bt2_search.cpp

@@ -878,9 +878,11 @@ static void printUsage(ostream& out) {
 	    << "                     at the expense of generating non-standard SAM." << endl
 	    << "  --xeq              Use '='/'X', instead of 'M,' to specify matches/mismatches in SAM record." << endl
 	    << "  --soft-clipped-unmapped-tlen" << endl
-	    << "                     Exclude soft-clipped bases when reporting TLEN" << endl
+	    << "                     Exclude soft-clipped bases when reporting TLEN." << endl
 	    << "  --sam-append-comment" << endl
-	    << "                     Append FASTA/FASTQ comment to SAM record" << endl
+	    << "                     Append FASTA/FASTQ comment to SAM record." << endl
+            << "  --sam-opt-config <config>" << endl
+            << "                     Use <config>, example '-MD,YP,-AS', to toggle SAM Optional fields." << endl
 	    << endl
 	    << " Performance:" << endl
 		//    << "  -o/--offrate <int> override offrate of index; must be >= index's offrate" << endl

doc/manual.html

@@ -23,9 +23,6 @@
     .display.math{display: block; text-align: center; margin: 0.5rem auto;}
   </style>
   <link rel="stylesheet" href="doc/style.css" />
-  <!--[if lt IE 9]>
-    <script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
-  <![endif]-->
 </head>
 <body>
 <h1>Table of Contents</h1>
@@ -372,24 +369,37 @@ <h3 id="building-with-libsais-support">Building with libsais
 <p>To build <code>bowtie2-build</code> with <a
 href="https://github.com/IlyaGrebnov/libsais">libsais</a> first make
 sure that the libsais submodule is available. This can be done in one of
-the following ways: * first time cloning bowtie2 --
-<code>git clone --recursive https://github.com/BenLangmead/bowtie2.git</code>
-* existing checkout of bowtie2 --
-<code>git submodule init &amp;&amp; git submodule update</code></p>
-<p>Issue the following command line to build libsais: * with OpenMP
-support -- <code>[g]make libsais USE_SAIS_OPENMP=1</code> * without
-OpenMP support -- <code>[g]make libsais USE_SAIS=1</code></p>
+the following ways:</p>
+<ul>
+<li>first time cloning bowtie2 --
+<code>git clone --recursive https://github.com/BenLangmead/bowtie2.git</code></li>
+<li>existing checkout of bowtie2 --
+<code>git submodule init &amp;&amp; git submodule update</code></li>
+</ul>
+<p>Issue the following command line to build libsais:</p>
+<ul>
+<li>with OpenMP support --
+<code>[g]make libsais USE_SAIS_OPENMP=1</code></li>
+<li>without OpenMP support --
+<code>[g]make libsais USE_SAIS=1</code></li>
+</ul>
 <p>The choice of using OpenMP will determine whether or not the
 algorithm runs multithreaded. The [<code>-p/--threads</code>] argument
 to <code>bowtie2-build</code> will be ignored when libsais is compiled
 without OpenMP support.</p>
-<p>Finally, building the build executable: * with OpenMP support --
-<code>[g]make bowtie2-build-s USE_SAIS_OPENMP=1</code> * without OpenMP
-support -- <code>[g]make bowtie2-build-s USE_SAIS=1</code></p>
+<p>Finally, building the bowtie2-build executable:</p>
+<ul>
+<li>with OpenMP support --
+<code>[g]make bowtie2-build-s USE_SAIS_OPENMP=1</code></li>
+<li>without OpenMP support --
+<code>[g]make bowtie2-build-s USE_SAIS=1</code></li>
+</ul>
 <h3 id="building-with-cmake">Building with CMake</h3>
 <p>To build Bowtie2 with SRA and libsais support issue the following
-command: *
-<code>cmake . -D USE_SRA=1 -D USE_SAIS=1 &amp;&amp; cmake --build .</code></p>
+command:</p>
+<ul>
+<li><code>cmake . -D USE_SRA=1 -D USE_SAIS=1 &amp;&amp; cmake --build .</code></li>
+</ul>
 <p>CMake will take care of building and linking against the specified
 dependencies.</p>
 <h2 id="adding-to-path">Adding to PATH</h2>
@@ -1027,7 +1037,7 @@ <h3 id="setting-function-options">Setting function options</h3>
 necessary for an alignment to be considered valid, and <code>x</code> is
 the read length.</p>
 <h3 id="usage">Usage</h3>
-<pre><code>bowtie2 [options]* -x &lt;bt2-idx&gt; {-1 &lt;m1&gt; -2 &lt;m2&gt; | -U &lt;r&gt; | --interleaved &lt;i&gt; | --sra-acc &lt;acc&gt; | b &lt;bam&gt;} -S [&lt;sam&gt;]</code></pre>
+<pre><code>bowtie2 [options]* -x &lt;bt2-idx&gt; {-1 &lt;m1&gt; -2 &lt;m2&gt; | -U &lt;r&gt; | --interleaved &lt;i&gt; | --sra-acc &lt;acc&gt; | -b &lt;bam&gt;} -S [&lt;sam&gt;]</code></pre>
 <h3 id="main-arguments">Main arguments</h3>
 <table>
 <tr>
@@ -2183,6 +2193,22 @@ <h4 id="sam-options">SAM options</h4>
 everything after the first space in the read name.</p>
 </td>
 </tr>
+<tr>
+<td id="bowtie2-options-sam-opt-config">
+<pre><code>--sam-opt-config &lt;config&gt;</code></pre>
+</td>
+<td>
+<p>Use <code>&lt;config&gt;</code> to toggle SAM Optional Fields where
+<code>&lt;config&gt;</code> is a string of comma delimited,
+case-insensitive, two-letter tags. Tags prefixed with a "-" will be
+turned off and hence will not be included in the SAM output. The example
+below turns off the "MD" tag and enables the
+<code>bowtie2</code>-specific "YP" tag. The config is additive, so, any
+default OPT flags that need to be turned off will have to explicitly
+specified.</p>
+<pre><code>`bowtie2 ... --sam-opt-config &quot;-md,yp&quot;</code></pre>
+</td>
+</tr>
 </table>
 <h4 id="performance-options">Performance options</h4>
 <table>

doc/website/manual.ssi

@@ -338,24 +338,37 @@ index type for uncompressed and gzipped inputs.</p>
 <p>To build <code>bowtie2-build</code> with <a
 href="https://github.com/IlyaGrebnov/libsais">libsais</a> first make
 sure that the libsais submodule is available. This can be done in one of
-the following ways: * first time cloning bowtie2 --
-<code>git clone --recursive https://github.com/BenLangmead/bowtie2.git</code>
-* existing checkout of bowtie2 --
-<code>git submodule init &amp;&amp; git submodule update</code></p>
-<p>Issue the following command line to build libsais: * with OpenMP
-support -- <code>[g]make libsais USE_SAIS_OPENMP=1</code> * without
-OpenMP support -- <code>[g]make libsais USE_SAIS=1</code></p>
+the following ways:</p>
+<ul>
+<li>first time cloning bowtie2 --
+<code>git clone --recursive https://github.com/BenLangmead/bowtie2.git</code></li>
+<li>existing checkout of bowtie2 --
+<code>git submodule init &amp;&amp; git submodule update</code></li>
+</ul>
+<p>Issue the following command line to build libsais:</p>
+<ul>
+<li>with OpenMP support --
+<code>[g]make libsais USE_SAIS_OPENMP=1</code></li>
+<li>without OpenMP support --
+<code>[g]make libsais USE_SAIS=1</code></li>
+</ul>
 <p>The choice of using OpenMP will determine whether or not the
 algorithm runs multithreaded. The [<code>-p/--threads</code>] argument
 to <code>bowtie2-build</code> will be ignored when libsais is compiled
 without OpenMP support.</p>
-<p>Finally, building the build executable: * with OpenMP support --
-<code>[g]make bowtie2-build-s USE_SAIS_OPENMP=1</code> * without OpenMP
-support -- <code>[g]make bowtie2-build-s USE_SAIS=1</code></p>
+<p>Finally, building the bowtie2-build executable:</p>
+<ul>
+<li>with OpenMP support --
+<code>[g]make bowtie2-build-s USE_SAIS_OPENMP=1</code></li>
+<li>without OpenMP support --
+<code>[g]make bowtie2-build-s USE_SAIS=1</code></li>
+</ul>
 <h3 id="building-with-cmake">Building with CMake</h3>
 <p>To build Bowtie2 with SRA and libsais support issue the following
-command: *
-<code>cmake . -D USE_SRA=1 -D USE_SAIS=1 &amp;&amp; cmake --build .</code></p>
+command:</p>
+<ul>
+<li><code>cmake . -D USE_SRA=1 -D USE_SAIS=1 &amp;&amp; cmake --build .</code></li>
+</ul>
 <p>CMake will take care of building and linking against the specified
 dependencies.</p>
 <h2 id="adding-to-path">Adding to PATH</h2>
@@ -993,7 +1006,7 @@ the function <code>f(x)</code> sets the minimum alignment score
 necessary for an alignment to be considered valid, and <code>x</code> is
 the read length.</p>
 <h3 id="usage">Usage</h3>
-<pre><code>bowtie2 [options]* -x &lt;bt2-idx&gt; {-1 &lt;m1&gt; -2 &lt;m2&gt; | -U &lt;r&gt; | --interleaved &lt;i&gt; | --sra-acc &lt;acc&gt; | b &lt;bam&gt;} -S [&lt;sam&gt;]</code></pre>
+<pre><code>bowtie2 [options]* -x &lt;bt2-idx&gt; {-1 &lt;m1&gt; -2 &lt;m2&gt; | -U &lt;r&gt; | --interleaved &lt;i&gt; | --sra-acc &lt;acc&gt; | -b &lt;bam&gt;} -S [&lt;sam&gt;]</code></pre>
 <h3 id="main-arguments">Main arguments</h3>
 <table>
 <tr>
@@ -2149,6 +2162,22 @@ matches/mismatches in SAM record</p>
 everything after the first space in the read name.</p>
 </td>
 </tr>
+<tr>
+<td id="bowtie2-options-sam-opt-config">
+<pre><code>--sam-opt-config &lt;config&gt;</code></pre>
+</td>
+<td>
+<p>Use <code>&lt;config&gt;</code> to toggle SAM Optional Fields where
+<code>&lt;config&gt;</code> is a string of comma delimited,
+case-insensitive, two-letter tags. Tags prefixed with a "-" will be
+turned off and hence will not be included in the SAM output. The example
+below turns off the "MD" tag and enables the
+<code>bowtie2</code>-specific "YP" tag. The config is additive, so, any
+default OPT flags that need to be turned off will have to explicitly
+specified.</p>
+<pre><code>`bowtie2 ... --sam-opt-config &quot;-md,yp&quot;</code></pre>
+</td>
+</tr>
 </table>
 <h4 id="performance-options">Performance options</h4>
 <table>

doc/website/recent_news.ssi

@@ -1,3 +1,21 @@
+<h2 id="version-2.5.4---may-15-2024">Version 2.5.4 - May 15, 2024</h2>
+<h3 id="bowtie2">bowtie2</h3>
+<ul>
+  <li>Added <code><a href="manual.shtml#bowtie2-sam-opt-config">--sam-opt-config</a></code> command line option for toggling
+    SAM Opt flags. See MANUAL for details.</li>
+  <li>Fixed an issue causing <code>bowtie2</code>’s memory usage to
+    increase over time when aligning BAM files.</li>
+  <li>Changed bowtie2 to continue flushing output in the event of a
+    partial write.</li>
+  <li>Changed the behavior of <code>bowtie2-build</code> to throw an
+    exception if it is unable to write the BWT (.1.bt2, .1.rev.bt2). In
+    prior versions <code>bowtie2-build</code>, would silently ignore the
+    error which has led some to report the absence of the BWT files in a
+    “completed” index build.</li>
+  <li>Reverted the changes made in v2.5.0 that sometimes caused unique
+    concordant alignments to be overcounted.</li>
+</ul>
+
 <h2>New Indexes for Cow, Chimp, and Rat, added to the sidebar - May 08, 2024</h2>
 <ul>
    <li><a href="https://genome-idx.s3.amazonaws.com/bt/ARS-UCD2.0.zip"><i>B. taurus</i>, ARS-UCD2.0</a></li>