Improve LCOV coverage statistics and fix bugs that showed up in pcre2…

…grep while doing this.

ChangeLog

@@ -51,6 +51,24 @@ a 64-bit system.
 
 13. Implemented -Z aka --null in pcre2grep.
 
+14. A minor change to pcre2test and the addition of several new pcre2grep tests
+have improved LCOV coverage statistics. At the same time, code in pcre2grep and 
+elsewhere that can never be obeyed in normal testing has been excluded from
+coverage.
+
+15. Fixed a bug in pcre2grep that could cause an extra newline to be written
+after output generaed by --output.
+
+16. If a file has a .bz2 extension but is not in fact compressed, pcre2grep 
+should process it as a plain text file. A bug stopped this happening; now fixed 
+and added to the tests.
+
+17. When pcre2grep was running not in UTF mode, if a string specified by
+--output or obtained from a callout in a pattern contained a character (byte)
+greater than 127, it was incorrectly output in UTF-8 format.
+
+18. Added some casts after warnings from Clang sanitize.
+
 
 Version 10.40 15-April-2022
 ---------------------------

doc/html/pcre2grep.html

@@ -136,14 +136,15 @@ <h1>pcre2grep man page</h1>
 </P>
 <br><a name="SEC3" href="#TOC1">SUPPORT FOR COMPRESSED FILES</a><br>
 <P>
-It is possible to compile <b>pcre2grep</b> so that it uses <b>libz</b> or
-<b>libbz2</b> to read compressed files whose names end in <b>.gz</b> or
+Compile-time options for <b>pcre2grep</b> can set it up to use <b>libz</b> or
+<b>libbz2</b> for reading compressed files whose names end in <b>.gz</b> or
 <b>.bz2</b>, respectively. You can find out whether your <b>pcre2grep</b> binary
 has support for one or both of these file types by running it with the
 <b>--help</b> option. If the appropriate support is not present, all files are
-treated as plain text. The standard input is always so treated. When input is
-from a compressed .gz or .bz2 file, the <b>--line-buffered</b> option is
-ignored.
+treated as plain text. The standard input is always so treated. If a file with
+a <b>.gz</b> or <b>.bz2</b> extension is not in fact compressed, it is read as a
+plain text file. When input is from a compressed .gz or .bz2 file, the
+<b>--line-buffered</b> option is ignored.
 </P>
 <br><a name="SEC4" href="#TOC1">BINARY FILES</a><br>
 <P>
@@ -1066,7 +1067,7 @@ <h1>pcre2grep man page</h1>
 </P>
 <br><a name="SEC16" href="#TOC1">REVISION</a><br>
 <P>
-Last updated: 30 July 2022
+Last updated: 07 August 2022
 <br>
 Copyright &copy; 1997-2022 University of Cambridge.
 <br>

doc/pcre2grep.1

@@ -1,4 +1,4 @@
-.TH PCRE2GREP 1 "30 July 2022" "PCRE2 10.41"
+.TH PCRE2GREP 1 "07 August 2022" "PCRE2 10.41"
 .SH NAME
 pcre2grep - a grep with Perl-compatible regular expressions.
 .SH SYNOPSIS
@@ -103,14 +103,15 @@ The \fB--locale\fP option can be used to override this.
 .SH "SUPPORT FOR COMPRESSED FILES"
 .rs
 .sp
-It is possible to compile \fBpcre2grep\fP so that it uses \fBlibz\fP or
-\fBlibbz2\fP to read compressed files whose names end in \fB.gz\fP or
+Compile-time options for \fBpcre2grep\fP can set it up to use \fBlibz\fP or
+\fBlibbz2\fP for reading compressed files whose names end in \fB.gz\fP or
 \fB.bz2\fP, respectively. You can find out whether your \fBpcre2grep\fP binary
 has support for one or both of these file types by running it with the
 \fB--help\fP option. If the appropriate support is not present, all files are
-treated as plain text. The standard input is always so treated. When input is
-from a compressed .gz or .bz2 file, the \fB--line-buffered\fP option is
-ignored.
+treated as plain text. The standard input is always so treated. If a file with
+a \fB.gz\fP or \fB.bz2\fP extension is not in fact compressed, it is read as a
+plain text file. When input is from a compressed .gz or .bz2 file, the
+\fB--line-buffered\fP option is ignored.
 .
 .
 .SH "BINARY FILES"
@@ -972,6 +973,6 @@ Cambridge, England.
 .rs
 .sp
 .nf
-Last updated: 30 July 2022
+Last updated: 07 August 2022
 Copyright (c) 1997-2022 University of Cambridge.
 .fi

doc/pcre2grep.txt

@@ -104,13 +104,15 @@ DESCRIPTION
 
 SUPPORT FOR COMPRESSED FILES
 
-       It  is  possible to compile pcre2grep so that it uses libz or libbz2 to
-       read compressed files whose names end in .gz or .bz2, respectively. You
-       can  find out whether your pcre2grep binary has support for one or both
-       of these file types by running it with the --help option. If the appro-
-       priate support is not present, all files are treated as plain text. The
-       standard input is always so treated. When input is  from  a  compressed
-       .gz or .bz2 file, the --line-buffered option is ignored.
+       Compile-time  options for pcre2grep can set it up to use libz or libbz2
+       for reading compressed files whose names end in .gz  or  .bz2,  respec-
+       tively.  You can find out whether your pcre2grep binary has support for
+       one or both of these file types by running it with the  --help  option.
+       If  the  appropriate  support  is not present, all files are treated as
+       plain text. The standard input is always so treated. If a file  with  a
+       .gz  or .bz2 extension is not in fact compressed, it is read as a plain
+       text file. When input is from  a  compressed  .gz  or  .bz2  file,  the
+       --line-buffered option is ignored.
 
 
 BINARY FILES
@@ -1036,5 +1038,5 @@ AUTHOR
 
 REVISION
 
-       Last updated: 30 July 2022
+       Last updated: 07 August 2022
        Copyright (c) 1997-2022 University of Cambridge.

src/pcre2_compile.c

@@ -8554,7 +8554,7 @@ do {
             op == OP_SCBRA || op == OP_SCBRAPOS)
      {
      int n = GET2(scode, 1+LINK_SIZE);
-     int new_map = bracket_map | ((n < 32)? (1u << n) : 1);
+     unsigned int new_map = bracket_map | ((n < 32)? (1u << n) : 1);
      if (!is_startline(scode, new_map, cb, atomcount, inassert)) return FALSE;
      }
 

src/pcre2_context.c

@@ -7,7 +7,7 @@ and semantics are as close as possible to those of the Perl 5 language.
 
                        Written by Philip Hazel
      Original API code Copyright (c) 1997-2012 University of Cambridge
-          New API code Copyright (c) 2016-2018 University of Cambridge
+          New API code Copyright (c) 2016-2022 University of Cambridge
 
 -----------------------------------------------------------------------------
 Redistribution and use in source and binary forms, with or without
@@ -443,8 +443,11 @@ mcontext->offset_limit = limit;
 return 0;
 }
 
-/* This function became obsolete at release 10.30. It is kept as a synonym for
-backwards compatibility. */
+/* These functions became obsolete at release 10.30. The first is kept as a
+synonym for backwards compatibility. The second now does nothing. Exclude both 
+from coverage reports. */
+
+/* LCOV_EXCL_START */
 
 PCRE2_EXP_DEFN int PCRE2_CALL_CONVENTION
 pcre2_set_recursion_limit(pcre2_match_context *mcontext, uint32_t limit)
@@ -464,6 +467,9 @@ pcre2_set_recursion_memory_management(pcre2_match_context *mcontext,
 return 0;
 }
 
+/* LCOV_EXCL_STOP */
+
+
 /* ------------ Convert context ------------ */
 
 PCRE2_EXP_DEFN int PCRE2_CALL_CONVENTION