From 4e492ee446fff7858665800aa1e1ff0e5ad6ba94 Mon Sep 17 00:00:00 2001 From: fcambus Date: Tue, 17 Mar 2020 14:38:25 +0000 Subject: [PATCH] miller: update to 5.7.0. ChangeLog: Features The new remove-empty-columns and skip-trivial-records are keystroke-savers for things which would other require DSL syntax, as tracked in #274. Bugfixes A bug regarding optional regex-pattern groups was fixed in #277. As of #294 you can now specify --implicit-csv-header for the join-file in mlr join. A bug with spaces in XTAB-file values was fixed on #296. A bug with missing final newline for XTAB-formatted files using MMAP files was fixed on #301. Documentation Look-and-feel at http://johnkerl.org/miller/doc/ is (hopefully) improved, including clearer visual indication of which section/page you're currently looking at. Note that this change has been live for a few weeks, as look-and-feel-related doc-mods from post-5.6.2 were backported to http://johnkerl.org/miller/doc/. #282 improves DSL-function documentation at http://johnkerl.org/miller/doc/reference-dsl.html#Built-in_functions_for_filter_and_put,_summary Note Support for mmap mode has been entirely discontinued. This is an invisible change and should not affect you at all. For anyone interested in lower-level details, though, the summary is as follows: For an incremental performance gain (perhaps 10-20% run time at most, but see below), within the C source code one can use the mmap system call to access input files via pointer arithmetic rather than malloc-and-memcopy using stdio. However mmap is not available when reading from standard input -- it cannot be memory-mapped. This means all file-format readers are implemented twice within the Miller source code. While I try to regression-test Miller thoroughly, running all canned tests through mmap and stdio mode, I've nonetheless found my mmap implementations liable to corner-cases which I miss but users find: for example #29, #102, and #296. As tracked on #160, various operating systems do not release mmapped pages after use as one might intuit, meaning that for large files and/or large numbers of files, I've for a long time now needed to have Miller opt out of mmap usage for precisely those cases which most need the performance gain: see #160, #181, and #256. Additionally, mmap is not used at all for Windows/MSYS2 so there is nothing to lose there. For these reasons, keeping mmap mode isn't worth the development overhead. As of release 5.6.3, the mlr executable will still accept the --mmap and --no-mmap command-line flags as no-ops, for backward compatibility. The caveat for you is that for everyday small files, the default was previously mmap mode and is now stdio (except mlr ... < filename or ... | mlr ... which have always used stdio). There is the off chance that this will newly reveal an old, latent bug or two somewhere. I've re-run regressions in valgrind mode to aggressively catch any errors, but, please let me know ASAP via GitHub issue of any unexpected behavior in 5.7.0. --- textproc/miller/Makefile | 4 ++-- textproc/miller/distinfo | 10 +++++----- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/textproc/miller/Makefile b/textproc/miller/Makefile index ec3463a45072..785b96cfbffb 100644 --- a/textproc/miller/Makefile +++ b/textproc/miller/Makefile @@ -1,6 +1,6 @@ -# $NetBSD: Makefile,v 1.16 2020/03/06 08:18:31 fcambus Exp $ +# $NetBSD: Makefile,v 1.17 2020/03/17 14:38:25 fcambus Exp $ -DISTNAME= mlr-5.6.2 +DISTNAME= mlr-5.7.0 PKGNAME= ${DISTNAME:S/mlr/miller/} CATEGORIES= devel MASTER_SITES= ${MASTER_SITE_GITHUB:=johnkerl/} diff --git a/textproc/miller/distinfo b/textproc/miller/distinfo index 8dd8a6906730..35217b7551db 100644 --- a/textproc/miller/distinfo +++ b/textproc/miller/distinfo @@ -1,6 +1,6 @@ -$NetBSD: distinfo,v 1.15 2020/03/06 08:18:31 fcambus Exp $ +$NetBSD: distinfo,v 1.16 2020/03/17 14:38:25 fcambus Exp $ -SHA1 (mlr-5.6.2.tar.gz) = 4a3fb995a65a9960bb2e53bd565081d491aba8b1 -RMD160 (mlr-5.6.2.tar.gz) = 51e6d16ca6d012e47d8cad29d643c7da943a0535 -SHA512 (mlr-5.6.2.tar.gz) = d5c984c1db045219c79564251193ec4887582987cde980df6705e10e246d230d92fd9197e2c207545133f96e7cd292fc1eb494e8c57384d6ba0a90a83c4f1dd9 -Size (mlr-5.6.2.tar.gz) = 1280257 bytes +SHA1 (mlr-5.7.0.tar.gz) = 50b18bb10bbf8a5e03f58b2ecd7809844834ed43 +RMD160 (mlr-5.7.0.tar.gz) = 3b3454bdd5ff771c8521a07ccd66e5932fc67af8 +SHA512 (mlr-5.7.0.tar.gz) = a801c9e6e10422cd0cc79c117ce7d49268f6d2bbe7c625fed0dd9144fcb7f5d2382a4450772f468f3d31b61912206bd8686f9c78fca2dbd372ee1e003f9cff88 +Size (mlr-5.7.0.tar.gz) = 1266826 bytes