Integer and float arithmetic, improved documentation, minor feature enhancements

Integer/float arithmetic

The key feature of the 3.0.0 release, and the reason for the major version increment, is that previously all numbers were scanned into mlr put and mlr filter functions as floating-point -- then, only recast to integer as necessary for integer operations. Since IEEE doubles have 53 bits of precision (52 mantissa bits along with implicit leading one) while 64-bit integers have 64, this meant that full 64-bit integer signficance could not be passed through Miller functions.

As of the 3.0.0 release, numbers in Miller are int (64 bits) or float (double-precision). Numbers scannable as integers are treated as integers. The sum, difference, and product of two integers is another integer -- except when overflow would occur, at which point a floating-point result is produced. Integer division is pythonic, namely, 7/2 is 3.5, and 7//2 is 3. Mixed integer/float operations produce float. Bitwise operators are now supported.

You now have more control over arithmetic, not less. The only real compatibility change is that some numbers will now be printing like 123 rather than 123.0000.

For full details please see http://johnkerl.org/miller/doc/reference.html#Arithmetic.

New functions for filter and put

• Since integers are now fully supported in mlr put and mlr filter, it is now possible to have the bitwise operators | ^ & << >>. These operate on 64-bit integers and produce 64-bit-integer results.
• Modular arithmetic is implemented by madd, msub, mmul, and mexp.
• urandint and urand32 are in addition to the existing urand.
• sgn complements abs.
• strftime and strptime are generalizations of sec2gmt and gmt2sec. There are pass-throughs to system strftime and strptime; see your local manpages for available time-formatting options.
• Please see http://johnkerl.org/miller/doc/reference.html#Functions_for_filter_and_put for more information.

Verbs

• mlr grep: http://johnkerl.org/miller/doc/reference.html#grep
• mlr cat -n option: http://johnkerl.org/miller/doc/reference.html#cat
• mlr stats1 skewness and mlr stats1 kurtosis: http://johnkerl.org/miller/doc/reference.html#stats1
• mlr bar allows for some simple terminal-level visualization: http://johnkerl.org/miller/doc/reference.html#bar
• mlr join now has full support for heterogeneous data: records lacking all the join keys are treated the same as any other left-unpaired or right-unpaired records. This was tracked on issue #82.

I/O options

• mlr --xvright for XTAB output
• mlr --headerless-csv-output for CSV/CSV-lite output

Documentation

• The mlr.1 manpage is now autogenerated.
• There is now documentation on operator precedence and function semantics.
• HTML pages at http://johnkerl.org/miller/doc/ are now PDF-renderable.
• Per-release documents are available at http://johnkerl.org/miller/doc/release-docs.html. (The documents at http://johnkerl.org/miller/doc/ have always tracked head, and they continue to do so.)