feature request: apply global --ofmt to floating-pointable fields on output

[aborruso]

Hi,
with this kind of input

Product,Base_Price,Promotion_Price,Discount
Shampoo,1.999999,1.409999,15.9988999
Biscuit,2.999999,2.409999,15.9988999

Miller knows that Base_Price, Promotion_Price, and Discount are floating. In example if you use mlr to create a JSON you have:

{ "Product": "Shampoo", "Base_Price": 1.999999, "Promotion_Price": 1.409999, "Discount": 15.9988999 }
{ "Product": "Biscuit", "Base_Price": 2.999999, "Promotion_Price": 2.409999, "Discount": 15.9988999 }

My feature request is to add --ofmt option also in cat

mlr --csv --ofmt %.2f cat input.csv

to have

Product,Base_Price,Promotion_Price,Discount
Shampoo,2.00,1.41,16.00
Biscuit,3.00,2.41,16.00

[johnkerl]

OK. So the idea is to leave the input as-is. Another issue is that JSON (unlike all the other formats) has quotes around strings else not, which is something that other formats don't have. So Miller with JSON output tries to test if a given value is a number, and so print the value as-is.

So this isn't about mlr cat so much as output-formatting for JSON numbers ...

There are already an awful lot of JSON-specific --j... flags and we can make a --jofmt or some such which says, apply the global ofmt for printing ... .

This seems inelegant but I'm not sure of a more elegant solution honestly.

[johnkerl]

This is in file-formats.html#Formatting_JSON_options (see also head docs http://johnkerl.org/miller-releases/miller-head/doc/file-formats.html#Formatting_JSON_options) and will go out in 5.6.0.

[aborruso]

@johnkerl thank you. But probably I must explain myself better.

If I have this CSV

Product,Base_Price,Promotion_Price,Discount
Shampoo,1.999999,1.409999,15.9988999
Biscuit,2.999999,2.409999,15.9988999

and I want to have all numbers as float number with 2 decimals

Product,Base_Price,Promotion_Price,Discount
Shampoo,1.99,1.40,15.99
Biscuit,2.99,2.40,15.99

I can use DSL and fmtnum for the fields mapped as numeric.

But looking at the JSON output, mlr knows that Base_Price, Promotion_Price, Discount are numeric.

{ "Product": "Shampoo", "Base_Price": 1.999999, "Promotion_Price": 1.409999, "Discount": 15.9988999 }
{ "Product": "Biscuit", "Base_Price": 2.999999, "Promotion_Price": 2.409999, "Discount": 15.9988999 }

My feature request is to have something like mlr --csv --ofmt %.2f cat input.csv, as a global way to apply number formatting to all fields mapped by miller as numeric.

[johnkerl]

Ah OK.

[johnkerl]

Easiest I think will be a new verb, mlr float-format perhaps. It can loop through all the values & for any that are float-able (using the same logic as the JSON writer) format them with the specified number of digits.

[johnkerl]

And I'll revert the --jofmt business. :)

[aborruso]

Easiest I think will be a new verb, mlr float-format perhaps. It can loop through all the values & for any that are float-able (using the same logic as the JSON writer) format them with the specified number of digits.

What about mlr number-format, to apply also something like fmtnum($n, "%03d") and pass from 1 to 001?

[johnkerl]

OK. So there are strings, ints, and floats which can be auto-detected. mlr format could take -s, -i, and -f flags, defaulting to %s, %d, and %lf respectively, but each one overrideable.

[johnkerl]

http://johnkerl.org/miller-releases/miller-head/doc/reference-verbs.html#format-values

will be in 5.6.0!

[aborruso]

Wow I have no more words!!

Thank you very much

[johnkerl]

:)

[sjackman]

I just discovered format-values! Thank you!