Support comments in mlr -s files

[johnkerl]

Originally posted by @janxkoci in #1154 (comment)

[janxkoci]

Oh, thanks! 😀 I suggest rewriting the title to something like "allow comments in regular miller scripts (loaded with mlr -s)" or similar..

[johnkerl]

Oops I thought I did -- didn't push 'save' or something -- thank you!

[johnkerl]

@janxkoci here's the catch:

https://github.com/johnkerl/miller/blob/6.8.0/internal/pkg/climain/mlrcli_shebang.go#L62-L64

Namely, if the script file is like

#!/usr/bin/env mlr -s
--c2p
filter '${#quantity} != 20'
then count-distinct -f shape
then fraction -f count

-- which is admittedly a corner-case -- then a naïvely coded replace of #.* to end of lines with "" will not do the right thing.

I think the best option (most often correct) will be for mlr -s to by default replace #.* to end of line with "" unconditionally, and with (alas, yet another Miller command-line flag) let people disable that for rarer situations such as the above.

[janxkoci]

Wait, doesn't the parser treat code in quotes differently? Or in the parentheses for names with special characters - I thought special treatment was the whole point of those 😳

[johnkerl]

Sorry @janxkoci I think the auto-close of this issue was from PR #1359 being merged -- re-opening!

[johnkerl]

@janxkoci yes the DSL parser definitely treats code in quotes differently. But the comment-stripping here has to be before the DSL parser is invoked. It's a big of a chicken-and-egg situationj ...

[johnkerl]

Handling these two # differently:

#!/usr/bin/env mlr -s
--c2p
filter '${#quantity} != 20'
then count-distinct -f shape # comment here
then fraction -f count

would (if I understand correctly) require re-implementing a shell parser in Miller ...

[janxkoci]

No worries, and you can even close it, if it's implemented. I mostly don't have such problem with field names anyway, or I can just re-label them. I was just surprised that the parser works like this, that's all.. 🤷