jt.1

.\" generated with Ronn/v0.7.3
.\" http://github.com/rtomayko/ronn/tree/0.7.3
.
.TH "JT" "1" "July 2017" "" "JT MANUAL"
.
.SH "NAME"
\fBjt\fR \- transform JSON data to tabular format
.
.SH "SYNOPSIS"
\fBjt\fR \fB[\-hV]\fR
.
.br
\fBjt\fR \fB\-u\fR \fIstring\fR
.
.br
\fBjt\fR [\fB\-acj\fR] [\fBCOMMAND\fR \.\.\.]
.
.SH "DESCRIPTION"
\fBJt\fR reads UTF\-8 encoded JSON forms from \fIstdin\fR and writes tab separated values (or CSV) to \fIstdout\fR\. A simple stack\-based programming language is used to extract values from the JSON input for printing\.
.
.SH "OPTIONS"
.
.TP
\fB\-h\fR
Print usage info and exit\.
.
.TP
\fB\-V\fR
Print version and license info and exit\.
.
.TP
\fB\-a\fR
Explicit iteration mode: require the \fB\.\fR command to iterate over arrays instead of iterating automatically\.
.
.TP
\fB\-c\fR
CSV output mode: write RFC 4180 compliant CSV records\.
.
.TP
\fB\-j\fR
Inner join mode: discard rows with missing columns\.
.
.TP
\fB\-s\fR
A no\-op, included for compatibility with earlier versions\.
.
.TP
\fB\-u\fR \fIstring\fR
Unescape JSON \fIstring\fR, print it, and exit\. Double\-quotes around \fIstring\fR are optional\.
.
.SH "OPERATION"
Non\-option arguments are words (commands) in a stack\-based programming language\. Commands operate on the stacks provided by the \fBjt\fR runtime:
.
.IP "\(bu" 4
The \fBDATA\fR stack contains JSON values commands will operate on\.
.
.IP "\(bu" 4
The \fBGOSUB\fR stack contains pointers into the data stack\.
.
.IP "\(bu" 4
The \fBINDEX\fR stack contains pointers to iterators during iteration\.
.
.IP "\(bu" 4
The \fBLOOP\fR stack contains pointers to iterators between iterations\.
.
.IP "\(bu" 4
The \fBOUTPUT\fR stack contains values to print after each iteration\.
.
.IP "" 0
.
.P
When \fBjt\fR starts, the following happens:
.
.IP "1." 4
One JSON form is read from \fIstdin\fR, parsed, and pushed onto the data stack\.
.
.IP "2." 4
If the top of the data stack is a JSON array the next item in the array is pushed onto the data stack based on state previously saved on the loop stack\. The loop stack is then popped and pushed onto the index stack\.
.
.IP "3." 4
The next command is executed\.
.
.IP "4." 4
Steps 2 and 3 are repeated until no commands are left to execute\.
.
.IP "5." 4
Values in the output stack are printed, separated by tabs and followed by a newline (see \fBOUTPUT FORMAT\fR below)\. Data, output, and gosub stacks are reset to their states as they were after step 1\.
.
.IP "6." 4
Loop variables are popped off of the index stack\. They are then incremented and pushed onto the loop stack as necessary\.
.
.IP "7." 4
Steps 2 through 6 are repeated until the loop stack is empty (i\.e\. there are no more iterations left)\.
.
.IP "" 0
.
.P
This process is repeated until there is no more JSON to read\.
.
.SH "COMMANDS"
\fBJt\fR provides the following commands:
.
.TP
\fB[\fR
Save the state of the data stack: the current data stack pointer is pushed onto the gosub stack\.
.
.TP
\fB]\fR
Restore the data stack to a saved state: pop the gosub stack and restore the data stack pointer to that state\.
.
.TP
\fB%\fR
Push the value at the top of the data stack onto the output stack\.
.
.TP
\fB%=\fR\fINAME\fR
Like \fB%\fR, but also sets the heading for this column to \fINAME\fR\. Headers will be printed as the first line of output when this command has been used\.
.
.TP
\fB^\fR
Push the value at the top of the index stack onto the output stack\. Note that the index stack will always have at least one item: the index of the last JSON object read from stdin\.
.
.TP
\fB^=\fR\fINAME\fR
Like \fB^\fR, but also sets the heading for this column to \fINAME\fR\. Headers will be printed as the first line of output when this command has been used\.
.
.TP
\fB@\fR
Print the keys of the object at the top of the data stack and exit\.
.
.IP
If the value at the top of the data stack is an array the first item in the array is pushed onto the data stack and the command is reevaluated\.
.
.IP
If the value at the top of the data stack is not an object or an array then its type is printed in brackets, eg\. \fI[array]\fR, \fI[string]\fR, \fI[number]\fR, \fI[boolean]\fR, or \fI[null]\fR\. If there is no value then \fI[none]\fR is printed\.
.
.TP
\fB+\fR
Parse embedded JSON: if the item at the top of the data stack is a string, parse it and push the result onto the data stack\.
.
.IP
If the item at the top of the data stack is not a string or if there is an error parsing the JSON then nothing is done (the operation is a no\-op)\.
.
.TP
\fB\.\fR
Iterate over the values of the object at the top of the data stack\. The current value will be pushed onto the data stack and the current key will be pushed onto the index stack\.
.
.TP
\fB[\fR\fIKEY\fR\fB]\fR
Drill down: get the value of the \fIKEY\fR property of the object at the top of the data stack and push that value onto the data stack\.
.
.IP
If the item at the top of the data stack is not an object or if the object has no \fIKEY\fR property a blank field is printed, unless the \fB\-j\fR option was specified in which case the entire row is removed from the output\.
.
.IP
If the \fIKEY\fR property of the object is an array subsequent commands will operate on one of the items in the array, chosen automatically by \fBjt\fR\. The array index will be available to subsequent commands via the index stack\.
.
.TP
\fIKEY\fR
See \fB[\fR\fIKEY\fR\fB]\fR above \(em the \fB[\fR and \fB]\fR may be omitted if the property name \fIKEY\fR does not conflict with any \fBjt\fR command\.
.
.SH "OUTPUT FORMAT"
The format of printed output from \fBjt\fR (see \fB%\fR in \fBCOMMANDS\fR, above) depends on the type of thing being printed:
.
.SS "Primitives"
JSON primitives (i\.e\. \fBnull\fR, \fBtrue\fR, and \fBfalse\fR) and numbers are printed verbatim\. \fBJt\fR does not process them in any way\. Numbers can be of arbitrary precision, as long as they conform to the JSON parsing grammar\.
.
.P
If special formatting is required the \fBprintf\fR program is your friend:
.
.IP "" 4
.
.nf

$ printf %\.0f 2\.99792458e9
2997924580
.
.fi
.
.IP "" 0
.
.SS "Strings"
Strings are printed verbatim, minus the enclosing double quotes\. No unescaping is performed because tabs or newlines in JSON strings would break the tabular output format\.
.
.P
If unescaped values are desired either use the \fB\-c\fR flag to enable CSV output (see \fBCSV\fR below) or use the \fB\-u\fR option and pass the string as an argument:
.
.IP "" 4
.
.nf

$ jt \-u \'i love music \eu266A\'
i love music ♪
.
.fi
.
.IP "" 0
.
.SS "Collections"
Objects and arrays are printed as JSON with whitespace removed\. Note that it is only possible to print arrays when the \fB\-a\fR option is specified (see the \fBIteration (Arrays)\fR and \fBExplicit Iteration\fR sections below)\.
.
.SS "CSV"
The \fB\-c\fR flag enables CSV output conforming to RFC 4180\. This format supports strings containing tabs, newlines, etc\.:
.
.IP "" 4
.
.nf

$ jt \-c [ foo % ] [ bar % ] baz % <<EOT
\- {
\-   "foo": "one\ettwo",
\-   "bar": "three\enfour",
\-   "baz": "music \e"\eu266A\e""
\- }
\- EOT
"one    two","three
four","music ""♪"""
.
.fi
.
.IP "" 0
.
.SH "EXIT STATUS"
\fBJt\fR will exit with a status of 1 if an error occurred, or 0 otherwise\.
.
.SH "EXAMPLES"
Below are a number of examples demonstrating how to use \fBjt\fR commands to do some simple exploration and extraction of data from JSON and JSON streams\.
.
.SS "Explore"
The \fB@\fR command prints information about the item at the top of the data stack\. When the item is an object \fB@\fR prints its keys:
.
.IP "" 4
.
.nf

$ jt @ <<EOT
\- {
\-   "foo": 100,
\-   "bar": 200,
\-   "baz": 300
\- }
\- EOT
foo
bar
baz
.
.fi
.
.IP "" 0
.
.P
When the top item is an array \fB@\fR prints information about the first item in the array:
.
.IP "" 4
.
.nf

$ jt @ <<EOT
\- [
\-   {"foo": 100, "bar": 200},
\-   {"baz": 300, "baf": 400}
\- ]
\- EOT
foo
bar
.
.fi
.
.IP "" 0
.
.P
Otherwise, \fB@\fR prints the type of the item, or \fI[none]\fR if there is no value:
.
.IP "" 4
.
.nf

$ echo \'["hello", "world"]\' | jt @
[string]
.
.fi
.
.IP "" 0
.
.IP "" 4
.
.nf

$ echo \'[1, 2]\' | jt @
[number]
.
.fi
.
.IP "" 0
.
.IP "" 4
.
.nf

$ echo \'[true, false]\' | jt @
[boolean]
.
.fi
.
.IP "" 0
.
.IP "" 4
.
.nf

$ echo \'[null, null]\' | jt @
[null]
.
.fi
.
.IP "" 0
.
.IP "" 4
.
.nf

$ echo \'[]\' | jt @
[none]
.
.fi
.
.IP "" 0
.
.SS "Drill Down"
Property names are also commands\. Use \fBfoo\fR here as a command to drill down into the \fIfoo\fR property and then use \fB@\fR to print its keys:
.
.IP "" 4
.
.nf

$ jt foo @ <<EOT
\- {
\-   "foo": {
\-     "bar": 100,
\-     "baz": 200
\-   }
\- }
\- EOT
bar
baz
.
.fi
.
.IP "" 0
.
.P
A property name that conflicts with a \fBjt\fR command may be wrapped with square brackets to drill down:
.
.IP "" 4
.
.nf

$ jt [@] @ <<EOT
\- {
\-   "@": {
\-     "bar": 100,
\-     "baz": 200
\-   }
\- }
\- EOT
bar
baz
.
.fi
.
.IP "" 0
.
.P
\fBNote:\fR The property name must be a JSON escaped string, so if the property name in the JSON input contains eg\. tab characters they must be escaped in the command, like this:
.
.IP "" 4
.
.nf

$ jt \'foo\etbar\' @ <<EOT
\- {
\-   "foo\etbar": {
\-     "bar": 100,
\-     "baz": 200
\-   }
\- }
\- EOT
bar
baz
.
.fi
.
.IP "" 0
.
.SS "Extract"
The \fB%\fR command prints the item at the top of the data stack\. Note that when the top item is a collection it is printed as JSON (insiginificant whitespace removed):
.
.IP "" 4
.
.nf

$ jt % <<EOT
\- {
\-   "foo": 100,
\-   "bar": 200
\- }
\- EOT
{"foo":100,"bar":200}
.
.fi
.
.IP "" 0
.
.P
Drill down and print:
.
.IP "" 4
.
.nf

$ jt foo bar % <<EOT
\- {
\-   "foo": {
\-     "bar": 100
\-   }
\- }
\- EOT
100
.
.fi
.
.IP "" 0
.
.P
The \fB%\fR command can be used multiple times\. The printed values will be separated by tabs:
.
.IP "" 4
.
.nf

$ jt % foo % bar % <<EOT
\- {
\-   "foo": {
\-     "bar": 100
\-   }
\- }
\- EOT
{"foo":{"bar":100}}     {"bar":100}     100
.
.fi
.
.IP "" 0
.
.SS "Save / Restore"
The \fB[\fR and \fB]\fR commands provide a way to save the data stack\'s state before performing some operations and restore it afterwards\. This makes it possible to drill down into one part of an object and then "rewind" to drill down into another part of the same object\.
.
.P
The \fB[\fR command saves the state of the data stack and the \fB]\fR command restores the data stack to the corresponding saved state\. It is an error to use the \fB]\fR command without a corresponding \fB[\fR command\.
.
.IP "" 4
.
.nf

$ jt [ foo % ] bar % <<EOT
\- {
\-   "foo": 100,
\-   "bar": 200
\- }
\- EOT
100     200
.
.fi
.
.IP "" 0
.
.P
The \fB[\fR and \fB]\fR commands can be nested:
.
.IP "" 4
.
.nf

$ jt [ foo [ bar % ] [ baz % ] ] baf % <<EOT
\- {
\-   "foo": {
\-     "bar": 100,
\-     "baz": 200
\-   },
\-   "baf": "quux"
\- }
\- EOT
100     200     quux
.
.fi
.
.IP "" 0
.
.SS "Iteration (Arrays)"
\fBJt\fR automatically iterates over arrays (unless this behavior is disabled \(em see \fBExplicit Iteration\fR below), producing one tab\-delimited record per iteration, records separated by newlines:
.
.IP "" 4
.
.nf

$ jt [ foo % ] bar baz % <<EOT
\- {
\-   "foo": 100,
\-   "bar": [
\-     {"baz": 200},
\-     {"baz": 300},
\-     {"baz": 400}
\-   ]
\- }
\- EOT
100     200
100     300
100     400
.
.fi
.
.IP "" 0
.
.P
The \fB^\fR command includes the array index as a column in the result:
.
.IP "" 4
.
.nf

$ jt [ foo % ] bar ^ baz % <<EOT
\- {
\-   "foo": 100,
\-   "bar": [
\-     {"baz": 200},
\-     {"baz": 300},
\-     {"baz": 400}
\-   ]
\- }
\- EOT
100     0       200
100     1       300
100     2       400
.
.fi
.
.IP "" 0
.
.P
The \fB^\fR command is scoped to the innermost enclosing loop:
.
.IP "" 4
.
.nf

$ jt foo ^ bar ^ % <<EOT
\- {
\-   "foo": [
\-     {"bar": [100, 200]},
\-     {"bar": [300, 400]}
\-   ]
\- }
\- EOT
0       0       100
0       1       200
1       0       300
1       1       400
.
.fi
.
.IP "" 0
.
.SS "Iteration (Objects)"
The \fB\.\fR command iterates over the values of an object:
.
.IP "" 4
.
.nf

$ jt \. % <<EOT
\- {
\-   "foo": 100,
\-   "bar": 200,
\-   "baz": 300
\- }
\- EOT
100
200
300
.
.fi
.
.IP "" 0
.
.P
When iterating over an object the \fB^\fR command prints the name of the current property:
.
.IP "" 4
.
.nf

$ jt \. ^ % <<EOT
\- {
\-   "foo": 100,
\-   "bar": 200,
\-   "baz": 300
\- }
\- EOT
foo     100
bar     200
baz     300
.
.fi
.
.IP "" 0
.
.SS "JSON Streams"
\fBJt\fR automatically iterates over entities in a JSON stream (optionally delimited by whitespace):
.
.IP "" 4
.
.nf

$ jt [ foo % ] bar % <<EOT
\- {"foo": 100, "bar": 200}
\- {"foo": 200, "bar": 300}
\- {"foo": 300, "bar": 400}
\- EOT
100     200
200     300
300     400
.
.fi
.
.IP "" 0
.
.P
The \fB^\fR command prints the current stream index:
.
.IP "" 4
.
.nf

$ jt ^ [ foo % ] bar % <<EOT
\- {"foo": 100, "bar": 200}
\- {"foo": 200, "bar": 300}
\- {"foo": 300, "bar": 400}
\- EOT
0       100     200
1       200     300
2       300     400
.
.fi
.
.IP "" 0
.
.SS "Nested JSON"
The \fB+\fR command parses JSON embedded in strings:
.
.IP "" 4
.
.nf

$ jt [ foo + bar % ] baz % <<EOT
\- {"foo":"{\e"bar\e":100}","baz":200}
\- {"foo":"{\e"bar\e":200}","baz":300}
\- {"foo":"{\e"bar\e":300}","baz":400}
\- EOT
100     200
200     300
300     400
.
.fi
.
.IP "" 0
.
.P
\fBNote:\fR The \fB+\fR command does not modify the original JSON:
.
.IP "" 4
.
.nf

$ jt [ foo + bar % ] % <<EOT
\- {"foo":"{\e"bar\e":100}","baz":200}
\- {"foo":"{\e"bar\e":200}","baz":300}
\- {"foo":"{\e"bar\e":300}","baz":400}
\- EOT
100     {"foo":"{\e"bar\e":100}","baz":200}
200     {"foo":"{\e"bar\e":200}","baz":300}
300     {"foo":"{\e"bar\e":300}","baz":400}
.
.fi
.
.IP "" 0
.
.SS "Column Headings"
The \fB%=\fR\fINAME\fR and \fB^=\fR\fINAME\fR commands are like \fB%\fR and \fB^\fR above, but they also assign column headings, which are printed as the first line of output:
.
.IP "" 4
.
.nf

$ jt [ foo %=Foo ] bar ^=Bar baz %=Baz <<EOT
\- {
\-   "foo": 100,
\-   "bar": [
\-     {"baz": 200},
\-     {"baz": 300},
\-     {"baz": 400}
\-   ]
\- }
\- EOT
Foo     Bar     Baz
100     0       200
100     1       300
100     2       400
.
.fi
.
.IP "" 0
.
.P
\fBNote:\fR The heading \fINAME\fR must be a JSON escaped string (eg\. to have a column heading with embedded tab characters use \fB%=foo\etbar\fR instead of \fB"%=foo bar"\fR)\.
.
.SS "Joins"
Notice the empty column \(em some objects don\'t have the \fIbar\fR key:
.
.IP "" 4
.
.nf

$ jt [ foo % ] bar % <<EOT
\- {"foo":100,"bar":1000}
\- {"foo":200}
\- {"foo":300,"bar":3000}
\- EOT
100     1000
200
300     3000
.
.fi
.
.IP "" 0
.
.P
Enable inner join mode with the \fB\-j\fR flag\. This removes output rows when a key in the traversal path doesn\'t exist:
.
.IP "" 4
.
.nf

$ jt \-j [ foo % ] bar % <<EOT
\- {"foo":100,"bar":1000}
\- {"foo":200}
\- {"foo":300,"bar":3000}
\- EOT
100     1000
300     3000
.
.fi
.
.IP "" 0
.
.P
Note that this does not remove rows when the key exists and the value is empty:
.
.IP "" 4
.
.nf

$ jt \-j [ foo % ] bar % <<EOT
\- {"foo":100,"bar":1000}
\- {"foo":200,"bar":""}
\- {"foo":300,"bar":3000}
\- EOT
100     1000
200
300     3000
.
.fi
.
.IP "" 0
.
.SS "Explicit Iteration"
Sometimes the implicit iteration over arrays is awkward:
.
.IP "" 4
.
.nf

$ jt \. ^ \. ^ % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
0       bar     100
1       bar     200
.
.fi
.
.IP "" 0
.
.P
Should the first \fB^\fR be printing the array index of the implicit iteration (which it does, in this case) or the object key (i\.e\. \fIfoo\fR) of the explicit iteration of the \fB\.\fR command?
.
.P
Another awkward case is printing arrays:
.
.IP "" 4
.
.nf

$ jt foo % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
{"bar":100}
{"bar":200}
.
.fi
.
.IP "" 0
.
.P
The array can not be printed with the \fB%\fR command because it is being iterated over implicitly\. Instead, the items in the array are printed, which may not be the desired behavior\.
.
.P
The \fB\-a\fR flag eliminates the ambiguity by enabling explicit iteration\. In this mode the \fB\.\fR command must be used to iterate over both objects and arrays \(em arrays are not automatically iterated over\.
.
.P
Now the array can be printed:
.
.IP "" 4
.
.nf

$ jt \-a foo % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
[{"bar":100},{"bar":200}]
.
.fi
.
.IP "" 0
.
.P
Or the first \fB^\fR can print the array index, as before:
.
.IP "" 4
.
.nf

$ jt \-a \. \. ^ \. ^ % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
0       bar     100
1       bar     200
.
.fi
.
.IP "" 0
.
.P
Or it can print the object key:
.
.IP "" 4
.
.nf

$ jt \-a \. ^ \. \. ^ % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
foo     bar     100
foo     bar     200
.
.fi
.
.IP "" 0
.
.P
Or, with the addition of one more \fB^\fR command, it can print both:
.
.IP "" 4
.
.nf

$ jt \-a \. ^ \. ^ \. ^ % <<EOT
\- {
\-   "foo": [
\-     {"bar":100},
\-     {"bar":200}
\-   ]
\- }
\- EOT
foo     0       bar     100
foo     1       bar     200
.
.fi
.
.IP "" 0
.
.SH "SEE ALSO"
\fBJt\fR is based on ideas from the excellent \fBjshon\fR tool, which can be found at \fIhttp://kmkeen\.com/jshon/\fR\. Source code for \fBjt\fR is available at \fIhttps://github\.com/micha/json\-table\fR\.
.
.SH "BUGS"
Please open an issue here: \fIhttps://github\.com/micha/json\-table/issues\fR\.
.
.SH "COPYRIGHT"
Copyright © 2017 Micha Niskin \fB<micha\.niskin@gmail\.com>\fR, distributed under the Eclipse Public License, version 1\.0\. This is free software: you are free to change and redistribute it\. There is NO WARRANTY, to the extent permitted by law\.