chore: Add more benchmarks

README.md

@@ -107,7 +107,7 @@ q '.' pokemons.json --no-colors
 
 [This report](./benchmarks/report.md) is not an exhaustive performance report of both tools, it's a overview for the percieved performance of the user. Here I don't profile each tool and try to see what are the bootlenecks, since I assume that both tools have the penalty of parsing a JSON file. Simply run a bash script and analyze the results.
 
-Aside from that, **query-json** doesn't have feature parity with **jq** which is ok at this point, but **jq** contains a ton of functionality that query-json misses and some of the **jq** operations aren't native, are builtin with the runtime. In order to do a proper comparision all of this above would need to take into consideration.
+Aside from that, **query-json** doesn't have feature parity with **jq** which is ok at this point, but **jq** contains a ton of functionality that query-json misses. Adding the missing operations on **query-json** won't affect the performance of it, that could not be true for features like "modules" or "tests", which they will not be implemented in **query-json**.
 
 The report shows that **query-json** is between 2x and 5x faster than **jq** in all operations tested and same speed (~1.1x) with huge files (> 100M).
 

benchmarks/hyper.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env sh
+
+hyperfine --style basic --warmup 10 --min-runs 3 --export-markdown benchmarks/boot-time-report.md "query-json '.' esy.json" "jq '.' esy.json" "faq '.' esy.json" "fx esy.json '.'"

benchmarks/report.md

@@ -1,13 +1,25 @@
 # Benchmark Report
 The benchmarks run on a 13" MacBook Pro (2020) with a 1.4 GHz Quad-Core i5 and 16GB 2133MHz RAM.
 
-This benchmark consists in steps: one using [hyperfine](https://github.com/sharkdp/hyperfine) which evaluates the simple identity of each tool, to check boot time performance and the other running a time with a different kind of queries on a set of different json file sizes.
+This benchmark consists twoo different steps, the boot time and a timer on a few operations over different (json) file sizes.
 
-Executing `$ ./benchmarks/run.sh` runs both steps.
+Executing `$ ./benchmarks/hyper.sh` runs hyperfile.
+Executing `$ ./benchmarks/run.sh` runs the timers.
 
 ## Output
 
-`$ ./benchmarks/run.sh`
+`$ ./benchmarks/hyper.sh`
+
+| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
+|:---|---:|---:|---:|---:|
+| `query-json . esy.json` | 7.9 ± 0.4 | 7.3 | 11.3 | 1.00 |
+| `jq . esy.json` | 27.6 ± 0.5 | 26.8 | 29.0 | 3.50 ± 0.21 |
+| `faq . esy.json` | 55.3 ± 1.1 | 53.7 | 60.0 | 7.00 ± 0.42 |
+| `fx esy.json .` | 64.5 ± 2.7 | 62.2 | 72.7 | 8.16 ± 0.58 |
+
+---
+
+## Running [run.sh](./run.sh)
 
 ```bash
 query-json: 0.5.8
@@ -16,62 +28,6 @@ faq: 0.0.6
 fx: 20.0.2
 ```
 
-#### Benchmark #1: jq . esy.json
-```bash
-  Time (mean ± σ):      27.6 ms ±   1.0 ms    [User: 25.4 ms, System: 1.2 ms]
-  Range (min … max):    26.5 ms …  32.1 ms    71 runs
-```
-
-#### Benchmark #1: query-json . esy.json
-```bash
-  Time (mean ± σ):       7.8 ms ±   0.2 ms    [User: 3.4 ms, System: 2.1 ms]
-  Range (min … max):     7.3 ms …   8.9 ms    267 runs
-```
-#### Benchmark #2: jq . esy.json
-```bash
-  Time (mean ± σ):      27.6 ms ±   1.1 ms    [User: 25.3 ms, System: 1.2 ms]
-  Range (min … max):    26.4 ms …  34.5 ms    98 runs
-```
-### **Summary** `query-json . esy.json` ran 3.56 ± 0.18 times faster than `jq . esy.json`
-
-#### Benchmark #1: faq . esy.json
-```bash
-  Time (mean ± σ):      55.1 ms ±   0.6 ms    [User: 54.5 ms, System: 5.1 ms]
-  Range (min … max):    53.9 ms …  57.2 ms    43 runs
-```
-#### Benchmark #1: query-json . esy.json
-```bash
-  Time (mean ± σ):       7.8 ms ±   0.2 ms    [User: 3.4 ms, System: 2.1 ms]
-  Range (min … max):     7.3 ms …   8.9 ms    280 runs
-```
-#### Benchmark #2: faq . esy.json
-```bash
-  Time (mean ± σ):      55.2 ms ±   0.7 ms    [User: 54.8 ms, System: 5.2 ms]
-  Range (min … max):    53.9 ms …  57.2 ms    51 runs
-```
-### **Summary** `query-json . esy.json` ran 7.11 ± 0.23 times faster than `faq . esy.json`
-
-#### Benchmark #1: fx esy.json .
-```bash
-  Time (mean ± σ):      60.5 ms ±   1.8 ms    [User: 48.6 ms, System: 13.3 ms]
-  Range (min … max):    59.1 ms …  70.9 ms    40 runs
-```
-#### Benchmark #1: query-json . esy.json
-```bash
-  Time (mean ± σ):       7.8 ms ±   0.2 ms    [User: 3.4 ms, System: 2.1 ms]
-  Range (min … max):     7.4 ms …   8.8 ms    256 runs
-```
-#### Benchmark #2: fx esy.json .
-```bash
-  Time (mean ± σ):      60.3 ms ±   0.6 ms    [User: 48.5 ms, System: 12.9 ms]
-  Range (min … max):    59.3 ms …  62.1 ms    47 runs
-```
-### **Summary** `query-json . esy.json` ran 7.73 ± 0.24 times faster than `fx esy.json .`
-
----
-
-## Running [run.sh](./run.sh)
-
 ### Select an attribute (`.first.id`) on a small (4kb) JSON file
 ```bash
 query-json        0.00 real         0.00 user         0.00 sys

benchmarks/run.sh

@@ -1,6 +1,4 @@
-#!/usr/bin/env sh
-
-trash="benchmarks/trash.json"
+#!/usr/bin/env bash
 
 title () {
   echo ""
@@ -19,60 +17,50 @@ test () {
   /usr/bin/time "$@"
 }
 
-hyperfine --prepare 'query-json . esy.json' 'jq . esy.json'
-hyperfine 'query-json . esy.json' 'jq . esy.json'
-hyperfine --prepare 'query-json . esy.json' 'faq . esy.json'
-hyperfine 'query-json . esy.json' 'faq . esy.json'
-hyperfine --prepare 'query-json . esy.json' 'fx esy.json .'
-hyperfine 'query-json . esy.json' 'fx esy.json .'
-
-touch $trash;
+title "### Pipe a json to stdin"
+test cat benchmarks/big.json | query-json --kind=inline '.' > /dev/null
+test cat benchmarks/big.json | jq '.' > /dev/null
+test cat benchmarks/big.json | faq '.' > /dev/null
+test cat benchmarks/big.json | fx '.' > /dev/null
+test cat benchmarks/big.json | jet --from json --to json '.' > /dev/null
 
 title "### Select an attribute on a small (4kb) JSON file" ".first.id"
-test query-json '.first.id' benchmarks/small.json >> $trash
-test jq '.first.id' benchmarks/small.json >> $trash
-test faq '.first.id' benchmarks/small.json >> $trash
-test fx benchmarks/small.json '.first.id' >> $trash
+test query-json '.first.id' benchmarks/small.json > /dev/null
+test jq '.first.id' benchmarks/small.json > /dev/null
+test faq '.first.id' benchmarks/small.json > /dev/null
+test fx benchmarks/small.json '.first.id' > /dev/null
 
 title "### Select an attribute on a medium (132k) JSON file" "."
-#
-test query-json '.' benchmarks/medium.json >> $trash
-test jq '.' benchmarks/medium.json >> $trash
-test faq '.' benchmarks/medium.json >> $trash
-test fx benchmarks/medium.json '.' >> $trash
+test query-json '.' benchmarks/medium.json > /dev/null
+test jq '.' benchmarks/medium.json > /dev/null
+test faq '.' benchmarks/medium.json > /dev/null
+test fx benchmarks/medium.json '.' > /dev/null
 
 title "### Select an attribute on a big JSON (604k) file" "map(.)"
-#
-test query-json 'map(.)' benchmarks/big.json >> $trash
-test jq 'map(.)' benchmarks/big.json >> $trash
-test faq 'map(.)' benchmarks/big.json >> $trash
-test fx benchmarks/big.json '.map(x => x)' >> $trash
+test query-json 'map(.)' benchmarks/big.json > /dev/null
+test jq 'map(.)' benchmarks/big.json > /dev/null
+test faq 'map(.)' benchmarks/big.json > /dev/null
+test fx benchmarks/big.json '.map(x => x)' > /dev/null
 
 title "### Simple operation on a small (4kb) JSON file" ".second.store.books | map(.price + 10)"
-#
-test query-json '.second.store.books | map(.price + 10)' benchmarks/small.json >> $trash
-test jq '.second.store.books | map(.price + 10)' benchmarks/small.json >> $trash
-test fx benchmarks/small.json '.second.store.books.map(x => x.price + 10)' >> $trash
+test query-json '.second.store.books | map(.price + 10)' benchmarks/small.json > /dev/null
+test jq '.second.store.books | map(.price + 10)' benchmarks/small.json > /dev/null
+test fx benchmarks/small.json '.second.store.books.map(x => x.price + 10)' > /dev/null
 
 title "### Simple operation on a medium (132k) JSON file" "map(.time)"
-#
-test query-json 'map(.time)' benchmarks/medium.json >> $trash
-test jq 'map(.time)' benchmarks/medium.json >> $trash
-test faq 'map(.time)' benchmarks/medium.json >> $trash
-test fx benchmarks/medium.json '.map(x => x.time)' >> $trash
+test query-json 'map(.time)' benchmarks/medium.json > /dev/null
+test jq 'map(.time)' benchmarks/medium.json > /dev/null
+test faq 'map(.time)' benchmarks/medium.json > /dev/null
+test fx benchmarks/medium.json '.map(x => x.time)' > /dev/null
 
 title "### Simple operation an attribute on a big JSON (604k) file" "map(select(.base.Attack > 100)) | map(.name.english)"
-#
-test query-json 'filter(.base."Attack" > 100) | map(.name.english)' benchmarks/big.json >> $trash
-test jq 'map(select(.base.Attack > 100)) | map(.name.english)' benchmarks/big.json >> $trash
-# test faq 'filter(.base."Attack" > 100) | map(.name.english)' benchmarks/big.json >> $trash
-test fx benchmarks/big.json '.filter(x => x.base["Attack"] > 100).map(x => x.name.english)' >> $trash
+test query-json 'filter(.base."Attack" > 100) | map(.name.english)' benchmarks/big.json > /dev/null
+test jq 'map(select(.base.Attack > 100)) | map(.name.english)' benchmarks/big..json > /dev/null
+# test faq 'filter(.base."Attack" > 100) | map(.name.english)' benchmarks/big.json > /dev/null
+test fx benchmarks/big.json '.filter(x => x.base["Attack"] > 100).map(x => x.name.english)' > /dev/null
 
 title "### Simple operation an attribute on a huge JSON (110M) file" "keys"
-#
-test query-json 'keys' benchmarks/huge.json >> $trash
-test jq 'keys' benchmarks/huge.json >> $trash
-test faq 'keys' benchmarks/huge.json >> $trash
-test fx benchmarks/huge.json 'Object.keys' >> $trash
-
-rm $trash
+test query-json 'keys' benchmarks/huge.json > /dev/null
+test jq 'keys' benchmarks/huge.json > /dev/null
+test faq 'keys' benchmarks/huge.json > /dev/null
+test fx benchmarks/huge.json 'Object.keys' > /dev/null