-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
135 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,132 @@ | ||
| k6 v0.26.0 is here! :tada: | ||
|
|
||
| This release contains mostly bug fixes, though it also has several new features and enhancements! They include a new JS compatibility mode option, exporting the end-of-test summary to a JSON report file, speedups to the InfluxDB and JSON outputs, `http.batch()` improvements, a brand new CSV output, multiple layered HTTP response body decompression, being able to use `console` in the init context, a new optional column in the summary, and Docker improvements! | ||
|
|
||
|
|
||
| Thanks to @Sirozha1337, @openmohan, @MMartyn, @KajdeMunter, @dmitrytokarev and @dimatock for contributing to this release! | ||
|
|
||
| ## New features and enhancements! | ||
|
|
||
| ### A new JavaScript compatibility mode option (#1206) | ||
|
|
||
| This adds a way to disable the automatic script transformation by [Babel](https://babeljs.io/) (v6.4.2) and loading of [core-js (v2)](https://github.com/zloirock/core-js) polyfills, bundled in k6. With the new `base` compatibility mode, k6 will instead rely only on the [goja](https://github.com/dop251/goja/) runtime and what is built into k6. | ||
| This can be configured through the new `--compatibility-mode` CLI flag and the `K6_COMPATIBILITY_MODE` environment variable. The possible values currently are: | ||
|
|
||
| - `extended`: this is the default and current compatibility mode, which uses Babel and core.js to achieve ES6+ compatibility. | ||
| - `base`: an optional mode that disables loading of Babel and core.js, running scripts with only goja's native ES5.1+ compatibility. If the test scripts don't require ES6 compatibility (e.g. they were previously transformed by Babel), this option can be used to reduce RAM usage during test runs. | ||
|
|
||
| More info what this means can be found in the [documentation](https://docs.k6.io/docs/javascript-compatibility-mode). | ||
|
|
||
| Our benchmarks show a considerable drop in memory usage - around 80% for simple scripts, and around 50% in the case of 2MB script with a lot of static data in it. The CPU usage is mostly unchanged, except that k6 initializes test runs a lot faster. All of those benefits will be most noticeable if k6 is used with big number of VUs (1k+). More performance comparisons can be found in #1167. | ||
|
|
||
| ### JSON export of the end-of-test summary report (#1168) | ||
|
|
||
| This returns (from the very early days of k6) the ability to output the data from the end of test summary in a machine-readable JSON file. | ||
| This report can be enabled by the `--summary-export <file_path>` CLI flag or the `K6_SUMMARY_EXPORT` environment variable. The resulting JSON file will include data for all test metrics, checks and thresholds. | ||
|
|
||
| ### New CSV output (#1067) | ||
|
|
||
| There is an entirely new `csv` output that can be enabled by using the `--out csv` CLI flag. There are two things that can be configured: the output file with `K6_CSV_FILENAME` (by default it's `file.csv`), and the interval of pushing metrics to disk, which is configured with `K6_CSV_SAVE_INTERVAL` (1 second by default). Both of those can be configured by the CLI as well: `--out csv=somefile.csv` will output to `somefile.csv` and `--out file_name=somefile.csv,save_interval=2s` will output again to `somefile.csv`, but will flush the data every 2 seconds instead of every second. | ||
|
|
||
| The first line of the output is the names of columns and looks like: | ||
| ``` | ||
| metric_name,timestamp,metric_value,check,error,error_code,group,method,name,proto,status,subproto,tls_version,url,extra_tags | ||
| http_reqs,1573131887,1.000000,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/, | ||
| http_req_duration,1573131887,116.774321,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/, | ||
| http_req_blocked,1573131887,148.691247,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/, | ||
| http_req_connecting,1573131887,112.593448,,,,,GET,http://httpbin.org/,HTTP/1.1,200,,,http://httpbin.org/, | ||
| ``` | ||
|
|
||
| All thanks to @Sirozha1337! | ||
|
|
||
|
|
||
| ### JSON output optimizations (#1114) | ||
|
|
||
| The JSON output no longer blocks the goroutine sending samples to the file, but instead (like all other outputs) buffers the samples and writes them at regular intervals (100ms and is currently not configurable). It also uses a slightly faster way of encoding the data, which should decrease the memory usage by a small amount. | ||
|
|
||
| Another improvement is the ability to compress the generated JSON file by simply adding `.gz` to the end of the file name. Compressed files are typically 30x smaller. | ||
|
|
||
| ### InfluxDB output improvements (#1113) | ||
|
|
||
| The InfluxDB output has been updated to use less memory and try to send smaller and consistent chunks of data to InfluxDB, in order to not drop packets and be more efficient. This is primarily done by sending data in parallel, as this seems to be better from a performance perspective, and more importantly, queuing data in separate packets, so that we don't send the data for a big time period all at once. Also, the used library was updated, which also decreased the memory usage. | ||
|
|
||
| Two new options were added: | ||
| - `K6_INFLUXDB_PUSH_INTERVAL` - configures at what interval the collected data is queued to be sent to InfluxDB. By default this is "1s". | ||
| - `K6_INFLUXDB_CONCURRENT_WRITES` - configures the number of concurrent write calls to InfluxDB. If this limit is reached the next writes will be queued and made when a slot is freed. By default this is 10. | ||
|
|
||
| ### `console` is now available in the init context (#982): | ||
|
|
||
| This wasn't supported for the longest time, which made debugging things outside of VU code much harder, but now it's here! :tada: | ||
|
|
||
| In order to get this feature shipped in a timely manner, it currently has a known bug. The output of `console` calls in the init context will always be written to the `stderr`, even if the `--console-output` option is specified. This bug is tracked in https://github.com/loadimpact/k6/issues/1131 | ||
|
|
||
| ### HTTP response body decompression with multiple layered algorithms (#1125) | ||
|
|
||
| In v0.25.0 compressing bodies was added and it had support for multiple layered algorithms. Now this is also true for decompressing bodies when k6 gets them as responses. | ||
|
|
||
|
|
||
| ### New optional `count` column in the end-of-test summary (#1143) | ||
|
|
||
| The `--summary-trend-stats` now also recognizes `count` as a valid column and will output the count of samples in all `Trend` metrics. This could be especially useful for custom `Trend` metrics, since with them you no longer need to specify a separate accompanying `Counter` metric. | ||
|
|
||
| ### Docker Compose refactor (#1183) | ||
|
|
||
| The example docker-compose that enabled easy running of InfluxDB+Grafana+k6 was refactored and all the images were updated to use the latest stable versions. | ||
|
|
||
| Thanks, @KajdeMunter! | ||
|
|
||
| Also the k6 `Dockerfile` Alpine version was bumped to 3.10. Thanks @dmitrytokarev! | ||
|
|
||
| ### `http.batch()` improvements and optimizations (#1259) | ||
|
|
||
| We made several small improvements to the mechanism for executing multiple HTTP requests simultaneously from a single VU: | ||
| - Calling `http.batch()` should now be more efficient, especially for many requests, because of reduced locking, type conversions, and goroutine spawning. | ||
| - The default value for `batchPerHost` has been reduced from `0` (unlimited) to `6`, to more closely match browser behavior. The default value for the `batch` option remains unchanged at `20`. | ||
| - Calling `http.batch(arg)`, where `arg` is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in [#767](https://github.com/loadimpact/k6/issues/767)... Now `http.batch()` will return an array when you pass it an array, and return an object when you pass an object. | ||
|
|
||
| ## UX | ||
|
|
||
| * Better timeout messages for `setup` and `teardown` timeouts, including hints on how to fix them. (#1173) | ||
| * When a folder is passed to `open()`, the resulting error message will now include the path to the specified folder. (#1238) | ||
| * The `k6 version` output will now include more information - the git commit it was built from (in most cases), as well as the used Go version and architecture. (#1235) | ||
|
|
||
| ## Bugs fixed! | ||
| * Cloud: Stop sending metrics to the cloud output when the cloud returns that you have reached the limit. (#1130) | ||
| * JS: Fail a `check` if an uncaught error is thrown inside of it. (#1137) | ||
| * HTTP: Replace any user credentials in the metric sample tags with `*` when emitting HTTP metrics. (#1132) | ||
| * WS: Many fixes: | ||
| - return an error instead of panicking if an error occurs during the making of the WebSocket connection (#1127) | ||
| - calling the `error` handler on an error when closing the WebSocket, instead of calling with a null (#1118) | ||
| - correctly handle server initiated close (#1186) | ||
| * JSON: Better error messages when parsing JSON fails. Now telling you at which line and row the error is instead of just the offset. Thanks, @openmohan! (#905) | ||
| * HTTP: Use Request's `GetBody` in order to be able to get the body multiple times for a single request as needed in 308 redirects of posts and if the server sends GOAWAY with no error. (#1093) | ||
| * JS: Don't export internal go struct fields of script options.(#1151) | ||
| * JS: Ignore `minIterationDuration` for `setup` and `teardown`. (#1175) | ||
| * HTTP: Return error on any request that returns 101 status code as k6 currently doesn't support any protocol upgrade behaviour. (#1172) | ||
| * HTTP: Correctly capture TCP reset by peer and broken pipe errors and give them the appropriate `error_code` metric tag values. (#1164) | ||
| * Config: Don't interpret non-`K6_` prefixed environment variables as k6 configuration, most notably `DURATION` and `ITERATIONS`. (#1215) | ||
| * JS/html: `Selection.map` was not wrapping the nodes it was outputting, which lead to wrongly using the internal `Goquery.Selection` instead of k6's `Selection`. Thanks to @MMartyn for reporting this! (#1198) | ||
| * HTTP: When there are redirects, k6 will now correctly set the cookie for the current URL, instead of for the one the current response is redirecting to. Thanks @dimatock! (#1201) | ||
| * Cloud: Add token to make calls to the cloud API idempotent. (#1208) | ||
| * Cloud: Improve aggregation of HTTP metrics for requests to different URLs, but with the same explicitly set `name` tag. (#1220) | ||
| * Cloud: Fix a bug where you weren't able to run a script, outputting to cloud, if it was using the shortcut urls for github/cdnjs. (#1237) | ||
| * Config: The previous default value for `batchPerHost` of `20` wasn't propagated properly and was instead `0`. (#1264) | ||
|
|
||
|
|
||
| ## Internals | ||
|
|
||
| * CI: Stop using external service for testing WebSockets (#1138) and remove the last use of the external `httpbin.org`. (#1213) | ||
| * Switched to Go 1.13.5 for building and testing k6, removed official support for 1.11. | ||
| * CI: Fix a test on MacOS. (#1142) | ||
| * CI: Fixing flaky tests. (#1149, #1223) | ||
| * Drop an external dependency for getting user's `configdir`. (#1162) | ||
| * Use bitmask for checking whether system tags are enabled, adding some small speedup where this is required. (#1148) | ||
| * Update `envconfig` as it was very old and the newer versions had fixes and features we want. (#1214) | ||
| * Metrics: Emit iterations as part of `netext.NetTrail`, instead of as a standalone one. Also cutting down on amount of bytes we sent to the cloud output. (#1203) | ||
| * JS: goja has been updated to the latest `master` version (commit [`007eef3`](https://github.com/dop251/goja/commit/007eef3bc40fd33b3dbb80ec16da59e8b63b8572)) (#1259) | ||
|
|
||
| ## Breaking changes | ||
|
|
||
| We had to make a few minor breaking changes in the course of improving `http.batch()` (#1259): | ||
| - The default value for `batchPerHost` has been reduced from `0` (unlimited) to `6`, to more closely match browser behavior. The default value for the `batch` option remains unchanged at `20`. | ||
| - Calling `http.batch(arg)`, where `arg` is an array, would now return an array. Previously, this would have returned an object with integer keys, as explained in [#767](https://github.com/loadimpact/k6/issues/767)... Now `http.batch()` will return an array when you pass it an array, and return an object when you pass an object. |