v0.40.0 release notes

release notes/v0.40.0.md

@@ -0,0 +1,253 @@
+k6 v0.40.0 is here! :tada: This release includes the promised experimental modules, some other small fixes and a lot of changes needed for the metric time series, unindexable tags and native ESM support. [Roadmap and future plans section](#roadmap-and-future-plans) makes another appearance with more info on how things are going and what should be expected
+
+# New Features and notable changes!
+
+## ⚠Breaking change⚠: Main module/script no longer taints the global scope [#2571](https://github.com/grafana/k6/pull/2571)
+
+As part of the ESM changes, it was found that anything defined in the main module's scope was also accessible globally. This was because it was directly evaluated in the global scope.
+This has now been remedied and is no longer the case. This is a *breaking* change, but given that the whole point of modules, CommonJS or ESM, is to separate modules, this is obviously rather a bug than a feature.
+
+On the other note, we've seen [reports](https://github.com/grafana/k6/issues/2623) by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, and only a few usages in a script. So if you need something accessed globally, our suggested workaround is to set it explicitly on the global object `globalThis`.
+
+## ⚠Breaking change⚠: `k6/ws` take `throw` option into consideration [#2247](https://github.com/grafana/k6/issues/2247)
+
+`k6/http` has been using the [`throw` option](https://k6.io/docs/using-k6/k6-options/reference/#throw) to figure out if should throw an exception on errors or whether it should return an response object with error set on it (and log it).
+
+This functionality is finally also available for `k6/ws` which previous to this would've *always* thrown an exception making scripting around it more involved. [#2616](https://github.com/grafana/k6/pull/2616)
+
+This also is a small breaking change as by default `throw` is `false` so it will now no longer throw an exception, but instead now it will return a Response with `error` property.
+
+Thank you @fatelei for making this change 🙇🎉!
+
+## Experimental modules [#2630](https://github.com/grafana/k6/pull/2630) and [#2656](https://github.com/grafana/k6/pull/2656)
+
+As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try them and provide us feedback using community forums or GitHub issues. 
+
+This release contains three experimental modules that are bringing support for interaction with [redis](https://redis.io/), a new Websockets API that copies the "web" [WebSocket API](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) and [setTimeout](https://developer.mozilla.org/en-US/docs/Web/API/setTimeout) and friends as the third one.
+
+Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, by the very nature of the import path, starting with `k6/experimental`, when they stop being experimental, that will break user code. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.
+
+### Redis example
+
+Here is fairly big example showcasing using [xk6-redis](https://github.com/grafana/xk6-redis), but as an experimental module, to keep track of data in redis.
+
+```javascript
+import { check } from "k6";
+import http from "k6/http";
+import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
+import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";
+
+export const options = {
+  scenarios: {
+    usingRedisData: {
+      executor: "shared-iterations",
+      vus: 10,
+      iterations: 200,
+      exec: "measureUsingRedisData",
+    },
+  },
+};
+
+// Instantiate a new redis client
+const redisClient = new redis.Client({
+  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
+  password: __ENV.REDIS_PASSWORD || "",
+});
+
+// Prepare an array of crocodile ids for later use
+// in the context of the measureUsingRedisData function.
+const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);
+
+export function setup() {
+  redisClient.sadd("crocodile_ids", ...crocodileIDs);
+}
+
+export function measureUsingRedisData() {
+  // Pick a random crocodile id from the dedicated redis set,
+  // we have filled in setup().
+  redisClient
+    .srandmember("crocodile_ids")
+    .then((randomID) => {
+      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
+      const res = http.get(url);
+
+      check(res, {
+        "status is 200": (r) => r.status === 200,
+        "content-type is application/json": (r) =>
+          r.headers["Content-Type"] === "application/json",
+      });
+
+      return url;
+    })
+    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
+}
+
+export function teardown() {
+  redisClient.del("crocodile_ids");
+}
+
+export function handleSummary(data) {
+  redisClient
+    .hgetall("k6_crocodile_fetched")
+    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
+    .then((data) =>
+      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
+    )
+    .then(() => redisClient.del("k6_crocodile_fetched"));
+
+  return {
+    stdout: textSummary(data, { indent: "  ", enableColors: true }),
+  };
+}
+```
+
+This example also showcases how to write some data and clean up after yourself.
+
+TODO maybe have a bunch of issues this can be used to solve 🤔
+
+The extension does *not* run a redis server. All the infrastructure connected to and the actual running and scaling of redis needs to be handled separately.
+
+You can see more example and documentation in the [repo](https://github.com/grafana/xk6-redis) of the extension is based on, as well as in the [official k6 documentation](https://k6.io/docs/javascript-api/k6-experimental-redis/).
+
+### WebSockets example
+
+This is a rewrite of the current WebSocket example at https://test-api.k6.io/.
+
+
+This showcases how 1 VU can run multiple WebSockets connection asynchronously and how to stop them after a period using the `setTimeout` and friends.
+```javascript
+import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
+import { WebSocket } from "k6/experimental/websockets"
+import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"
+
+let chatRoomName = 'publicRoom'; // choose your chat room name
+let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m
+
+
+export default function() {
+  for (let i = 0; i < 4; i++) {
+    startWSWorker(i)
+  }
+}
+
+function startWSWorker(id) {
+  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
+  let ws = new WebSocket(url);
+  ws.addEventListener("open", () => {
+    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));
+
+    ws.addEventListener("message", (e) => {
+      let msg = JSON.parse(e.data);
+      if (msg.event === 'CHAT_MSG') {
+        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
+      }
+      else if (msg.event === 'ERROR') {
+        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
+      }
+      else {
+        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
+      }
+    })
+
+
+    let intervalId = setInterval(() => {
+      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
+    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds
+
+
+    let timeout1id = setTimeout(function() {
+      clearInterval(intervalId)
+      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
+      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
+    }, sessionDuration);
+
+    let timeout2id = setTimeout(function() {
+      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
+      ws.close();
+    }, sessionDuration + 3000);
+
+    ws.addEventListener("close", () => {
+      clearTimeout(timeout1id);
+      clearTimeout(timeout2id);
+      console.log(`VU ${__VU}:${id}: disconnected`);
+    })
+  });
+}
+```
+
+It is important to note that no k6 iterations do not finish if there is any WebSocket still open or if a timeout/interval is not cleared or triggered. This means that you need to be careful when writing the script to clear all interval and close the websocket in some point in time. k6 will still kill the whole process if it takes too long to stop after the maximum test duration is reached.
+
+Current issues and future improvements for the WebSockets API can be found in its [issue tracker](https://github.com/grafana/xk6-websockets/issues). Currently, documentation is available through [MDN](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) although some features are not yet supported:
+- no Blob binary type - ArrayBuffer is the default
+- no `onMessage` and co. - only addEventListener
+
+## First class support for JavaScript Classes
+
+As part of [updating goja](https://github.com/grafana/k6/pull/2610) k6 got native, as in not by transpiling by the internal babel, support for classes 🎉. Because this actually implements classes as they described in the latest ECMAScript specification this also means we get a ton of additional class features that were never previously supported such as private fields. Additionally at least one bug [#1763](https://github.com/grafana/k6/issues/1763) was fixed as a result of this, but probably many more as well.
+
+Due to this fairly big change there are chances that some code will now behave differently. Please report any issues, though take into consideration that it's possible that the new behavior is just the correct one.
+
+Other updates from goja are:
+- optimizations around using strings and some access patterns
+- support for `\u{01234}` Unicode point syntax in regexp
+- Fixed a case where interrupting the VM would not have worked, especially around try/catch usage. This was particularly problematic for k6 as this could lead to the whole k6 hanging forever. [#2600](https://github.com/grafana/k6/pull/2600)
+
+Many thanks to @dop251 for continuing to improve goja! 
+
+### New Test runtime for module extension developers [#2598](https://github.com/grafana/k6/issues/2598)
+
+While we are developing extensions internally it constantly comes up that you need to do create the same structure over and over again when you run stuff in tests. With the addition of the event loops it is now required to setup it as well. Even k6 team members get parts of this wrong every once in a while, so we decided to add a small type to be used by (extension) module developers to write tests easier. [#2602](https://github.com/grafana/k6/pull/2602)
+
+[This API](https://pkg.go.dev/go.k6.io/[email protected]/js/modulestest#Runtime) will likely change and evolve as we add more functionality or as we change the k6 internal API.
+
+
+# Breaking changes
+
+- [#2632](https://github.com/grafana/k6/pull/2632) As part of refactoring to set tags provided to `metric.add` in the order they are provided, it was found out that you can provide tags as multiple objects/maps. This has never been the intend use and has never been documented. As such and as that makes no sense given every other API we have decided to remove this ability.
+- [#2571](https://github.com/grafana/k6/pull/2571) Main module definitions no longer accessible globally. If need to use `globalThis` to set and access things globally.
+- [#2582](https://github.com/grafana/k6/pull/2582) If `RegisterCallback` result is called twice this will now lead to a panic, instead of the second call silently breaking the event loop. This has never been the expected behavior and calling it twice is always a bug in the code using it.
+- As part of [#2596](https://github.com/grafana/k6/pull/2596) the `tainted` property of the Metric type is no longer outputted by the JSON output. That property was (likely) always going to have the value `false` as it was outputted at the beginning of the test.
+- [#2616](https://github.com/grafana/k6/pull/2616) `throw` is now taking into consideration for `k6/ws` connections and will by default return a response object with `error` property instead of throwing an exception.
+
+# Bug fixes
+
+- [#2585](https://github.com/grafana/k6/pull/2585) `http.batch()` now displays an error if not exactly 1 argument is given to it([#1289](https://github.com/grafana/k6/issues/1289)). Thanks, @vanshaj!
+- [#2596](https://github.com/grafana/k6/pull/2596) Fix a potential data race in the JSON output. Includes a breaking change where `tainted` property is no longer outputted. That property was (likely) always going to have the value `false` as it was outputted at the beginning of the test.
+- [#2604](https://github.com/grafana/k6/pull/2604) Fix SSL keylogger not working with absolute paths.
+- [#2637](https://github.com/grafana/k6/pull/2637) Fix setting the options `rps` to `0` or below leading to exceptions. Now setting it to 0 or below disables the limit. Thanks, @tbourrely. [#2613](https://github.com/grafana/k6/issues/2613)
+- [#2278](https://github.com/grafana/k6/issues/2278) Reading directly `options.tags` was not possible. This was fixed by accident by [#2631](https://github.com/grafana/k6/pull/2631). It is still recommended that `k6/execution` is used instead to access the [final options of the test](https://k6.io/docs/javascript-api/k6-execution/#get-test-options).
+
+# Maintenance and internal improvements
+
+- [#2590](https://github.com/grafana/k6/pull/2590) Update direct dependencies without any interesting changes apart goja.
+- [#2591](https://github.com/grafana/k6/pull/2591) Changes to the CI process to always build rpm/deb and windows packages and use nfpm to do it. 
+- [#2593](https://github.com/grafana/k6/pull/2593) Internal cleanup after finally removing `common.Bind`.
+- [#2597](https://github.com/grafana/k6/pull/2597) Fix go benchmarks we have broken over time.
+- [#2599](https://github.com/grafana/k6/pull/2599) Reformat `//nolint` comments as part of getting ready for go 1.19.
+- A bunch of fixes for tests [#2589](https://github.com/grafana/k6/pull/2589), [#2620](https://github.com/grafana/k6/pull/2620), [#2625](https://github.com/grafana/k6/pull/2625), [#2643](https://github.com/grafana/k6/pull/2543), [#2647](https://github.com/grafana/k6/pull/2647), [#2648](https://github.com/grafana/k6/pull/2648),
+- [#2607](https://github.com/grafana/k6/pull/2607) Fix build badge in the README. Thanks @AetherUnbound!
+- [#2614](https://github.com/grafana/k6/pull/2614) Fix advice for rpm install on Amazone Linux.
+- [#2615](https://github.com/grafana/k6/pull/2615) Improve documentation of RegisterCallback due to feedback on how hard it was to understand how to use it properly.
+- [#2627](https://github.com/grafana/k6/pull/2627) Create distinct test state objects for the pre-init and run phases.
+- [#2635](https://github.com/grafana/k6/pull/2635) Drop License header in each file.
+- [#2636](https://github.com/grafana/k6/pull/2636) Add [SECURITY.md](https://github.com/grafana/k6/security/policy) with instructions how to report security issues responsibly.
+- [#2641](https://github.com/grafana/k6/pull/2641) Fix spelling of `lose`. Thanks @spazm!
+- Update to golangci-lint v1.47.2 and enable a bunch more linters. [#2609](https://github.com/grafana/k6/pull/2609), [#2611](https://github.com/grafana/k6/pull/2611). Also, drop obsolete configurations [#2619](https://github.com/grafana/k6/pull/2619).
+
+# Roadmap and future plans
+
+## Native support for ECMAScript modules
+
+UPDATE THIS
+
+## Refactoring metrics
+https://github.com/grafana/k6/pull/2629
+https://github.com/grafana/k6/pull/2631
+https://github.com/grafana/k6/pull/2634
+https://github.com/grafana/k6/pull/2655
+
+Update this whole section
+
+### Prometheus remote-write output
+
+### Tracing support