Global JS event loops in every VU

[na--]

To actually support a lot of cool features and to achieve better compatibility with a ton of JavaScript APIs and libraries out there, we need to be able to support asynchronous execution inside of VUs via a global event loop. We have a localized one in the current websocket implementation, but even that's not enough in a lot of cases.

Things which will benefit from or need an event loop:

• Better websockets (How to allow VU to keep working while keeping WebSocket open #865)
• raw TCP/UDP sockets
• gRPC (Support for gRPC protocol #441) streaming + multiple concurrent unary requests from a single VU (i.e. a http.batch() equivalent)
• HTTP SSE (Server-Sent Events) (HTTP SSE (Server-Sent Event) support for k6 #746)
• HTTP long polling (Support HTTP long polling #1122)
• HTTP/2 server push (HTTP/2 server push #881)
• and any asynchronous protocols, including a lot of messaging ones
• setTimeout(), setInterval(), etc. (Timers API: setTimeout, setInterval, clearTimeout, clearInterval #332)
• async/await (How to use imported async functions in test script?  #779)
• polyfills/mocks (hopefully in JS only, no Go code changes needed) for different browser APIs like XMLHttpRequest or Fetch, so we have a broader support for browser JS libraries and browserified node libraries
• allow us to eventually mock some parts of the node API, if we want to
• likely other nice things, considering that the most of the JS ecosystem is kind of based on it...

Of course, the current looping VU model of k6 will necessitate certain restrictions in the way event loops are normally implemented. For one, each VU in k6 has its own separate JS runtime, so each VU will also need to have its own event loop.

Also, events probably shouldn't cross over between iterations of the same VU, since that would be very confusing and likely quite error-prone, especially with the arrival-rate based executor. Memory leaks would also be a concern... And even if cross-iteration events are easy to implement and we decide to allow them, I think they shouldn't be enabled by default. So some type of configurable post-iteration timeout option should probably be introduced that will be used to control how long k6 will wait for queued events to fire and finish up before it discards them and continues to the next iteration.

As a simple event loop implementation we may look to for inspiration and ideas, the author of goja (the JS runtime k6 uses) also has this repo: https://github.com/dop251/goja_nodejs

[ofauchon]

Hi na,

Do you already have an implementation of a simple event loop in k6 ?

I'd be interested in working on a http sse implementation that can callback or send events to main js code.

Thanks
Olivier

[na--]

Thanks, @ofauchon! There's a simple event loop implementation from goja's author here, but I haven't investigated how easy it would be to integrate something like it into k6 architecture. I assume there's some substantial refactoring involved...

We currently have a very localized event loop in k6 for the websocket handling code here, but unfortunately that type of localized event loop doesn't scale very well and it has some other issues, which is why we're pursuing global event loops.

[mstoykov]

After watching this explanation of how the event loop works in V8, I am of the opinion we should probably get started with this in the most simple way possible:
We add setTimeout (and family), that adds to a queue.
And the iterations works as :

1. When the default function ends we check the queue or if there is anything that will be added to the queue and get it (or wait for it to get added)
2. repeat until nothing in queue or to be added
3. end current iteration

This has the problem that we do not add a way to make an http request asyncly. We should probably just add functions with callbacks on them that does this as is XMLHttpRequest as well as XMLHttpRequest?

Alternatively we can make it possible that on golang calls (http.get for example) k6 checks the queue and possibly starts executing from it. This will need some way of ... copying the stack or creating new stacks for the JS VM ? It will also be complicated as we can touch JS values from inside golang's code and we do it .. for various reasons. As a whole this approach will be more golangish and ... arguably better but it will also be very non-JS and I am of the opinion we should NOT do it, but instead be as JSier as possible in order for more libraries to work out of the box.

This will mean that we will need to change/add the http and the ws api to be async. This can probably be done by adding a callback function to the parameters of the http.* functions (the ws api will need a rewrite) which if present will make the call async?

There is definitely a lot more to be discussed and thought but I think it is about time we make at least some experimental implementation.

[na--]

we should probably get started with this in the most simple way possible

I'd usually agree, but in my head, we should start not with the absolute simplest implementation possible, but figure out roughly what we want and start with an exploratory (i.e. simple and potentially throwaway) implementation that covers our initial list of requirements, so we can see where we're having issues and what's missing.

Specifically, due to the current k6 architecture, I think it's likely a hard requirement for us to have a way to restrict the execution of event loop jobs at some point. It's still unclear to me whether that point is after each iteration is done or after a scheduler/executor is done, but there should be some sort of a timeout after which any outstanding event loop jobs are killed and discarded. And maybe a simple Stop() (like the one in the example event loop here) would be enough, or maybe we'll need more elaborate constructs - for example with contexts and such.

Like everything else with this issue, that's still not clear to me, thus I think we'll probably have to throw away our first implementation 😄

We add setTimeout (and family), that adds to a queue.
And the iterations works as :

1. When the default function ends we check the queue or if there is anything that will be added to the queue and get it (or wait for it to get added)
2. repeat until nothing in queue or to be added
3. end current iteration

This seems reasonable as a start, though I think we should investigate whether there are other opportunities for working on (i.e. executing outstanding events from) the event loop.

Specifically, I think it's probably a bad idea to try to clear the event loop when we're doing HTTP requests, simply because k6 is a load testing tool and it's not very acceptable for us to delay HTTP requests. Which can happen, if some event takes longer to complete than the HTTP request during which we scheduled it. sleep() on the other hand seems like a nice place where we can try and execute some outstanding tasks from the event loop.

This has the problem that we do not add a way to make an http request asyncly. We should probably just add functions with callbacks on them that does this as is XMLHttpRequest as well as XMLHttpRequest?
Alternatively we can make it possible that on golang calls (http.get for example) k6 checks the queue and possibly starts executing from it.

I'm not sure callbacks are the best way to go about here. Instead, if we add a new async: true/false option to the http.Params object, we can have the http.{get,post,request,batch,...} return a promise if async is true and its normal response when async is false. That way, the resulting k6 API will be both backwards compatible and a lot more usable and composable. And we can also re-implement the XMLHttpRequest and the newer Fetch API as pure JS shims that wrap around the k6 http module.

Unfortunately, for that functionality, we'd probably need to extend goja with native support for promises... That shouldn't be too complicated, given Go's async support, but it won't exactly be easy either 😞 It has merits on it's own though, since having an event loop, but only accessing it with setTimeout() / setInterval(), is far from ideal, so we'd likely want promises as well.

This will need some way of ... copying the stack or creating new stacks for the JS VM ?

Actually, I'm not sure that's a problem. Given how the current group() function works (goja calls k6 Go code which calls a new function in goja), I'm not sure we need to worry about the JS stacks, as long as we're executing only a single thing on the runtime.

It will also be complicated as we can touch JS values from inside golang's code and we do it .. for various reasons

Not sure what you mean by that, sorry.

[mstoykov]

It will also be complicated as we can touch JS values from inside golang's code and we do it ..  for various reasons

Not sure what you mean by that, sorry.

What I meant is that we do handle (at least) the parameters to all the functions so if you have something like (pseudo code):

var a = {b: 32};
async function () {a.b = 21;}
k6.golang.function(a)

If we decide to during k6.golang.function we can schedule something from the queue specifically the call from the previous line and we don't actually copy that a in some way we have a race condition. We do this with http.* at least. So we will have to have a mechanism that in the middle of k6/* call we can say "okay now we can 1. start in parallel what the call was suppose to do and 2. start executing something from the queue".
Sleep seems as just as bad an idea ... "you said you want to sleep for 10 seconds but because we actually execute some function that takes forever instead of just sleeping you will wait for 30 seconds". Yes I know that sleeps are "at least as long" but still I don't think this is a good idea.

But apart from this ... this is not how JS works anywhere else ... and I think it doesn't work like that anywhere else specifically because it will be a nightmare to reason about. The way JS event loop works (and all other event loops AFAIK) is that you get 1 function that is executed that can queue other functions and when it finishes those functions can be executed. Not they queue other functions that might be executed in the middle of the execution of this function.

I do agree that SetTimeout(and family) are not particularly user friendly, but I am not proposing that we juts get an eventloop and SetTimeout release it and not touch ever again. I am saying that nomatter how I look at this we do need:

1. eventloop
2. SetTimeout.
   Technically we need some way to add to the queue but this is exactly what SetTimeout does so I propose we just implement it (and the other ClearTimeout. SetInterval and ClearInterval just seem like an easy addition that will mean that we will probably be able to use this polyfill to get Promises 🎊 )

I do agree that ultimately we will probably need http.* (and others) to return Promise in some cases (the params sounds good). But this seem like way too much work as a first experiment and as such I prefer if the first experiment is the bare minimum to get ANY amount of eventloopines. The polyfll mention above also have denodefy which makes a callback based function into a Promise. We can also have

function http_get(url, params) {
    return new Promise(function(resolve, reject) {
       try {
           var resp = http.get(url, [arams);
           resolve(resp);
       } catch(err) {
            reject(err);  
       }
    });
}

Which although will not help you have more than 1 request being made at the same time will show us (somewhat) how code may look when you try to write async code with http.get.

[mstoykov]

I found video explaining the event loop and the difference between executing things with SetTimeout and Promise from the perspective of when will they run.

p.s. The whole video is interesting, but the linked portion is the more relevant part

[na--]

Another use case for asynchronous requests in a single VU: https://community.k6.io/t/run-one-vu-at-a-time-but-have-varying-ramp-up-pattern-within-duration/1377