Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reworks "life cycle" explanation page #624

Closed
wants to merge 15 commits into from
Closed

Reworks "life cycle" explanation page #624

Changes from 5 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
931ba19
Reworks "life cycle" explanation page
MattDodsonEnglish Apr 18, 2022
9c5cd2f
life cycle, makes list consistent and parallel
MattDodsonEnglish Apr 18, 2022
3784c54
life-cycle, restructures section and table
MattDodsonEnglish Apr 18, 2022
a1fb10c
Makes table more accurate
MattDodsonEnglish Apr 19, 2022
9508949
life-cycle, removes unnecessary details
MattDodsonEnglish Apr 19, 2022
9638337
life-cycle, more informative intro
MattDodsonEnglish Apr 19, 2022
85bef48
fixes nested bullets
MattDodsonEnglish Apr 19, 2022
534db8c
lifecycle, restructure, adds diagram and def list
MattDodsonEnglish Apr 22, 2022
09a9bb7
small text fixes
MattDodsonEnglish Apr 22, 2022
5201831
life cycle; removes unneeded info
MattDodsonEnglish Apr 25, 2022
0bdedcf
better init code example
MattDodsonEnglish Apr 25, 2022
894e68d
Better summary of how performance improves
MattDodsonEnglish Apr 25, 2022
6ce0d9e
Merge branch 'main' into lifecycle-edits
MattDodsonEnglish Apr 25, 2022
72712b9
Merge branch 'lifecycle-edits'
MattDodsonEnglish Apr 25, 2022
e86434d
Merge branch 'main' into lifecycle-edits
MattDodsonEnglish Apr 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 77 additions & 61 deletions src/data/markdown/translated-guides/en/02 Using k6/06 Test life cycle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
---
title: 'Test life cycle'
excerpt: 'The four distinct life cycle stages in a k6 test are "init", "setup", "VU" and "teardown".'
excerpt: 'The four distinct life cycle stages in a k6 test are "init", "setup", "VU", and "teardown".'
---

The four distinct life cycle stages in a k6 test are "init", "setup", "VU" and "teardown"
Throughout the documentation, you will also see us referring to it as "init code", "VU code" etc.
A k6 test has four distinct stages, which always run in the same order:

1. *Init code* initializes VUs
2. *Setup code* sets up data and the test environment
MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved
3. *VU code* runs the test
4. *Teardown code* cleans up data and stops the test environment
MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved


<CodeGroup labels={["The four life cycle stages"]} lineNumbers={[true]}>

Expand All @@ -26,10 +31,26 @@ export function teardown(data) {

</CodeGroup>

## A quick overview of test stages

This table provides the essential information about each stage.
The rest of this page goes into deeper technical detail.

| Test stage | Used to | Example | Called | Required? |
|-----------------|------------------------------------------------------------|-----------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|-----------|
| **1. init** | Load local files, import modules, declare global variables | Open JSON file, Import module | Once per VU\* | Required |
| **2. Setup** | Set up data for processing, share data among VUs | Call API to start test environment | Once | Optional |
| **3. VU code** | Run the test function, usually `default` | Make https requests, validate responses | Once per iteration, as many times as the test options require | Required |
| **4. Teardown** | Process result of setup code, stop test environment | Validate that setup had a certain result, send webhook notifying that test has finished | Once per script | Optional |

\* In cloud scripts, init code might be called more often.



## Init and VU stages

Scripts must contain, at the very least, a `default` function - this defines the entry point
for your VUs, similar to the `main()` function in many other languages:
Scripts must contain, at the very least, a `default` function.
MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved
The `default` function defines the entry point for your VUs, similar to the `main()` function in other languages:

<CodeGroup labels={["Default/Main function"]} lineNumbers={[true]}>

Expand All @@ -41,33 +62,41 @@ export default function () {

</CodeGroup>

_"Why not just run my script normally, from top to bottom"_, you might ask - the answer is: we
do, but code **inside** and **outside** your `default` function can do different things.
Why not just run my script normally, from top to bottom?

This is a good question, and the answer is: we do.
But code *inside* and *outside* your `default` function do different things.

Code inside `default` is called *VU code*.
VU code runs over and over through the test duration.
Besides setup and teardown, code outside of `default` is *init code*.
It runs only once per VU.

VU code can make HTTP requests, emit metrics, and generally do everything you'd expect a load test to do.
There are a few important exceptions. VU code:
* *Does not* load files from your local filesystem,
* *Does not* import any other modules.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this explains what you can do and where, but not why which arguably is what the question above asks.

The short answer is that "we need the script to be executed in order to know what k6 should do". That includes:

  • getting hte configuration
  • knowing which functions to be called on setup/teardown and iterations
  • all the things above that you have mentioned above are also separated in order to
    • on one side not generate metrics whenever we have to run the "init code" and to be able to do it fairly repeatable
    • getting an archive with everything needed to run the code somewhere else (like in the cloud)

And that last two points is where a lot of the restrictions come from. But ultimately we just use the fact that we need to be able to run "init code" to get the options and the definitions to do that.

Also, ultimately running

let b = 5;

twice in the same javascript context will give you an error as b has already been defined. Putting it in the function and calling the function is what makes a new context and let you do

function f () {
  let res = http.request()
  // something with res
}
f();
f();

So there is also kind of another technical reason for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these explanations! I think you are getting at something very essential to keep in mind when explaining these internals.

We always must ask: what does this information mean to the user? What bad thing will happen if they don't know this information?

It's tricky sometimes to separate what information matters for implementers from what matters for end users.


Code inside `default` is called "VU code", and is run over and over for as long as the test is
running. Code outside of it is called "init code", and is run only once per VU.
Instead of VU code, init code does these jobs.

VU code can make HTTP requests, emit metrics, and generally do everything you'd expect a load
test to do - with a few important exceptions: you can't load anything from your local filesystem,
or import any other modules. This all has to be done from the init code.
## Benefits of separating init and VU code

We have two reasons for this. The first is, of course: performance.
There are two chief reasons we separate these code stages:

If you read a file from disk on every single script iteration, it'd be needlessly slow; even
if you cache the contents of the file and any imported modules, it'd mean the _first run_ of the
script would be much slower than all the others. Worse yet, if you have a script that imports
or loads things based on things that can only be known at runtime, you'd get slow iterations
thrown in every time you load something new.
* Performance is faster and more accurate.
* The design is more flexible and modular.

But there's another, more interesting reason. By forcing all imports and file reads into the
init context, we make an important design goal possible; we want to support three different
execution modes without the need for you to modify your scripts; local, cloud and clustered
execution. In the case of cloud and clustered execution we know which files will be needed, so
we distribute only those files. We know which modules will be imported, so we can bundle them
up from the get-go. And, tying into the performance point above, the other nodes don't even
need writable filesystems - everything can be kept in-memory.
On the performance side, if you read a file from disk every iteration, the script would be needlessly slow.
Even if you cached the contents of the file and imported modules, the _first iteration_ of the script would be much slower than all subsequent iterations.
Worse, if your script imports or loads data dynamically, you'd get slow iterations each time you loaded something new.

As an added bonus, you can use this to reuse data between iterations (but only for the same VU):
Besides improving performance, these stages let you execute in different modes without modifying your scripts.
You should be able to run a k6 script in local, cloud, and clustered systems.
In the case of cloud and clustered execution, we know which files are needed, so we distribute only those files.
We know which modules to import, so we can bundle them from the get-go.
And, tying into the performance benefits, the other nodes don't even need writable filesystems&mdash;everything can be kept in-memory.

As a bonus, you can reuse data between iterations (but only for the same VU):

<CodeGroup labels={[]}>

Expand All @@ -83,29 +112,25 @@ export default function () {

## The default function life-cycle

A VU will execute the default function from start to end in sequence. Nothing out of the ordinary
so far, but here's the important part; once the VU reaches the end of the default function it will
loop back to the start and execute the code all over.
A VU executes the default function from start to end in sequence.
Once the VU reaches the end of the default function, it loops back to the start and executes the code all over.

As part of this "restart" process, the VU is reset. Cookies are cleared and TCP connections
might be torn down, depending on your test configuration options.
As part of this "restart" process, k6 resets the VU.
Cookies are cleared and TCP connections
might be torn down (depending on your test configuration options).

> Make sure to use `sleep()` statements to pace your VUs properly. An appropriate amount of
> sleep/think time at the end of the default function is often needed to properly simulate a
> user reading content on a page. If you don't have a `sleep()` statement at the end of
> the default function your VU might be more "aggressive" than you've planned.
> Make sure to use `sleep()` statements to pace your VUs properly, simulating a user reading content on your page.
> If you don't have a `sleep()` statement at the end of the default function, your VU might be more "aggressive" than you've planned.
>
> VU without any `sleep()` is akin to a user who constantly presses F5 to refresh the page.
MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved

## Setup and teardown stages

Beyond the required init and VU stages, which is code run for each VU, k6 also supports test-wide
setup and teardown stages, like many other testing frameworks and tools. The `setup` and
`teardown` functions, like the `default` function, needs to be exported functions. But unlike
the `default` function `setup` and `teardown` are only called once for a test. `setup` is called
at the beginning of the test, after the init stage but before the VU stage (`default` function),
and `teardown` is called at the end of a test, after the VU stage (`default` function). Therefore,
VU number is 0 while executing the `setup` and `teardown` functions.
Like `default`, `setup` and `teardown` functions must be exported functions.
But unlike the `default` function, k6 calls `setup` and `teardown` only once per test.

* `setup` is called at the beginning of the test, after the init stage but before the VU stage.
* `teardown` is called at the end of a test, after the VU stage (`default` function).

Again, let's have a look at the basic structure of a k6 test:

Expand All @@ -129,19 +154,15 @@ export function teardown(data) {

</CodeGroup>

You might have noticed the function signature of the `default` function and `teardown` function
takes an argument, which we here refer to as `data`.
You might have noticed the function signatures of the `default` and `teardown` functions
take an argument, referred to here as `data`.

This `data` will be whatever is returned in the `setup` function, so a mechanism for passing data
from the setup stage to the subsequent VU and teardown stages in a way that, again, is compatible
with our goal of supporting local, cloud and clustered execution modes without requiring script
changes when switching between them. (it might or might not be the same node that runs the setup
and teardown stages in the cloud or clustered execution mode).
This `data` is whatever the `setup` function returns.

To support all of those modes, only data (i.e. JSON) can be passed between `setup()` and the
other stages, any passed functions will be stripped.
You can pass only data (i.e. JSON) between `setup()` and the other stages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a note that it's a copy of what setup returns. So that it:

  1. uses more memory if it's big
  2. you can't actually change it in the VU code and see the changes in the teardown. Although any VU will see it's changes to the data. This though is likely to change as we might freeze the whole object just to make it less obvious that it won't work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although any VU will see it's changes to the data. This though is likely to change as we might freeze the whole object just to make it less obvious that it won't work

@mstoykov

Sorry, I'm a bit confused here. So, VU code can manipulate data from setup, but it can't pass that data to teardown. Is that correct?

And, in the table, I note that setup code can be used to " share data among VUs". How is this data shared. Is it from one VU to the next VU? Or does each VU grab "clean" data from the setup code?

However it works, it seems like, if VU code can manipulate data that doesn't end up in the teardown stage, then the data is not really "passed" from Setup to VU to Default.

Rather, setup passes data in two places. Once, directly to teardown. And once (or many times??) directly to default.

I've made a small diagram to illustrate how I conceive this. Excuse me, I'm sure said some inaccurate things here, but I thought it may be easier to correct than to explain :-) .

image

Copy link
Contributor

@mstoykov mstoykov Apr 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more or less -yes to all of those :).

In practice implementing data being modified and all that modifications being visible by :

  1. all VUs
  2. teardown

Will be ... extremely complicated and (computationally) expensive especially in distributed execution. Which is why we haven't even discussed it ever. But long long time ago I made fixes so that they have separate copies - previous to that this was sometimes panicking as multiple VUs were writing to the same not thread safe structure.

edit: the data is passed to each VU separately - but that shouldn't really be depended upon or documented. We do in general really want if it's a singel copy that is just unmodifiable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, this sounds pretty complicated. I think I'm just going to just suggest that it's a "copy," and leave it at that. Especially since it sounds like this data may get "frozen" in the VU stage in some later k6 version anyway.

For the people who are motivated enough to actually use this implementation in some advanced way, well they can look at the source code (and find this discussion). 🙂

You cannot pass functions.

Here's an example of doing just that, passing some data from setup to VU and teardown stages:
Here's an example of passing some data from setup to VU and teardown stages:

<CodeGroup labels={["Setup/Teardown"]} lineNumbers={[true]}>

Expand All @@ -163,9 +184,8 @@ export function teardown(data) {

</CodeGroup>

A big difference between the init stage and setup/teardown stages is that you have the full k6
API available in the latter, you can for example make HTTP requests in the setup and teardown
stages:
You can call the full k6 API in the setup and teardown stages, unlike the init stage.
For example, you can for make HTTP requests:

<CodeGroup labels={["Setup/Teardown with HTTP request"]} lineNumbers={[true]}>

Expand All @@ -188,14 +208,10 @@ export default function (data) {

</CodeGroup>

Note that any requests made in the setup and teardown stages will be counted in the end-of-test
summary. Those requests will be tagged appropriately with the `::setup` and `::teardown` values
for the `group` metric tag, so that you can filter them in JSON output or InfluxDB.

## Skip setup and teardown execution
MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved

It is possible to skip the execution of setup and teardown stages using the two options `--no-setup` and
`--no-teardown` respectively.
You can skip the execution of setup and teardown stages using the options `--no-setup` and
`--no-teardown`.

<CodeGroup labels={["Skipping setup/teardown execution"]} lineNumbers={[true]}>

Expand Down