Dwell-time based load shape testing

[msarahan]

Is your feature request related to a problem? Please describe.

I have a load shape defined, following the helpful examples from https://github.com/locustio/locust/tree/master/examples/custom_shape

My effective user spawn rate is very slow, topping out around 1-2 users/sec (likely something I need to optimize, but that's outside the scope of this proposal). I try to compute appropriately long duration steps, but even still, I often see that the shape runner moves on to the next step when the current step hasn't reached its target user level. After that, Locust seems like it is always playing catch-up, and my shape is wonky.

Describe the solution you'd like

I'd like to propose a new kind of shape runner that takes different arguments:

• users: the number of users desired at a given step
• dwell time: the amount of time to wait, once the target number of users has been reached

The duration is not defined by the user: it is an implicit property that comes from the sum of (actual) spawn times, plus all the dwell times. The spawn rate is also not explicitly defined. It is implicitly "as fast as you can go" because all that matters is that the target is reached.

Describe alternatives you've considered

I looked at adding the existing "wait" parameter to the existing runner's start method, but because the current load shape is based on checking elapsed time and pre-specified duration, I could not think of a way to make it work. At best, some crazy additions to durations or perhaps some timer reset tricks. My brain hurts.

Additional context

I need this for my own work and would be happy to submit a PR if the proposed functionality is agreeable to you.

[cyberw]

I think I understand the use case, but couldnt this be implemented fairly easy as a shape subclass? (similar to https://github.com/locustio/locust/blob/master/examples/custom_shape/stages.py) It doesnt sound like the math would be that hard?

If the shape is ”wonky” then we should try to fix it. Do you have a good repro case for that?

[msarahan]

I couldn't think of a good way to do that because the LoadTestShape base class does not provide any means of knowing what the current number of users is. Thus, the tick() method has no way of knowing when it should advance to the next step. That state is a property of the runner. I think the only way to maintain that separation of state is to implement a new kind of runner, but I'd be happy if I were wrong.

To be clear, "wonky" shape is not a bug on your end. It is that the entire shape gets skewed rightward and never reaches the specified number of users.

[cyberw]

Aha, ok! Well I guess it may make sense to expose the number of currently running users to tick() (or passing the entire environment or runner to the shape’s constructor). I’m sort of +/-0 to doing it.

[max-rocket-internet]

the LoadTestShape base class does not provide any means of knowing what the current number of users is

Exactly. It only knows time passed.

@msarahan I think you need to adjust the time in the steps to be longer to allow your load test to reach the required users. This will require some trial and error. And could vary somewhat between load tests also.

I'd like to propose a new kind of shape runner that takes different arguments:

I think just making the total user count available to the LoadTestShape some how will solve this for you and enable some more complex load tests also 🙂

[msarahan]

Great, thanks for the feedback. I'll work on a PR adding that, with a new example of using the user count.

[msarahan]

resolved by #1719