Loadshed Middleware - Proportional Request Rejection Based on Probabilistic CPU Load #899
Conversation
|
Please take a look at the readme in the root and expand it as well as the workflows |
@Behzad-Khokher don´t forget this part |
Behzad-Khokher
force-pushed
the
feature/loadshed-middleware
branch
from
a92a5be to
fc827f4
Compare
|
Really hope this gets merged! Was a bit new with the workflows and had no idea that those were also suppose to be implemented, @ReneWerner87 thanks for pointing me in the right direction 😄 ! |
|
@Behzad-Khokher pls also add the new middleware to
|
|
@Behzad-Khokher can you please refresh with the master, I have rebuilt the 2 workflows so that they go through the middlewares dynamically and no one has to extend the workflows anymore |
Behzad-Khokher
force-pushed
the
feature/loadshed-middleware
branch
from
81ca27c to
5a76d60
Compare
Behzad-Khokher
force-pushed
the
feature/loadshed-middleware
branch
from
5a76d60 to
8636644
Compare
|
@ReneWerner87 I have refreshed it with master to ensure feature branch is up-to-date. I have also updated |
|
Thx, i will check it again, tomorrow morning |
|
@gofiber/maintainers you can check briefly, then I merge and release |
Loadshed middleware work by intentionally shedding or dropping some of the load (e.g., incoming requests), so the system can maintain acceptable performance and avoid potential failures.
The unique aspect of my implementation is Proportional Request Rejection Based on Probabilistic CPU Load. This means that as the CPU load increases, the probability of rejecting a request rises proportionally.
The formula used for this proportional rejection is:
rejectionProbability := (cpuUsage - cfg.LowerThreshold*100) / (cfg.UpperThreshold - cfg.LowerThreshold)
A: cpuUsage - cfg.LowerThreshold * 100: Calculates how much the current CPU usage exceeds the lower threshold
B: cfg.UpperThreshold - cfg.LowerThreshold: Calculates range between the lower and upper thresholds
C: A/B : division scales the exceeded amount as a fraction of the range between the lower and upper thresholds.
The result is a rejectionProbability. As you can see, keeping everything constant, as the cpuUsage increases, this results in an increase in rejectionProbability. Which can be used to reject request probabilistically.
If the CPU usage is below the LowerThreshold, no requests are rejected. If the CPU usage is above the LowerThreshold, every request has a probability of being rejected depending on how closer the cpu usage is to the UpperThreshold. Once usage exceeds UpperThreshold, all requests are rejected.
The Loadshed Middleware is designed to be extendable for other metrics. Currently the loadshed middleware sheds load based on CPU load.