-
Notifications
You must be signed in to change notification settings - Fork 122
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #899 from Behzad-Khokher/feature/loadshed-middleware
Loadshed Middleware - Proportional Request Rejection Based on Probabilistic CPU Load
- Loading branch information
Showing
11 changed files
with
507 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| name-template: "Loadshed - v$RESOLVED_VERSION" | ||
| tag-template: "loadshed/v$RESOLVED_VERSION" | ||
| tag-prefix: loadshed/v | ||
| include-paths: | ||
| - loadshed | ||
| categories: | ||
| - title: "❗ Breaking Changes" | ||
| labels: | ||
| - "❗ BreakingChange" | ||
| - title: "🚀 New Features" | ||
| labels: | ||
| - "✏️ Feature" | ||
| - title: "🧹 Updates" | ||
| labels: | ||
| - "🧹 Updates" | ||
| - "🤖 Dependencies" | ||
| - title: "🐛 Fixes" | ||
| labels: | ||
| - "☢️ Bug" | ||
| - title: "📚 Documentation" | ||
| labels: | ||
| - "📒 Documentation" | ||
| change-template: "- $TITLE (#$NUMBER)" | ||
| change-title-escapes: '\<*_&' # You can add # and @ to disable mentions, and add ` to disable code blocks. | ||
| exclude-contributors: | ||
| - dependabot | ||
| - dependabot[bot] | ||
| version-resolver: | ||
| major: | ||
| labels: | ||
| - "major" | ||
| - "❗ BreakingChange" | ||
| minor: | ||
| labels: | ||
| - "minor" | ||
| - "✏️ Feature" | ||
| patch: | ||
| labels: | ||
| - "patch" | ||
| - "📒 Documentation" | ||
| - "☢️ Bug" | ||
| - "🤖 Dependencies" | ||
| - "🧹 Updates" | ||
| default: patch | ||
| template: | | ||
| $CHANGES | ||
| **Full Changelog**: https://github.com/$OWNER/$REPOSITORY/compare/$PREVIOUS_TAG...loadshed/v$RESOLVED_VERSION | ||
| Thank you $CONTRIBUTORS for making this update possible. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| name: Release Drafter Loadshed | ||
|
|
||
| on: | ||
| push: | ||
| # branches to consider in the event; optional, defaults to all | ||
| branches: | ||
| - master | ||
| - main | ||
| paths: | ||
| - "loadshed/**" | ||
|
|
||
| jobs: | ||
| draft_release_loadshed: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - uses: release-drafter/release-drafter@v5 | ||
| with: | ||
| config-name: release-drafter-loadshed.yml | ||
| env: | ||
| GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| name: "Test Loadshed" | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - master | ||
| - main | ||
| paths: | ||
| - "loadshed/**" | ||
| pull_request: | ||
| paths: | ||
| - "loadshed/**" | ||
|
|
||
| jobs: | ||
| Tests: | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| matrix: | ||
| go-version: | ||
| - 1.18.x | ||
| - 1.19.x | ||
| - 1.20.x | ||
| - 1.21.x | ||
| steps: | ||
| - name: Fetch Repository | ||
| uses: actions/checkout@v4 | ||
| - name: Install Go | ||
| uses: actions/setup-go@v5 | ||
| with: | ||
| go-version: "${{ matrix.go-version }}" | ||
| - name: Run Test | ||
| working-directory: ./loadshed | ||
| run: go test -v -race ./... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| --- | ||
| id: loadshed | ||
| --- | ||
|
|
||
| # LoadShed | ||
|
|
||
|  | ||
| [](https://gofiber.io/discord) | ||
|  | ||
|  | ||
|  | ||
|
|
||
| The LoadShed middleware for [Fiber](https://github.com/gofiber/fiber) is designed to help manage server load by shedding requests based on certain load criteria. | ||
|
|
||
| **Note: Requires Go 1.19 and above** | ||
|
|
||
| ## Install | ||
|
|
||
| This middleware supports Fiber v2 | ||
|
|
||
| ``` | ||
| go get -u github.com/gofiber/fiber/v2 | ||
| go get -u github.com/gofiber/contrib/loadshed | ||
| ``` | ||
|
|
||
| ## Signatures | ||
|
|
||
| ```go | ||
| loadshed.New(config ...loadshed.Config) fiber.Handler | ||
| ``` | ||
|
|
||
| ## Examples | ||
|
|
||
| To use the LoadShed middleware in your Fiber application, import it and apply it to your Fiber app. Here's an example: | ||
|
|
||
| ```go | ||
| package main | ||
|
|
||
| import ( | ||
| "github.com/gofiber/fiber/v2" | ||
| loadshed "github.com/gofiber/contrib/loadshed" | ||
| ) | ||
|
|
||
| func main() { | ||
| app := fiber.New() | ||
|
|
||
| // Configure and use LoadShed middleware | ||
| app.Use(loadshed.New(loadshed.Config{ | ||
| Criteria: &loadshed.CPULoadCriteria{ | ||
| LowerThreshold: 0.75, // Set your own lower threshold | ||
| UpperThreshold: 0.90, // Set your own upper threshold | ||
| Interval: 10 * time.Second, | ||
| Getter: &loadshed.DefaultCPUPercentGetter{}, | ||
| }, | ||
| })) | ||
|
|
||
| app.Get("/", func(c *fiber.Ctx) error { | ||
| return c.SendString("Welcome!") | ||
| }) | ||
|
|
||
| app.Listen(":3000") | ||
| } | ||
| ``` | ||
|
|
||
| ## Config | ||
|
|
||
| The LoadShed middleware in Fiber offers various configuration options to tailor the load shedding behavior according to the needs of your application. | ||
|
|
||
| | Property | Type | Description | Default | | ||
| | :------- | :---------------------- | :--------------------------------------------------- | :---------------------- | | ||
| | Next | `func(*fiber.Ctx) bool` | Function to skip this middleware when returned true. | `nil` | | ||
| | Criteria | `LoadCriteria` | Interface for defining load shedding criteria. | `&CPULoadCriteria{...}` | | ||
|
|
||
| ## LoadCriteria | ||
|
|
||
| LoadCriteria is an interface in the LoadShed middleware that defines the criteria for determining when to shed load in the system. Different implementations of this interface can use various metrics and algorithms to decide when and how to shed incoming requests to maintain system performance. | ||
|
|
||
| ### CPULoadCriteria | ||
|
|
||
| `CPULoadCriteria` is an implementation of the `LoadCriteria` interface, using CPU load as the metric for determining whether to shed requests. | ||
|
|
||
| #### Properties | ||
|
|
||
| | Property | Type | Description | | ||
| | :------------- | :----------------- | :------------------------------------------------------------------------------------------------------------------------------------ | | ||
| | LowerThreshold | `float64` | The lower CPU usage threshold as a fraction (0.0 to 1.0). Requests are considered for shedding when CPU usage exceeds this threshold. | | ||
| | UpperThreshold | `float64` | The upper CPU usage threshold as a fraction (0.0 to 1.0). All requests are shed when CPU usage exceeds this threshold. | | ||
| | Interval | `time.Duration` | The time interval over which the CPU usage is averaged for decision making. | | ||
| | Getter | `CPUPercentGetter` | Interface to retrieve CPU usage percentages. | | ||
|
|
||
| #### How It Works | ||
|
|
||
| `CPULoadCriteria` determines the load on the system based on CPU usage and decides whether to shed incoming requests. It operates on the following principles: | ||
|
|
||
| - **CPU Usage Measurement**: It measures the CPU usage over a specified interval. | ||
| - **Thresholds**: Utilizes `LowerThreshold` and `UpperThreshold` values to decide when to start shedding requests. | ||
| - **Proportional Rejection Probability**: | ||
| - **Below `LowerThreshold`**: No requests are rejected, as the system is considered under acceptable load. | ||
| - **Between `LowerThreshold` and `UpperThreshold`**: The probability of rejecting a request increases as the CPU usage approaches the `UpperThreshold`. This is calculated using the formula: | ||
| ```plaintext | ||
| rejectionProbability := (cpuUsage - LowerThreshold*100) / (UpperThreshold - LowerThreshold) | ||
| ``` | ||
| - **Above `UpperThreshold`**: All requests are rejected to prevent system overload. | ||
| This mechanism ensures that the system can adaptively manage its load, maintaining stability and performance under varying traffic conditions. | ||
| ## Default Config | ||
| This is the default configuration for `LoadCriteria` in the LoadShed middleware. | ||
| ```go | ||
| var ConfigDefault = Config{ | ||
| Next: nil, | ||
| Criteria: &CPULoadCriteria{ | ||
| LowerThreshold: 0.90, // 90% CPU usage as the start point for considering shedding | ||
| UpperThreshold: 0.95, // 95% CPU usage as the point where all requests are shed | ||
| Interval: 10 * time.Second, // CPU usage is averaged over 10 seconds | ||
| Getter: &DefaultCPUPercentGetter{}, // Default method for getting CPU usage | ||
| }, | ||
| } | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| package loadshed | ||
|
|
||
| import ( | ||
| "context" | ||
| "math/rand" | ||
| "time" | ||
|
|
||
| "github.com/shirou/gopsutil/cpu" | ||
| ) | ||
|
|
||
| // LoadCriteria interface for different types of load metrics. | ||
| type LoadCriteria interface { | ||
| Metric(ctx context.Context) (float64, error) | ||
| ShouldShed(metric float64) bool | ||
| } | ||
|
|
||
| // CPULoadCriteria for using CPU as a load metric. | ||
| type CPULoadCriteria struct { | ||
| LowerThreshold float64 | ||
| UpperThreshold float64 | ||
| Interval time.Duration | ||
| Getter CPUPercentGetter | ||
| } | ||
|
|
||
| func (c *CPULoadCriteria) Metric(ctx context.Context) (float64, error) { | ||
| percentages, err := c.Getter.PercentWithContext(ctx, c.Interval, false) | ||
| if err != nil || len(percentages) == 0 { | ||
| return 0, err | ||
| } | ||
| return percentages[0], nil | ||
| } | ||
|
|
||
| func (c *CPULoadCriteria) ShouldShed(metric float64) bool { | ||
| if metric > c.UpperThreshold*100 { | ||
| return true | ||
| } else if metric > c.LowerThreshold*100 { | ||
| rejectionProbability := (metric - c.LowerThreshold*100) / (c.UpperThreshold - c.LowerThreshold) | ||
| // #nosec G404 | ||
| return rand.Float64()*100 < rejectionProbability | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| type CPUPercentGetter interface { | ||
| PercentWithContext(ctx context.Context, interval time.Duration, percpu bool) ([]float64, error) | ||
| } | ||
|
|
||
| type DefaultCPUPercentGetter struct{} | ||
|
|
||
| func (_ *DefaultCPUPercentGetter) PercentWithContext(ctx context.Context, interval time.Duration, percpu bool) ([]float64, error) { | ||
| return cpu.PercentWithContext(ctx, interval, percpu) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| module loadshed | ||
|
|
||
| go 1.20 | ||
|
|
||
| require ( | ||
| github.com/gofiber/fiber/v2 v2.52.0 | ||
| github.com/shirou/gopsutil v3.21.11+incompatible | ||
| ) | ||
|
|
||
| require ( | ||
| github.com/andybalholm/brotli v1.0.5 // indirect | ||
| github.com/go-ole/go-ole v1.2.6 // indirect | ||
| github.com/google/uuid v1.5.0 // indirect | ||
| github.com/klauspost/compress v1.17.0 // indirect | ||
| github.com/mattn/go-colorable v0.1.13 // indirect | ||
| github.com/mattn/go-isatty v0.0.20 // indirect | ||
| github.com/mattn/go-runewidth v0.0.15 // indirect | ||
| github.com/rivo/uniseg v0.2.0 // indirect | ||
| github.com/stretchr/testify v1.8.4 // indirect | ||
| github.com/tklauser/go-sysconf v0.3.13 // indirect | ||
| github.com/tklauser/numcpus v0.7.0 // indirect | ||
| github.com/valyala/bytebufferpool v1.0.0 // indirect | ||
| github.com/valyala/fasthttp v1.51.0 // indirect | ||
| github.com/valyala/tcplisten v1.0.0 // indirect | ||
| github.com/yusufpapurcu/wmi v1.2.3 // indirect | ||
| golang.org/x/sys v0.15.0 // indirect | ||
| ) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| github.com/andybalholm/brotli v1.0.5 h1:8uQZIdzKmjc/iuPu7O2ioW48L81FgatrcpfFmiq/cCs= | ||
| github.com/andybalholm/brotli v1.0.5/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig= | ||
| github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= | ||
| github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY= | ||
| github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0= | ||
| github.com/gofiber/fiber/v2 v2.52.0 h1:S+qXi7y+/Pgvqq4DrSmREGiFwtB7Bu6+QFLuIHYw/UE= | ||
| github.com/gofiber/fiber/v2 v2.52.0/go.mod h1:KEOE+cXMhXG0zHc9d8+E38hoX+ZN7bhOtgeF2oT6jrQ= | ||
| github.com/google/uuid v1.5.0 h1:1p67kYwdtXjb0gL0BPiP1Av9wiZPo5A8z2cWkTZ+eyU= | ||
| github.com/google/uuid v1.5.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= | ||
| github.com/klauspost/compress v1.17.0 h1:Rnbp4K9EjcDuVuHtd0dgA4qNuv9yKDYKK1ulpJwgrqM= | ||
| github.com/klauspost/compress v1.17.0/go.mod h1:ntbaceVETuRiXiv4DpjP66DpAtAGkEQskQzEyD//IeE= | ||
| github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA= | ||
| github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg= | ||
| github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM= | ||
| github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY= | ||
| github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= | ||
| github.com/mattn/go-runewidth v0.0.15 h1:UNAjwbU9l54TA3KzvqLGxwWjHmMgBUVhBiTjelZgg3U= | ||
| github.com/mattn/go-runewidth v0.0.15/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w= | ||
| github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= | ||
| github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY= | ||
| github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc= | ||
| github.com/shirou/gopsutil v3.21.11+incompatible h1:+1+c1VGhc88SSonWP6foOcLhvnKlUeu/erjjvaPEYiI= | ||
| github.com/shirou/gopsutil v3.21.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA= | ||
| github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= | ||
| github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= | ||
| github.com/tklauser/go-sysconf v0.3.13 h1:GBUpcahXSpR2xN01jhkNAbTLRk2Yzgggk8IM08lq3r4= | ||
| github.com/tklauser/go-sysconf v0.3.13/go.mod h1:zwleP4Q4OehZHGn4CYZDipCgg9usW5IJePewFCGVEa0= | ||
| github.com/tklauser/numcpus v0.7.0 h1:yjuerZP127QG9m5Zh/mSO4wqurYil27tHrqwRoRjpr4= | ||
| github.com/tklauser/numcpus v0.7.0/go.mod h1:bb6dMVcj8A42tSE7i32fsIUCbQNllK5iDguyOZRUzAY= | ||
| github.com/valyala/bytebufferpool v1.0.0 h1:GqA5TC/0021Y/b9FG4Oi9Mr3q7XYx6KllzawFIhcdPw= | ||
| github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc= | ||
| github.com/valyala/fasthttp v1.51.0 h1:8b30A5JlZ6C7AS81RsWjYMQmrZG6feChmgAolCl1SqA= | ||
| github.com/valyala/fasthttp v1.51.0/go.mod h1:oI2XroL+lI7vdXyYoQk03bXBThfFl2cVdIA3Xl7cH8g= | ||
| github.com/valyala/tcplisten v1.0.0 h1:rBHj/Xf+E1tRGZyWIWwJDiRY0zc1Js+CV5DqwacVSA8= | ||
| github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc= | ||
| github.com/yusufpapurcu/wmi v1.2.3 h1:E1ctvB7uKFMOJw3fdOW32DwGE9I7t++CRUEMKvFoFiw= | ||
| github.com/yusufpapurcu/wmi v1.2.3/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0= | ||
| golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= | ||
| golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= | ||
| golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= | ||
| golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc= | ||
| golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= | ||
| gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= |
Oops, something went wrong.