Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Circuitbreaker not performing MaxRequests operations in Half-open state #53

Open
DaChartreux opened this issue Sep 14, 2023 · 4 comments
Open

Circuitbreaker not performing MaxRequests operations in Half-open state #53

DaChartreux opened this issue Sep 14, 2023 · 4 comments

Comments

@DaChartreux
Copy link

It is stated that

// MaxRequests is the maximum number of requests allowed to pass through
// when the CircuitBreaker is half-open.
// If MaxRequests is 0, the CircuitBreaker allows only 1 request.

Considering this, I assume that the circuit breaker will perform MaxRequests count of operations before opening? But it doesn't.

Here is a reproducible code

package main

import (
	"errors"
	"fmt"
	"github.com/sony/gobreaker"
	"time"
)

func main() {
	cb := gobreaker.NewCircuitBreaker(gobreaker.Settings{
		Name:          "sad",
		MaxRequests:   20,
		Interval:      time.Second * 10,
		Timeout:       time.Second * 1,
		ReadyToTrip: func(counts gobreaker.Counts) bool {
			failureRatio := float64(counts.TotalFailures) / float64(counts.Requests)
			if failureRatio > 0.7 {
				fmt.Printf("will trip? %+v\n", counts)
			}
			return counts.Requests >= 100 && failureRatio > 0.7
		},
		OnStateChange: func(name string, from gobreaker.State, to gobreaker.State) {
			fmt.Printf("Circuit breaker %v state change from %v to %v\n", name, from, to)
		},
	})

	for {
		cb.Execute(func() (interface{}, error) {
			return nil, fun()
		})
	}
}

func fun() error {
	fmt.Println("RUNNING")
	return errors.New("LMAO")
}

This prints...

RUNNING
will trip? {Requests:1 TotalSuccesses:0 TotalFailures:1 ConsecutiveSuccesses:0 ConsecutiveFailures:1}
RUNNING
will trip? {Requests:2 TotalSuccesses:0 TotalFailures:2 ConsecutiveSuccesses:0 ConsecutiveFailures:2}
RUNNING
will trip? {Requests:3 TotalSuccesses:0 TotalFailures:3 ConsecutiveSuccesses:0 ConsecutiveFailures:3}
RUNNING
will trip? {Requests:4 TotalSuccesses:0 TotalFailures:4 ConsecutiveSuccesses:0 ConsecutiveFailures:4}
RUNNING
will trip? {Requests:5 TotalSuccesses:0 TotalFailures:5 ConsecutiveSuccesses:0 ConsecutiveFailures:5}
RUNNING
will trip? {Requests:6 TotalSuccesses:0 TotalFailures:6 ConsecutiveSuccesses:0 ConsecutiveFailures:6}
RUNNING
will trip? {Requests:7 TotalSuccesses:0 TotalFailures:7 ConsecutiveSuccesses:0 ConsecutiveFailures:7}
RUNNING
will trip? {Requests:8 TotalSuccesses:0 TotalFailures:8 ConsecutiveSuccesses:0 ConsecutiveFailures:8}
RUNNING
will trip? {Requests:9 TotalSuccesses:0 TotalFailures:9 ConsecutiveSuccesses:0 ConsecutiveFailures:9}
RUNNING
will trip? {Requests:10 TotalSuccesses:0 TotalFailures:10 ConsecutiveSuccesses:0 ConsecutiveFailures:10}
RUNNING
will trip? {Requests:11 TotalSuccesses:0 TotalFailures:11 ConsecutiveSuccesses:0 ConsecutiveFailures:11}
RUNNING
will trip? {Requests:12 TotalSuccesses:0 TotalFailures:12 ConsecutiveSuccesses:0 ConsecutiveFailures:12}
RUNNING
will trip? {Requests:13 TotalSuccesses:0 TotalFailures:13 ConsecutiveSuccesses:0 ConsecutiveFailures:13}
RUNNING
will trip? {Requests:14 TotalSuccesses:0 TotalFailures:14 ConsecutiveSuccesses:0 ConsecutiveFailures:14}
RUNNING
will trip? {Requests:15 TotalSuccesses:0 TotalFailures:15 ConsecutiveSuccesses:0 ConsecutiveFailures:15}
RUNNING
will trip? {Requests:16 TotalSuccesses:0 TotalFailures:16 ConsecutiveSuccesses:0 ConsecutiveFailures:16}
RUNNING
will trip? {Requests:17 TotalSuccesses:0 TotalFailures:17 ConsecutiveSuccesses:0 ConsecutiveFailures:17}
RUNNING
will trip? {Requests:18 TotalSuccesses:0 TotalFailures:18 ConsecutiveSuccesses:0 ConsecutiveFailures:18}
RUNNING
will trip? {Requests:19 TotalSuccesses:0 TotalFailures:19 ConsecutiveSuccesses:0 ConsecutiveFailures:19}
RUNNING
will trip? {Requests:20 TotalSuccesses:0 TotalFailures:20 ConsecutiveSuccesses:0 ConsecutiveFailures:20}
RUNNING
will trip? {Requests:21 TotalSuccesses:0 TotalFailures:21 ConsecutiveSuccesses:0 ConsecutiveFailures:21}
RUNNING
will trip? {Requests:22 TotalSuccesses:0 TotalFailures:22 ConsecutiveSuccesses:0 ConsecutiveFailures:22}
RUNNING
will trip? {Requests:23 TotalSuccesses:0 TotalFailures:23 ConsecutiveSuccesses:0 ConsecutiveFailures:23}
RUNNING
will trip? {Requests:24 TotalSuccesses:0 TotalFailures:24 ConsecutiveSuccesses:0 ConsecutiveFailures:24}
RUNNING
will trip? {Requests:25 TotalSuccesses:0 TotalFailures:25 ConsecutiveSuccesses:0 ConsecutiveFailures:25}
RUNNING
will trip? {Requests:26 TotalSuccesses:0 TotalFailures:26 ConsecutiveSuccesses:0 ConsecutiveFailures:26}
RUNNING
will trip? {Requests:27 TotalSuccesses:0 TotalFailures:27 ConsecutiveSuccesses:0 ConsecutiveFailures:27}
RUNNING
will trip? {Requests:28 TotalSuccesses:0 TotalFailures:28 ConsecutiveSuccesses:0 ConsecutiveFailures:28}
RUNNING
will trip? {Requests:29 TotalSuccesses:0 TotalFailures:29 ConsecutiveSuccesses:0 ConsecutiveFailures:29}
RUNNING
will trip? {Requests:30 TotalSuccesses:0 TotalFailures:30 ConsecutiveSuccesses:0 ConsecutiveFailures:30}
RUNNING
will trip? {Requests:31 TotalSuccesses:0 TotalFailures:31 ConsecutiveSuccesses:0 ConsecutiveFailures:31}
RUNNING
will trip? {Requests:32 TotalSuccesses:0 TotalFailures:32 ConsecutiveSuccesses:0 ConsecutiveFailures:32}
RUNNING
will trip? {Requests:33 TotalSuccesses:0 TotalFailures:33 ConsecutiveSuccesses:0 ConsecutiveFailures:33}
RUNNING
will trip? {Requests:34 TotalSuccesses:0 TotalFailures:34 ConsecutiveSuccesses:0 ConsecutiveFailures:34}
RUNNING
will trip? {Requests:35 TotalSuccesses:0 TotalFailures:35 ConsecutiveSuccesses:0 ConsecutiveFailures:35}
RUNNING
will trip? {Requests:36 TotalSuccesses:0 TotalFailures:36 ConsecutiveSuccesses:0 ConsecutiveFailures:36}
RUNNING
will trip? {Requests:37 TotalSuccesses:0 TotalFailures:37 ConsecutiveSuccesses:0 ConsecutiveFailures:37}
RUNNING
will trip? {Requests:38 TotalSuccesses:0 TotalFailures:38 ConsecutiveSuccesses:0 ConsecutiveFailures:38}
RUNNING
will trip? {Requests:39 TotalSuccesses:0 TotalFailures:39 ConsecutiveSuccesses:0 ConsecutiveFailures:39}
RUNNING
will trip? {Requests:40 TotalSuccesses:0 TotalFailures:40 ConsecutiveSuccesses:0 ConsecutiveFailures:40}
RUNNING
will trip? {Requests:41 TotalSuccesses:0 TotalFailures:41 ConsecutiveSuccesses:0 ConsecutiveFailures:41}
RUNNING
will trip? {Requests:42 TotalSuccesses:0 TotalFailures:42 ConsecutiveSuccesses:0 ConsecutiveFailures:42}
RUNNING
will trip? {Requests:43 TotalSuccesses:0 TotalFailures:43 ConsecutiveSuccesses:0 ConsecutiveFailures:43}
RUNNING
will trip? {Requests:44 TotalSuccesses:0 TotalFailures:44 ConsecutiveSuccesses:0 ConsecutiveFailures:44}
RUNNING
will trip? {Requests:45 TotalSuccesses:0 TotalFailures:45 ConsecutiveSuccesses:0 ConsecutiveFailures:45}
RUNNING
will trip? {Requests:46 TotalSuccesses:0 TotalFailures:46 ConsecutiveSuccesses:0 ConsecutiveFailures:46}
RUNNING
will trip? {Requests:47 TotalSuccesses:0 TotalFailures:47 ConsecutiveSuccesses:0 ConsecutiveFailures:47}
RUNNING
will trip? {Requests:48 TotalSuccesses:0 TotalFailures:48 ConsecutiveSuccesses:0 ConsecutiveFailures:48}
RUNNING
will trip? {Requests:49 TotalSuccesses:0 TotalFailures:49 ConsecutiveSuccesses:0 ConsecutiveFailures:49}
RUNNING
will trip? {Requests:50 TotalSuccesses:0 TotalFailures:50 ConsecutiveSuccesses:0 ConsecutiveFailures:50}
RUNNING
will trip? {Requests:51 TotalSuccesses:0 TotalFailures:51 ConsecutiveSuccesses:0 ConsecutiveFailures:51}
RUNNING
will trip? {Requests:52 TotalSuccesses:0 TotalFailures:52 ConsecutiveSuccesses:0 ConsecutiveFailures:52}
RUNNING
will trip? {Requests:53 TotalSuccesses:0 TotalFailures:53 ConsecutiveSuccesses:0 ConsecutiveFailures:53}
RUNNING
will trip? {Requests:54 TotalSuccesses:0 TotalFailures:54 ConsecutiveSuccesses:0 ConsecutiveFailures:54}
RUNNING
will trip? {Requests:55 TotalSuccesses:0 TotalFailures:55 ConsecutiveSuccesses:0 ConsecutiveFailures:55}
RUNNING
will trip? {Requests:56 TotalSuccesses:0 TotalFailures:56 ConsecutiveSuccesses:0 ConsecutiveFailures:56}
RUNNING
will trip? {Requests:57 TotalSuccesses:0 TotalFailures:57 ConsecutiveSuccesses:0 ConsecutiveFailures:57}
RUNNING
will trip? {Requests:58 TotalSuccesses:0 TotalFailures:58 ConsecutiveSuccesses:0 ConsecutiveFailures:58}
RUNNING
will trip? {Requests:59 TotalSuccesses:0 TotalFailures:59 ConsecutiveSuccesses:0 ConsecutiveFailures:59}
RUNNING
will trip? {Requests:60 TotalSuccesses:0 TotalFailures:60 ConsecutiveSuccesses:0 ConsecutiveFailures:60}
RUNNING
will trip? {Requests:61 TotalSuccesses:0 TotalFailures:61 ConsecutiveSuccesses:0 ConsecutiveFailures:61}
RUNNING
will trip? {Requests:62 TotalSuccesses:0 TotalFailures:62 ConsecutiveSuccesses:0 ConsecutiveFailures:62}
RUNNING
will trip? {Requests:63 TotalSuccesses:0 TotalFailures:63 ConsecutiveSuccesses:0 ConsecutiveFailures:63}
RUNNING
will trip? {Requests:64 TotalSuccesses:0 TotalFailures:64 ConsecutiveSuccesses:0 ConsecutiveFailures:64}
RUNNING
will trip? {Requests:65 TotalSuccesses:0 TotalFailures:65 ConsecutiveSuccesses:0 ConsecutiveFailures:65}
RUNNING
will trip? {Requests:66 TotalSuccesses:0 TotalFailures:66 ConsecutiveSuccesses:0 ConsecutiveFailures:66}
RUNNING
will trip? {Requests:67 TotalSuccesses:0 TotalFailures:67 ConsecutiveSuccesses:0 ConsecutiveFailures:67}
RUNNING
will trip? {Requests:68 TotalSuccesses:0 TotalFailures:68 ConsecutiveSuccesses:0 ConsecutiveFailures:68}
RUNNING
will trip? {Requests:69 TotalSuccesses:0 TotalFailures:69 ConsecutiveSuccesses:0 ConsecutiveFailures:69}
RUNNING
will trip? {Requests:70 TotalSuccesses:0 TotalFailures:70 ConsecutiveSuccesses:0 ConsecutiveFailures:70}
RUNNING
will trip? {Requests:71 TotalSuccesses:0 TotalFailures:71 ConsecutiveSuccesses:0 ConsecutiveFailures:71}
RUNNING
will trip? {Requests:72 TotalSuccesses:0 TotalFailures:72 ConsecutiveSuccesses:0 ConsecutiveFailures:72}
RUNNING
will trip? {Requests:73 TotalSuccesses:0 TotalFailures:73 ConsecutiveSuccesses:0 ConsecutiveFailures:73}
RUNNING
will trip? {Requests:74 TotalSuccesses:0 TotalFailures:74 ConsecutiveSuccesses:0 ConsecutiveFailures:74}
RUNNING
will trip? {Requests:75 TotalSuccesses:0 TotalFailures:75 ConsecutiveSuccesses:0 ConsecutiveFailures:75}
RUNNING
will trip? {Requests:76 TotalSuccesses:0 TotalFailures:76 ConsecutiveSuccesses:0 ConsecutiveFailures:76}
RUNNING
will trip? {Requests:77 TotalSuccesses:0 TotalFailures:77 ConsecutiveSuccesses:0 ConsecutiveFailures:77}
RUNNING
will trip? {Requests:78 TotalSuccesses:0 TotalFailures:78 ConsecutiveSuccesses:0 ConsecutiveFailures:78}
RUNNING
will trip? {Requests:79 TotalSuccesses:0 TotalFailures:79 ConsecutiveSuccesses:0 ConsecutiveFailures:79}
RUNNING
will trip? {Requests:80 TotalSuccesses:0 TotalFailures:80 ConsecutiveSuccesses:0 ConsecutiveFailures:80}
RUNNING
will trip? {Requests:81 TotalSuccesses:0 TotalFailures:81 ConsecutiveSuccesses:0 ConsecutiveFailures:81}
RUNNING
will trip? {Requests:82 TotalSuccesses:0 TotalFailures:82 ConsecutiveSuccesses:0 ConsecutiveFailures:82}
RUNNING
will trip? {Requests:83 TotalSuccesses:0 TotalFailures:83 ConsecutiveSuccesses:0 ConsecutiveFailures:83}
RUNNING
will trip? {Requests:84 TotalSuccesses:0 TotalFailures:84 ConsecutiveSuccesses:0 ConsecutiveFailures:84}
RUNNING
will trip? {Requests:85 TotalSuccesses:0 TotalFailures:85 ConsecutiveSuccesses:0 ConsecutiveFailures:85}
RUNNING
will trip? {Requests:86 TotalSuccesses:0 TotalFailures:86 ConsecutiveSuccesses:0 ConsecutiveFailures:86}
RUNNING
will trip? {Requests:87 TotalSuccesses:0 TotalFailures:87 ConsecutiveSuccesses:0 ConsecutiveFailures:87}
RUNNING
will trip? {Requests:88 TotalSuccesses:0 TotalFailures:88 ConsecutiveSuccesses:0 ConsecutiveFailures:88}
RUNNING
will trip? {Requests:89 TotalSuccesses:0 TotalFailures:89 ConsecutiveSuccesses:0 ConsecutiveFailures:89}
RUNNING
will trip? {Requests:90 TotalSuccesses:0 TotalFailures:90 ConsecutiveSuccesses:0 ConsecutiveFailures:90}
RUNNING
will trip? {Requests:91 TotalSuccesses:0 TotalFailures:91 ConsecutiveSuccesses:0 ConsecutiveFailures:91}
RUNNING
will trip? {Requests:92 TotalSuccesses:0 TotalFailures:92 ConsecutiveSuccesses:0 ConsecutiveFailures:92}
RUNNING
will trip? {Requests:93 TotalSuccesses:0 TotalFailures:93 ConsecutiveSuccesses:0 ConsecutiveFailures:93}
RUNNING
will trip? {Requests:94 TotalSuccesses:0 TotalFailures:94 ConsecutiveSuccesses:0 ConsecutiveFailures:94}
RUNNING
will trip? {Requests:95 TotalSuccesses:0 TotalFailures:95 ConsecutiveSuccesses:0 ConsecutiveFailures:95}
RUNNING
will trip? {Requests:96 TotalSuccesses:0 TotalFailures:96 ConsecutiveSuccesses:0 ConsecutiveFailures:96}
RUNNING
will trip? {Requests:97 TotalSuccesses:0 TotalFailures:97 ConsecutiveSuccesses:0 ConsecutiveFailures:97}
RUNNING
will trip? {Requests:98 TotalSuccesses:0 TotalFailures:98 ConsecutiveSuccesses:0 ConsecutiveFailures:98}
RUNNING
will trip? {Requests:99 TotalSuccesses:0 TotalFailures:99 ConsecutiveSuccesses:0 ConsecutiveFailures:99}
RUNNING
will trip? {Requests:100 TotalSuccesses:0 TotalFailures:100 ConsecutiveSuccesses:0 ConsecutiveFailures:100}
Circuit breaker sad state change from closed to open
Circuit breaker sad state change from open to half-open
RUNNING
Circuit breaker sad state change from half-open to open
Circuit breaker sad state change from open to half-open
RUNNING
Circuit breaker sad state change from half-open to open
^C

Why is it executing only once in half-open state?
@hankji
Copy link

hankji commented Sep 22, 2023

Same problem.
because all request must be success then state will be open in the interval duration.
look like it's a bug
Does anyone still maintain this repository?

@DaChartreux
Copy link
Author

According to "Release It! Design and Deploy Production-ready Software" book by Michael T. Nygard,
image

We make a single call in half open mode, if that fails, circuit breaker opens, else it closes.
For me, this was new coming from resilience4j

@plabondutta
Copy link

@DaChartreux
But then again, if that is so, the MaxRequests settings shouldn't be present at all. This just doesn't make any sense.

@YoshiyukiMineo
Copy link
Member

YoshiyukiMineo commented Oct 13, 2024 •
edited
Loading

This behavior is following the specification:

If any request fails, the circuit breaker assumes that the fault is still present so it reverts back to the Open state
https://learn.microsoft.com/en-us/previous-versions/msp-n-p/dn589784(v=pandp.10)?redirectedfrom=MSDN

If any operation fails even once, it will move to the Open state.

A limited number of requests from the application are allowed

A limited number (MaxRequests) of requests are allowed at most.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@hankji @YoshiyukiMineo @plabondutta @DaChartreux