Skip to content

Commit

Permalink
Issue ReactiveX#179: Migrated Spring Boot 2 Actuator endpoints
Browse files Browse the repository at this point in the history
  • Loading branch information
Robert Winkler committed Mar 19, 2018
1 parent 3e55b50 commit f242318
Show file tree
Hide file tree
Showing 12 changed files with 338 additions and 51 deletions.
1 change: 1 addition & 0 deletions libraries.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ ext {

// Micrometers addon
micrometer: "io.micrometer:micrometer-core:${micrometerVersion}",
micrometer_prometheus: "io.micrometer:micrometer-registry-prometheus:${micrometerVersion}",

// CircuitBreaker documentation
metrics_healthcheck: "io.dropwizard.metrics:metrics-healthchecks:${metricsVersion}",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,310 @@
=== Spring Boot Starter

==== Gradle

Add the Spring Boot 2 Starter of Resilience4j to your compile dependency:

```
repositories {
maven { url 'http://oss.jfrog.org/artifactory/oss-snapshot-local/' }
mavenCentral()
}


dependencies {
compile('io.github.resilience4j:resilience4j-spring-boot2:{release-version}')
}
```

==== Monitoring

Spring Boot Actuator health information can be used to check the status of your running application.
It is often used by monitoring software to alert someone if a production system has serious issues.

==== CircuitBreaker
This demo publishes the status and metrics of all CircuitBreakers via a custom `CircuitBreakerHealthIndicator`.
A closed CircuitBreaker state is mapped to UP, an open state to DOWN and a half-open state to UNKNOWN.

For example:

[source,json]
----
{
"status": "UP",
"backendACircuitBreaker": {
"status": "DOWN",
"failureRate": "60.0%",
"failureRateThreshold": "50.0%",
"maxBufferedCalls": 5,
"bufferedCalls": 5,
"failedCalls": 3,
"notPermittedCalls": 0
},
"backendBCircuitBreaker": {
"status": "UP",
"failureRate": "0.0%",
"failureRateThreshold": "50.0%",
"maxBufferedCalls": 10,
"bufferedCalls": 10,
"failedCalls": 0,
"notPermittedCalls": 0
}
}
----

Metrics are automatically published on the Metrics endpoint.
For example:

[source,json]
----
{
"names": [
"resilience4j.circuitbreaker.backendA.successful",
"resilience4j.circuitbreaker.backendA.failed",
"resilience4j.circuitbreaker.backendA.buffered",
"resilience4j.circuitbreaker.backendA.buffered_max",
"resilience4j.circuitbreaker.backendA.not_permitted",
"resilience4j.circuitbreaker.backendB.successful",
"resilience4j.circuitbreaker.backendB.failed",
"resilience4j.circuitbreaker.backendB.buffered",
"resilience4j.circuitbreaker.backendB.buffered_max",
"resilience4j.circuitbreaker.backendB.not_permitted"
]
}
----

When you want to publish CircuitBreaker endpoints on the Prometheus endpoint, you have to add the dependency `io.micrometer:micrometer-registry-prometheus`.
For example:

[source]
----
# HELP resilience4j_circuitbreaker_calls Circuit Breaker Call Stats
# TYPE resilience4j_circuitbreaker_calls gauge
resilience4j_circuitbreaker_calls{name="backendB",call_result="successful",} 0.0
resilience4j_circuitbreaker_calls{name="backendB",call_result="failed",} 0.0
resilience4j_circuitbreaker_calls{name="backendB",call_result="not_permitted",} 0.0
resilience4j_circuitbreaker_calls{name="backendB",call_result="buffered",} 0.0
resilience4j_circuitbreaker_calls{name="backendB",call_result="buffered_max",} 10.0
resilience4j_circuitbreaker_calls{name="backendA",call_result="successful",} 0.0
resilience4j_circuitbreaker_calls{name="backendA",call_result="failed",} 0.0
resilience4j_circuitbreaker_calls{name="backendA",call_result="not_permitted",} 0.0
resilience4j_circuitbreaker_calls{name="backendA",call_result="buffered",} 0.0
resilience4j_circuitbreaker_calls{name="backendA",call_result="buffered_max",} 5.0
# HELP resilience4j_circuitbreaker_states Circuit Breaker States
# TYPE resilience4j_circuitbreaker_states gauge
resilience4j_circuitbreaker_states{name="backendB",state="closed",} 1.0
resilience4j_circuitbreaker_states{name="backendB",state="open",} 0.0
resilience4j_circuitbreaker_states{name="backendB",state="half_open",} 0.0
resilience4j_circuitbreaker_states{name="backendA",state="closed",} 1.0
resilience4j_circuitbreaker_states{name="backendA",state="open",} 0.0
resilience4j_circuitbreaker_states{name="backendA",state="half_open",} 0.0
----

==== RateLimiter
This demo publishes the status and metrics of all RateLimiter via a custom `RateLimiterHealthIndicator`.
RateLimiterHealthIndicator changes its state DOWN only if there is some permission waiting threads
and they won't be able to unblock until timeout.

For example:

[source,json]
----
{
"status": "UP",
"backendARateLimiter": {
"status": "UP",
"availablePermissions": 10,
"numberOfWaitingThreads": 0
}
}
----

Metrics are automatically published on the Metrics endpoint.
For example:

[source,json]
----
{
"resilience4j.ratelimiter.backendA.available_permissions": 10,
"resilience4j.ratelimiter.backendA.number_of_waiting_threads": 0,
"resilience4j.ratelimiter.backendB.available_permissions": 6,
"resilience4j.ratelimiter.backendB.number_of_waiting_threads": 0
}
----

==== Configuration

===== CircuitBreaker
You can configure your CircuitBreakers in Spring Boot's `application.yml` config file.
For example

[source,yaml]
----
resilience4j.circuitbreaker:
backends:
backendA:
ringBufferSizeInClosedState: 5
ringBufferSizeInHalfOpenState: 3
waitInterval: 5000
failureRateThreshold: 50
eventConsumerBufferSize: 10
registerHealthIndicator: true
backendB:
ringBufferSizeInClosedState: 10
ringBufferSizeInHalfOpenState: 5
waitInterval: 5000
failureRateThreshold: 50
eventConsumerBufferSize: 10
registerHealthIndicator: true
----

===== RateLimiter
You can configure your CircuitBreakers in Spring Boot's `application.yml` config file.
For example

[source,yaml]
----
resilience4j.ratelimiter:
limiters:
backendA:
limitForPeriod: 10
limitRefreshPeriodInMillis: 1000
timeoutInMillis: 0
subscribeForEvents: true
registerHealthIndicator: true
eventConsumerBufferSize: 100
backendB:
limitForPeriod: 6
limitRefreshPeriodInMillis: 500
timeoutInMillis: 3000
----

===== Explicit ordering for CircuitBreaker and RateLimiter aspects
You can adjust `RateLimiterProperties.rateLimiterAspectOrder` and `CircuitBreakerProperties.circuitBreakerAspectOrder`
and explicitly define `CircuitBreaker` and `RateLimiter` execution sequence.
By default `CircuitBreaker` will be executed BEFORE `RateLimiter`.

WARNING: Please be careful changing of `CircuitBreaker`/`RateLimiter` ordering can drastically change application behavior.

==== Event Monitoring

===== CircuitBreaker

The emitted CircuitBreaker events are stored in a separate circular event consumer buffers. The size of a event consumer buffer can be configured per CircuitBreaker in the application.yml file (eventConsumerBufferSize).
The demo adds a custom Spring Boot Actuator endpoint which can be used to monitor the emitted events of your CircuitBreakers.
The endpoint `/actuator/circuitbreakers` lists the names of all CircuitBreaker instances.
For example:

----
{
"circuitBreakers": [
"backendA",
"backendB"
]
}
----

The endpoint `/management/circuitbreaker-events` lists the latest 100 emitted events of all CircuitBreaker instances.

----
{
"circuitBreakerEvents":[
{
"circuitBreakerName": "backendA",
"type": "ERROR",
"creationTime": "2017-01-10T15:39:17.117+01:00[Europe/Berlin]",
"errorMessage": "org.springframework.web.client.HttpServerErrorException: 500 This is a remote exception",
"durationInMs": 0
},
{
"circuitBreakerName": "backendA",
"type": "SUCCESS",
"creationTime": "2017-01-10T15:39:20.518+01:00[Europe/Berlin]",
"durationInMs": 0
},
{
"circuitBreakerName": "backendB",
"type": "ERROR",
"creationTime": "2017-01-10T15:41:31.159+01:00[Europe/Berlin]",
"errorMessage": "org.springframework.web.client.HttpServerErrorException: 500 This is a remote exception",
"durationInMs": 0
},
{
"circuitBreakerName": "backendB",
"type": "SUCCESS",
"creationTime": "2017-01-10T15:41:33.526+01:00[Europe/Berlin]",
"durationInMs": 0
}
]
}
----

The endpoint `/management/circuitbreaker/events/{circuitBreakerName}` lists the latest emitted events of a specific CircuitBreaker.
For example `/management/circuitbreaker/events/backendA`:

----
{
"circuitBreakerEvents":[
{
"circuitBreakerName": "backendA",
"type": "ERROR",
"creationTime": "2017-01-10T15:39:17.117+01:00[Europe/Berlin]",
"errorMessage": "org.springframework.web.client.HttpServerErrorException: 500 This is a remote exception",
"durationInMs": 0
},
{
"circuitBreakerName": "backendA",
"type": "SUCCESS",
"creationTime": "2017-01-10T15:39:20.518+01:00[Europe/Berlin]",
"durationInMs": 0
},
{
"circuitBreakerName": "backendA",
"type": "STATE_TRANSITION",
"creationTime": "2017-01-10T15:39:22.341+01:00[Europe/Berlin]",
"stateTransition": "CLOSED_TO_OPEN"
},
{
"circuitBreakerName": "backendA",
"type": "NOT_PERMITTED",
"creationTime": "2017-01-10T15:39:22.780+01:00[Europe/Berlin]"
}
]
}
----

===== RateLimiter
WARNING: Unlike the CircuitBreaker events, RateLimiter events require explicit subscription.
Use property resilience4j.ratelimiter.limiters.{yourBackendName}.registerHealthIndicator=true

There are literally the same endpoints implemented for RateLimiter,
so for detailed documentation please refer to previous section:

List of available endpoints:

* `/ratelimiters`
* `/ratelimiter-events`
* `/ratelimiter-events/{rateLimiterName}`

Example of response:
----
{
"eventsList": [
{
"rateLimiterName": "backendA",
"rateLimiterEventType": "SUCCESSFUL_ACQUIRE",
"rateLimiterCreationTime": "2017-05-05T21:29:40.463+03:00[Europe/Uzhgorod]"
},
{
"rateLimiterName": "backendA",
"rateLimiterEventType": "SUCCESSFUL_ACQUIRE",
"rateLimiterCreationTime": "2017-05-05T21:29:40.469+03:00[Europe/Uzhgorod]"
},
{
"rateLimiterName": "backendA",
"rateLimiterEventType": "FAILED_ACQUIRE",
"rateLimiterCreationTime": "2017-05-05T21:29:41.268+03:00[Europe/Uzhgorod]"
}
]
}
----
1 change: 1 addition & 0 deletions resilience4j-spring-boot2/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,6 @@ dependencies {
compile project(':resilience4j-ratelimiter')
compile project(':resilience4j-consumer')
testCompile ( libraries.spring_boot2_test )
testCompile ( libraries.micrometer_prometheus )
testCompile ( libraries.spring_boot2_web )
}
Original file line number Diff line number Diff line change
Expand Up @@ -73,9 +73,8 @@ public CircuitBreakerEndpoint circuitBreakerEndpoint(CircuitBreakerRegistry circ

@Bean
@ConditionalOnEnabledEndpoint
public CircuitBreakerEventsEndpoint circuitBreakerEventsEndpoint(EventConsumerRegistry<CircuitBreakerEvent> eventConsumerRegistry,
CircuitBreakerRegistry circuitBreakerRegistry) {
return new CircuitBreakerEventsEndpoint(eventConsumerRegistry, circuitBreakerRegistry);
public CircuitBreakerEventsEndpoint circuitBreakerEventsEndpoint(EventConsumerRegistry<CircuitBreakerEvent> eventConsumerRegistry) {
return new CircuitBreakerEventsEndpoint(eventConsumerRegistry);
}

@Bean
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,14 @@

import java.util.List;

import org.springframework.boot.actuate.endpoint.annotation.Endpoint;
import org.springframework.boot.actuate.endpoint.annotation.ReadOperation;
import org.springframework.boot.actuate.endpoint.web.annotation.WebEndpoint;

import io.github.resilience4j.circuitbreaker.CircuitBreaker;
import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry;


@WebEndpoint(id = "circuitbreaker")
@Endpoint(id = "circuitbreakers")
public class CircuitBreakerEndpoint {

private final CircuitBreakerRegistry circuitBreakerRegistry;
Expand Down
Loading

0 comments on commit f242318

Please sign in to comment.