Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks #15979

analogue · 2023-01-13T21:54:42Z

Description

WAN federated clusters are not able to failover all traffic from one datacenter to another via passive healthchecks when the control plane is down without being able to set MaxEjectionPercent to 100%. BaseEjectionTime just provides additional configurability on the backoff interval until the max ejection interval is hit.

Testing & Reproduction steps

Existing passive health check unit tests updated
Existing passive health check integration tests updated to verify configuration
Open to adding a functional integration test if it would be useful, however I'm planning to add support for these config values via K8s CRDs in the consul-k8s repo. e2e tests on K8s might be more useful.

PR Checklist

updated test coverage
external facing docs updated
not a security concern

…stream confings

david-yu · 2023-01-13T22:09:29Z

@analogue I assume a consul-k8s crd update is also in order?

analogue · 2023-01-13T22:15:45Z

@analogue I assume a consul-k8s crd update is also in order?

Yep, that is coming up next 😄.

boruszak

I reviewed 15979.txt, service-defaults.mdx, and envoy.mdx for edits to the documentation.

Overall LGTM. Confirm that the descriptions match phrasing before merging. Let me know if you have any questions or need any additional assistance.

Approving on behalf of consul-docs.

boruszak · 2023-01-17T16:34:47Z

website/content/docs/connect/config-entries/service-defaults.mdx

+                  name: 'MaxEjectionPercent',
+                  type: 'int: 10',
+                  description: `Measured in percent (%), the maximum percentage of hosts that can be ejected
+                    from a upstream cluster due to passive health check failures. If not specified, inherits 
+                    Envoy's default of 10% or at least one host.`,


Suggested change

name: 'MaxEjectionPercent',

type: 'int: 10',

description: `Measured in percent (%), the maximum percentage of hosts that can be ejected

from a upstream cluster due to passive health check failures. If not specified, inherits

Envoy's default of 10% or at least one host.`,

name: 'MaxEjectionPercent',

type: 'int: 10',

description: `Measured in percent (%), the maximum percentage of hosts that can be ejected

from an upstream cluster due to passive health check failures. If not specified, inherits

Envoy's default value of 10. Ejects at least one host when set to a non-zero value.`,

Two suggestions:

Don't use the % in the second sentence. The type is an integer and you already say "measured in percent" - so the default is 10 nor 10%.

I understand the "At least one host" statement, but the phrasing is a little muddled. The idea is that if the Max Ejection percent is set to 10 but there's only three hosts in the cluster, the percent is less than a single host makes up in the deployment. So at least one host must be ejected when the value is not 0. Feel free to adjust or reject if you disagree with my phrasing or understanding.

boruszak · 2023-01-17T16:51:47Z

website/content/docs/connect/config-entries/service-defaults.mdx

+                  name: 'BaseEjectionTime',
+                  type: 'duration: 30s',
+                  description: `The base time that a host is ejected for. The real time is equal to the base
+                    time multiplied by the number of times the host has been ejected and is capped by 
+                    max_ejection_time (Default 300s). If not speficied, inherits Envoy's default value of 30s.`,


Suggested change

name: 'BaseEjectionTime',

type: 'duration: 30s',

description: `The base time that a host is ejected for. The real time is equal to the base

time multiplied by the number of times the host has been ejected and is capped by

max_ejection_time (Default 300s). If not speficied, inherits Envoy's default value of 30s.`,

name: 'BaseEjectionTime',

type: 'duration: 30s',

description: `The base time that a host is ejected for. The real time is equal to the base

time multiplied by the number of times the host has been ejected and is capped by

`max_ejection_time` (default 300s) in the [Envoy proxy configuration](/consul/docs/connect/proxies/envoy#ejection). If not specified, inherits Envoy's default value of 30s.`,

Added a link directly to the values.

I curious what you mean by "real time" - is that word set in stone on your end? If not, I'd be more descriptive - e.g., "The amount of time that elapses before an ejected host can rejoin the cluster is equal to the base time multiplied by..."

boruszak · 2023-01-17T16:54:11Z

website/content/docs/connect/config-entries/service-defaults.mdx

+                  name: 'MaxEjectionPercent',
+                  type: 'int: 10',
+                  description: `Measured in percent (%), the maximum percentage of hosts that can be ejected
+                    from a upstream cluster due to passive health check failures. If not specified, inherits 
+                    Envoy's default of 10% or at least one host.`,
+                },
+                {
+                  name: 'BaseEjectionTime',
+                  type: 'duration: 30s',
+                  description: `The base time that a host is ejected for. The real time is equal to the base
+                    time multiplied by the number of times the host has been ejected and is capped by 
+                    max_ejection_time (Default 300s). If not speficied, inherits Envoy's default value of 30s.`,
+                },


Suggested change

name: 'MaxEjectionPercent',

type: 'int: 10',

description: `Measured in percent (%), the maximum percentage of hosts that can be ejected

from a upstream cluster due to passive health check failures. If not specified, inherits

Envoy's default of 10% or at least one host.`,

},

{

name: 'BaseEjectionTime',

type: 'duration: 30s',

description: `The base time that a host is ejected for. The real time is equal to the base

time multiplied by the number of times the host has been ejected and is capped by

max_ejection_time (Default 300s). If not speficied, inherits Envoy's default value of 30s.`,

},

name: 'MaxEjectionPercent',

type: 'int: 10',

description: `Measured in percent (%), the maximum percentage of hosts that can be ejected

from an upstream cluster due to passive health check failures. If not specified, inherits

Envoy's default value of 10. Ejects at least one host when set to a non-zero value.`,

},

{

name: 'BaseEjectionTime',

type: 'duration: 30s',

description: The base time that a host is ejected for. The real time is equal to the base

time multiplied by the number of times the host has been ejected and is capped by

`max_ejection_time` (default 300s) in the [Envoy proxy configuration](/consul/docs/connect/proxies/envoy#ejection). If not specified, inherits Envoy's default value of 30s.`,

},

Suggestion to match previous edits

boruszak · 2023-01-17T16:57:31Z

website/content/docs/connect/proxies/envoy.mdx

+  - `max_ejection_percent` - The maximum percentage of hosts that can be ejected
+    from a upstream cluster due to passive health check failures. If not specified,
+    inherits Envoy's default of 10% or at least one host.
+  - `base_ejection_time` - The base time that a host is ejected for. The real 
+     time is equal to the base time multiplied by the number of times the host has 
+     been ejected and is capped by max_ejection_time (Default 300s). If not 
+     speficied, inherits Envoy's default value of 30s.


Suggested change

- `max_ejection_percent` - The maximum percentage of hosts that can be ejected

from a upstream cluster due to passive health check failures. If not specified,

inherits Envoy's default of 10% or at least one host.

- `base_ejection_time` - The base time that a host is ejected for. The real

time is equal to the base time multiplied by the number of times the host has

been ejected and is capped by max_ejection_time (Default 300s). If not

speficied, inherits Envoy's default value of 30s.

<a name="ejection"/>

- `max_ejection_percent` - The maximum percentage of hosts that can be ejected

from an upstream cluster when passive health checks fail. If not specified,

inherits Envoy's default value of 10. Ejects at least one host when set to a non-zero value.

- `base_ejection_time` - The base time that a host is ejected for. The amount of time that elapses before an ejected host can rejoin the cluster is equal to the base time multiplied by the number of times the host has

been ejected, and is capped by `max_ejection_time` (default 300s). If not

specified, inherits Envoy's default value of 30s.

github-actions · 2023-03-27T01:32:12Z

This pull request has been automatically flagged for inactivity because it has not been acted upon in the last 60 days. It will be closed if no new activity occurs in the next 30 days. Please feel free to re-open to resurrect the change if you feel this has happened by mistake. Thank you for your contributions.

david-yu · 2023-04-26T17:20:14Z

Added backport/1.15 since 1.15 is now out.

david-yu · 2023-04-26T23:10:59Z

Looks like this did not backport at all for 1.14 and 1.15.

komapa · 2023-05-25T03:09:16Z

Thank you for this feature but can we also document it on https://developer.hashicorp.com/consul/docs/v1.14.x/connect/proxies/envoy#passive_health_check?

Thanks!

analogue added 8 commits January 12, 2023 13:38

Add MaxEjectionPercent to config entry

aad783a

Add BaseEjectionTime to config entry

941f6f3

Add MaxEjectionPercent and BaseEjectionTime to protobufs

f8a6bf6

Add MaxEjectionPercent and BaseEjectionTime to api

34cd119

Fix integration test breakage

b8b0ee1

Verify MaxEjectionPercent and BaseEjectionTime in integration test up…

0f41695

…stream confings

Website docs for MaxEjectionPercent and BaseEjection time

f9e220c

Add make docs to browse docs at http://localhost:3000

b5be02f

github-actions bot added theme/api Relating to the HTTP API interface theme/envoy/xds Related to Envoy support type/docs Documentation needs to be created/updated/clarified labels Jan 13, 2023

Changelog entry

07d54ee

analogue added the backport/1.14 label Jan 13, 2023

analogue requested review from erichaberkorn, david-yu and ChipV223 January 13, 2023 22:13

analogue marked this pull request as ready for review January 13, 2023 22:14

analogue requested a review from a team as a code owner January 13, 2023 22:14

boruszak approved these changes Jan 17, 2023

View reviewed changes

hashi-derek self-requested a review January 17, 2023 22:20

hashi-derek approved these changes Jan 18, 2023

View reviewed changes

analogue added 2 commits January 19, 2023 10:04

so that is the difference between consul-docker and dev-docker

3039ea1

Merge branch 'main' into spatel/NET-1646-add-max-ejection-percent

94f1f56

vercel bot deployed to Preview – consul January 19, 2023 16:20 View deployment

blah

ebd2707

github-actions bot added the meta/stale Automatically flagged for inactivity by stalebot label Mar 27, 2023

merge and resolve conflicts

5ada62f

vercel bot deployed to Preview – consul-ui-staging April 26, 2023 14:52 View deployment

vercel bot deployed to Preview – consul April 26, 2023 14:54 View deployment

Maliz added 2 commits April 26, 2023 08:24

update proto funcs

c52f5aa

update proto

9d2d8b1

david-yu added the backport/1.15 This release series is no longer active on CE. Use backport/ent/1.15. label Apr 26, 2023

malizz merged commit 5eaeb7b into main Apr 26, 2023

malizz deleted the spatel/NET-1646-add-max-ejection-percent branch April 26, 2023 22:59

malizz mentioned this pull request May 1, 2023

backport PR 15979 to 1.15.x #17208

Merged

4 tasks

malizz mentioned this pull request May 9, 2023

backport pr 15979 into 1.14.x #17287

Merged

3 tasks

p-linnane mentioned this pull request Jun 1, 2023

consul 1.15.3 Homebrew/homebrew-core#132601

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks #15979

Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks #15979

analogue commented Jan 13, 2023 •

edited

Loading

david-yu commented Jan 13, 2023

analogue commented Jan 13, 2023

boruszak left a comment

boruszak Jan 17, 2023

boruszak Jan 17, 2023

boruszak Jan 17, 2023

boruszak Jan 17, 2023

github-actions bot commented Mar 27, 2023

david-yu commented Apr 26, 2023

david-yu commented Apr 26, 2023

komapa commented May 25, 2023

Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks #15979

Support Envoy's MaxEjectionPercent and BaseEjectionTime config entries for passive health checks #15979

Conversation

analogue commented Jan 13, 2023 • edited Loading

Description

Testing & Reproduction steps

PR Checklist

david-yu commented Jan 13, 2023

analogue commented Jan 13, 2023

boruszak left a comment

Choose a reason for hiding this comment

boruszak Jan 17, 2023

Choose a reason for hiding this comment

boruszak Jan 17, 2023

Choose a reason for hiding this comment

boruszak Jan 17, 2023

Choose a reason for hiding this comment

boruszak Jan 17, 2023

Choose a reason for hiding this comment

github-actions bot commented Mar 27, 2023

david-yu commented Apr 26, 2023

david-yu commented Apr 26, 2023

komapa commented May 25, 2023

analogue commented Jan 13, 2023 •

edited

Loading