Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

phys/consul: Allow tuning of session ttl and lock wait time #4352

Merged
merged 5 commits into from
Apr 18, 2018

Conversation

vishalnayak
Copy link
Member

Fixes: #2949

Even after setting the session_ttl to "2s" and lock_wait_time to "2s", I am observing a 16s-18s delay in assuming active duty by the standby. I guess it is because of the doubling and jitter logics in Consul. It is certainly better then the ~45s delay observed without this change.

Note that the consul server imposes a lower limit of 10s on the session_ttl by default, which is applicable to the session_ttl in the lock options. In order to test this, consul server configuration should be overriding the session_ttl_min to a lower value.

@jefferai jefferai added this to the 0.10.1 milestone Apr 13, 2018
jefferai
jefferai previously approved these changes Apr 13, 2018
Copy link
Member

@jefferai jefferai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but can you check w/Consul team about whether your observed behavior is in fact expected?

One other thing: Please use parseutil.ParseDuration instead of time.ParseDuration as it better handles bare strings -- time.ParseDuration will consider it to be nanoseconds instead of seconds, which is likely not what the user expects.

@jefferai
Copy link
Member

Related to my other comment, there's somewhere else that time.ParseDuration is already used, please update that too!

@vishalnayak vishalnayak merged commit e6cc20d into master Apr 18, 2018
@vishalnayak vishalnayak deleted the consul-tunables branch April 18, 2018 17:10
vishalnayak added a commit that referenced this pull request Apr 18, 2018
* phys/consul: allow tuning of session ttl and lock wait time

* use parseutil

* udpate docs
@lulianbing58
Copy link

Fixes: #2949

Even after setting the session_ttl to "2s" and lock_wait_time to "2s", I am observing a 16s-18s delay in assuming active duty by the standby. I guess it is because of the doubling and jitter logics in Consul. It is certainly better then the ~45s delay observed without this change.

Note that the consul server imposes a lower limit of 10s on the session_ttl by default, which is applicable to the session_ttl in the lock options. In order to test this, consul server configuration should be overriding the session_ttl_min to a lower value.

I tried to set session_ttl_min =2s in consul configuration and in vault configuration, i set session_ttl=2s ,lock_wait_time=2s, but seems it still need 15s to swithover to standby.

anything more need to further tunning ?
Appreciate

@taitelman
Copy link

I also see 13seconds to standby taking leadership

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tunable Consul High Availability Lock Timeout Value
4 participants