Implement follow_only flag #1263
Conversation
|
@miekg In what scenario has this helped you so far? I worry that a cluster will degrade to only those nodes that are --follow-only and the cluster will mysteriously stop making any progress. |
|
[ Quoting [email protected] in "Re: [fleet] Implement follow_only f..." ]
We basically have two type of nodes in our clusters: worker and admin nodes. The
So, this is why I'm also adding metrics to fleet. Without admin nodes our /Miek Miek Gieben |
|
@miekg ok, that makes sense. Could you start a "scaling" document that helps people understand how to use these features to scale out fleet? |
| @@ -58,7 +58,7 @@ func New(reg *registry.EtcdRegistry, lManager lease.Manager, rStream pkg.EventSt | |||
| } | |||
| } | |||
|
|
|||
| func (e *Engine) Run(ival time.Duration, stop chan bool) { | |||
| func (e *Engine) Run(ival time.Duration, follower bool, stop chan bool) { | |||
There was a problem hiding this comment.
Shouldn't the Engine simply not run if follow-only == true?
|
[ Quoting [email protected] in "Re: [fleet] Implement follow_only f..." ]
Ok, will do. |
|
Ok, reworked PR to not start the scheduler at all and renamed the flag to |
|
Friendly ping @bcwaldon |
|
@mischief, can you take a look at this? This change is crucial in making fleet scale to ~1000s of machines. |
|
Dear fleet devs :) @jonboulle @mischief ... can we merge this as well? |
|
What made you choose "scheduler" rather than "engine"? |
|
wrt the naming in the PR? Wasn't I conscious choice, felt more natural to use 'scheduler', but 'engine' might be better describing what it does code wise. Or maybe 'no_master' . Happy to make rename it to whatever you like best. |
|
Oh, I see, @bcwaldon originally suggested it @ #1263 (comment) . Personally I would find |
|
I have no problem making that change. |
|
I think I'd like to see Other than that, the implementation seems fine, fairly straightforward. Thanks! |
If set this fleetd will never assume mastership in a fleet cluster. This will cut down on the interactions with etcd. Useful when when you have two type of nodes in our clusters: worker and admin nodes. The worker nodes come and go (in large numbers) and these should just be used to schedule jobs. We never want them to become leaders. The admin nodes control the entire cluster and we just want one of those to become the fleet leader. Add documentation and detail the consequences this can have.
|
Ok. Change to --disable-engine. Added docs, squashed the commits. |
|
Thanks! LGTM. |
|
Cool! Thank you.
|
ghost
mentioned this pull request
If set this fleetd will never assume mastership in a fleet cluster.
This will cut down on the interactions with Etcd.
Note: this PR is rather hackish, it would be nice to know if something like will be considered for merging (if cleaned up, etc .etc.). IOW: comments welcome!
Fix #1166