round 2

[pweil-]

No description provided.

[pmorie]

What are the units of 'Value'?

[pweil-]

Value is always considered a float in this case. My assumption is that all stats boil down to a number that can be compared against.

[pweil-]

To be more specific in the answer, I don't think that we care what the units are and that we leave it up to the statistics gather mechanism to publish how it stores stats (ms vs seconds, etc) and up to the user to define the correct value that matches the units.

[pweil-]

@smarterclayton please review prior to me merging into the main PR to make sure I'm on track with the feedback. Thanks.

[smarterclayton]

This is going to be somewhat problematic - how do you know which auto-scalers target you? How do you transactionally guarantee that? It also means every component that creates and manipulates replication controllers also has to know about auto-scalers. That's not necessarily a blocking issue, but it does create complexity when it comes to meaning. It's possible that the deployment process itself is the thing that should be taking the target of the autoscaler (via the resize verb), rather than the underlying individual replication controllers. In the OpenShift case, we have a deployment target (the deployment config) which can support resize. In the current Kube code, kubectl has no target, so it would have to talk to the controller instead.

It seems like the deployer really needs a way to communicate to the autoscaler that is tied to the replication controllers directly. If you had that, you could mark the replication controllers semantically as you went through a deployment. However, that has some issues too.

Basically, kubectl having to know about auto-scalers to do its job feels wrong. We need to establish whether it's impossible to keep kubectl isolated from the auto-scaler, or whether we just need a better abstraction (some combination of a marker on the RC + the autoscaler selector).

[pweil-]

Ok, so I think I've digested this. My thoughts are, we can solve the issue of knowing what autoscalers target a resizeable object by adding a registration component to the interface. When an autoscaler is initialized it can attempt to register itself with its target.

Transactionally guaranteeing the state of "which scalers target me" seems harder. Since autoscalers can be updated by external components (api calls and deployments) I don't think we can guarantee that the scalers registered to component A are still targeting component A at any point in time.

We could, perhaps, separate the decision making of the autoscaler from something that calls the resize verb. In that use case, the autoscaler would take two object references, one for the expected target of the resize (the replication controller) and one for the resizer object. Any action the autoscaler wants to take for the target produces an event with the semantics "resizer A: take action B on target C". If resizer A no longer targets target C the event is dropped.

We essentially end up with a big circle of communication in the case of 'hey replication controller D, retarget your auto scalers to E'. Repl controller D iterates through its registered autoscalers and calls a method that emits an event of "resizer A: take retarget action, target E if you currently target C". This design implies that operations within the resizer must be atomic.

Finally, I'm not sure where kubectl comes in to play. The use case that stands out to me (which is not currently called out in the proposal) is a scenario where manual changing of the replication controller conflicts with the autoscaler and how policy deals with that. Can you elaborate on what you were thinking there?

/cc @pmorie

[smarterclayton]

On Feb 11, 2015, at 5:43 PM, Paul [email protected] wrote:

In docs/proposals/autoscaling.md:

@@ -197,6 +178,34 @@ resolves to a value that can be checked against a configured threshold.
Of note: If the statistics gathering mechanisms can be initialized with a registry other components storing statistics can
potentially piggyback on this registry.

+### Interactions with a deployment
+
+In a deployment it is likely that multiple replication controllers must be monitored. For instance, in a rolling deployment
+there will be multiple replication controllers, with one scaling up and another scaling down. This means that an
+auto-scaler must be aware of the entire set of capacity that backs a service so it does not fight with the deployer. AutoScalerSpec.Selector
+is what provides this ability. By using a selector that spans the entire service the auto-scaler can monitor capacity
+of multiple replication controllers and check that capacity against the AutoScalerSpec.MaxAutoScaleCount and
+AutoScalerSpec.MinAutoScaleCount while still only targeting a specific ReplicationController.
+
+In the course of a rolling update it would be up to the deployment orchestration to re-target the auto-scaler.
Ok, so I think I've digested this. My thoughts are, we can solve the issue of knowing what autoscalers target a resizeable object by adding a registration component to the interface. When an autoscaler is initialized it can attempt to register itself with its target.

Transactionally guaranteeing the state of "which scalers target me" seems harder. Since autoscalers can be updated by external components (api calls and deployments) I don't think we can guarantee that the scalers registered to component A are still targeting component A at any point in time.

We could, perhaps, separate the decision making of the autoscaler from something that calls the resize verb. In that use case, the autoscaler would take two object references, one for the expected target of the resize (the replication controller) and one for the resizer object. Any action the autoscaler wants to take for the target produces an event with the semantics "resizer A: take action B on target C". If resizer A no longer targets target C the event is dropped.

We essentially end up with a big circle of communication in the case of 'hey replication controller D, retarget your auto scalers to E'. Repl controller D iterates through its registered autoscalers and calls a method that emits an event of "resizer A: take retarget action, target E if you currently target C". This design implies that operations within the resizer must be atomic.

The two references is reasonable. For deployment configs, we don't have a problem (one target, which hides the fact there are multiple items). For rolling update of rcs, the target has to remain fixed, but maybe we allow the count to be done via selector. Then it's the rolling updates responsibility to handle swapping out the target. I.e the rolling updater first creates a copy of the current rc, but with no label selector and size 0. Then it clears the label
Selector of the existing one (or disables it somehow) and updates the old one to have the right pattern. We may not have the primitives we need though.

We need to be having this discussion on the existing pull - can you update and summarize this thread?

Finally, I'm not sure where kubectl comes in to play. The use case that stands out to me (which is not currently called out in the proposal) is a scenario where manual changing of the replication controller conflicts with the autoscaler and how policy deals with that. Can you elaborate on what you were thinking there?

/cc @pmorie

—
Reply to this email directly or view it on GitHub.

[pweil-]

Will do, thanks for the initial feedback

[pweil-]

closed, adding to main proposal