Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resize verb #1

Closed
wants to merge 2 commits into from
Closed

Resize verb #1

wants to merge 2 commits into from

Conversation

pweil-
Copy link
Owner

@pweil- pweil- commented Dec 17, 2014

Making a local PR to make sure I'm not way off track with this.

This is an attempt to address: kubernetes#1629

Note: this is doing relative resizing right now, if it is on the right track I will add absolute resizing as noted in the comments of kubernetes#1629.

@pweil-
Copy link
Owner Author

pweil- commented Dec 17, 2014

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?

@derekwaynecarr
Copy link

I don't like the pattern here because it makes it too easy to circumvent admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a resizable interface. That way you get an object, call resize on object, and then ideally find a way to go through normal update pattern in rest handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.

@pweil-
Copy link
Owner Author

pweil- commented Dec 18, 2014

@derekwaynecarr that would be awesome. If we can discuss the appropriate implementation I'd love to have this ready for a formal PR before the break.

@smarterclayton
Copy link

We don't want to put interfaces on value objects really

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a resizable interface. That way you get an object, call resize on object, and then ideally find a way to go through normal update pattern in rest handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.

@smarterclayton
Copy link

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a resizable interface. That way you get an object, call resize on object, and then ideally find a way to go through normal update pattern in rest handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.

@derekwaynecarr
Copy link

I rash on my interface on resource. What I really don't want is multiple code paths to do an update above rest storage tier. So if we introduce a resize function, that is cool, but it should go through the same code path that is ultimately shared by rest handler update. If that means get the object, call a resizer that knows what field to modify, and then call the same code flow as rest handler update, then I am fine. Hope that makes sense.

Sent from my iPhone

On Dec 18, 2014, at 9:42 AM, Clayton Coleman [email protected] wrote:

We don't want to put interfaces on value objects really

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a resizable interface. That way you get an object, call resize on object, and then ideally find a way to go through normal update pattern in rest handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.

@derekwaynecarr
Copy link

A resize is an update, it should go through the update code path on APIServer/RESTStorage.

There may be some special logic to do before it goes through that code path, but it needs to go through a common code path at APIServer tier.

Implementation of a resize is done by some intelligent bit of code that knows to

  1. foo = RESTStorage.GET(name)
  2. foo.size = newsize
  3. call APIServer.Update(foo)
    a -> which calls RESTStorage.Update(object)

I basically want to avoid introducing code paths for updates that are not going through the same update flow.

If resize is implemented in a pattern as above, I am fine.

Thanks,
Derek
----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 9:44:24 AM
Subject: Re: [kubernetes] Resize verb (#1)

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a resizable interface. That way you get an object, call resize on object, and then ideally find a way to go through normal update pattern in rest handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67494736

@smarterclayton
Copy link

I don't know that I agree - let's have the discussion in the other issue though.

----- Original Message -----

A resize is an update, it should go through the update code path on
APIServer/RESTStorage.

There may be some special logic to do before it goes through that code path,
but it needs to go through a common code path at APIServer tier.

Implementation of a resize is done by some intelligent bit of code that knows
to

  1. foo = RESTStorage.GET(name)
  2. foo.size = newsize
  3. call APIServer.Update(foo)
    a -> which calls RESTStorage.Update(object)

I basically want to avoid introducing code paths for updates that are not
going through the same update flow.

If resize is implemented in a pattern as above, I am fine.

Thanks,
Derek
----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 9:44:24 AM
Subject: Re: [kubernetes] Resize verb (#1)

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent
admission control / quota (which I will link to my changeset tomorrow am).

I would at least prefer a pattern where a runtime.Object implemented a
resizable interface. That way you get an object, call resize on object,
and then ideally find a way to go through normal update pattern in rest
handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that
over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure
I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67494736


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67495555

@smarterclayton
Copy link

Breaking my own suggestion - the ability to resize should be limited to different people than the ability to update. I should be able to resize without being able to change the other attributes of a replication controller.

----- Original Message -----

I don't know that I agree - let's have the discussion in the other issue
though.

----- Original Message -----

A resize is an update, it should go through the update code path on
APIServer/RESTStorage.

There may be some special logic to do before it goes through that code
path,
but it needs to go through a common code path at APIServer tier.

Implementation of a resize is done by some intelligent bit of code that
knows
to

  1. foo = RESTStorage.GET(name)
  2. foo.size = newsize
  3. call APIServer.Update(foo)
    a -> which calls RESTStorage.Update(object)

I basically want to avoid introducing code paths for updates that are not
going through the same update flow.

If resize is implemented in a pattern as above, I am fine.

Thanks,
Derek
----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 9:44:24 AM
Subject: Re: [kubernetes] Resize verb (#1)

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent
admission control / quota (which I will link to my changeset tomorrow
am).

I would at least prefer a pattern where a runtime.Object implemented a
resizable interface. That way you get an object, call resize on object,
and then ideally find a way to go through normal update pattern in rest
handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that
over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure
I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67494736


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67495555

@derekwaynecarr
Copy link

I am fine with that.

I am just saying that the code path after the policy check (which you are handling diff with new verb), needs to go through same Update step internally.

We should not start introducing 10k new ways things can update. An update is an update is an update. We should introduce alternatives on how to do targeted updates that get around certain policy checks if needed, but internally, an update is an update is an update ;-)

----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 10:22:45 AM
Subject: Re: [kubernetes] Resize verb (#1)

Breaking my own suggestion - the ability to resize should be limited to different people than the ability to update. I should be able to resize without being able to change the other attributes of a replication controller.

----- Original Message -----

I don't know that I agree - let's have the discussion in the other issue
though.

----- Original Message -----

A resize is an update, it should go through the update code path on
APIServer/RESTStorage.

There may be some special logic to do before it goes through that code
path,
but it needs to go through a common code path at APIServer tier.

Implementation of a resize is done by some intelligent bit of code that
knows
to

  1. foo = RESTStorage.GET(name)
  2. foo.size = newsize
  3. call APIServer.Update(foo)
    a -> which calls RESTStorage.Update(object)

I basically want to avoid introducing code paths for updates that are not
going through the same update flow.

If resize is implemented in a pattern as above, I am fine.

Thanks,
Derek
----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 9:44:24 AM
Subject: Re: [kubernetes] Resize verb (#1)

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected] wrote:

I don't like the pattern here because it makes it too easy to circumvent
admission control / quota (which I will link to my changeset tomorrow
am).

I would at least prefer a pattern where a runtime.Object implemented a
resizable interface. That way you get an object, call resize on object,
and then ideally find a way to go through normal update pattern in rest
handler.

If you have 15 minutes tomorrow would like to synch on why I prefer that
over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make sure
I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67494736


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67495555


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67500692

@smarterclayton
Copy link

I think resize is a special case of update, which means policy should require that new verbs either reduce to generic behavior (watch is get, resize is update), or we try to cram them into one verb but with field specific behavior (??? yuck ???)

----- Original Message -----

I am fine with that.

I am just saying that the code path after the policy check (which you are
handling diff with new verb), needs to go through same Update step
internally.

We should not start introducing 10k new ways things can update. An update is
an update is an update. We should introduce alternatives on how to do
targeted updates that get around certain policy checks if needed, but
internally, an update is an update is an update ;-)

----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 10:22:45 AM
Subject: Re: [kubernetes] Resize verb (#1)

Breaking my own suggestion - the ability to resize should be limited to
different people than the ability to update. I should be able to resize
without being able to change the other attributes of a replication
controller.

----- Original Message -----

I don't know that I agree - let's have the discussion in the other issue
though.

----- Original Message -----

A resize is an update, it should go through the update code path on
APIServer/RESTStorage.

There may be some special logic to do before it goes through that code
path,
but it needs to go through a common code path at APIServer tier.

Implementation of a resize is done by some intelligent bit of code that
knows
to

  1. foo = RESTStorage.GET(name)
  2. foo.size = newsize
  3. call APIServer.Update(foo)
    a -> which calls RESTStorage.Update(object)

I basically want to avoid introducing code paths for updates that are not
going through the same update flow.

If resize is implemented in a pattern as above, I am fine.

Thanks,
Derek
----- Original Message -----
From: "Clayton Coleman" [email protected]
To: "pweil-/kubernetes" [email protected]
Cc: "Derek Carr" [email protected]
Sent: Thursday, December 18, 2014 9:44:24 AM
Subject: Re: [kubernetes] Resize verb (#1)

To clarify, resize is a verb and an optional interface on rest storage

On Dec 17, 2014, at 7:10 PM, Derek Carr [email protected]
wrote:

I don't like the pattern here because it makes it too easy to
circumvent
admission control / quota (which I will link to my changeset tomorrow
am).

I would at least prefer a pattern where a runtime.Object implemented a
resizable interface. That way you get an object, call resize on object,
and then ideally find a way to go through normal update pattern in rest
handler.

If you have 15 minutes tomorrow would like to synch on why I prefer
that
over what is here, demo how I am enforcing quota, and go from there?

Sent from my iPhone

On Dec 17, 2014, at 2:41 PM, Paul [email protected] wrote:

@derekwaynecarr @smarterclayton - do one of you have time to make
sure
I'm not going off in the wrong direction with this?


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67494736


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67495555


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67500692


Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67501705

@pweil-
Copy link
Owner Author

pweil- commented Dec 18, 2014

Another suggestion for implementation that eliminates the interface and instead uses a helper function that would go through the api server. The new resize verb handler and other code paths can then use the helper. The helper would be responsible for policy checking in this scenario:

 func Resize (obj, newSize) {         
      //cast to metadata interface
      //check policy
      //switch on the object type
      //call a function that knows how to deal with the specific object type
 }

 func resizeReplicationController(obj, newSize){
      //shouldn't be called directly or policy check needs to occur here
      //set the replica count
      //call apiserver.Update
 }

@smarterclayton
Copy link

I don't know why we would have helpers. The logic for resize is totally encapsulated inside the RESTStorage or the command client, and we already have a construct for generic resize from a client side (which deals with REST verbs, not code).

The generic resize is RESTHelper.Resize(namespace, name, -1), which tries to POST to ////op/resize, and fails if it doesn't exist. That's why it has to be a generic verb, since the POST structure has to be the same across multiple resource types.

----- Original Message -----

Another suggestion for implementation that eliminates the interface and
instead uses a helper function that would go through the api server. The
new resize verb handler and other code paths can then use the helper. The
helper would be responsible for policy checking in this scenario:

 func Resize (obj, newSize) {
      //cast to metadata interface
      //check policy
      //switch on the object type
      //call a function that knows how to deal with the specific object
      type
 }

 func ResizeReplicationController(obj, newSize){
      //set the replica count
      //call apiserver.Update
 }

Reply to this email directly or view it on GitHub:
https://github.com/pweil-/kubernetes/pull/1#issuecomment-67502469

pweil- pushed a commit that referenced this pull request Apr 24, 2015
pweil- pushed a commit that referenced this pull request May 1, 2015
Change cadvisor port and point kubelet to find it
pweil- pushed a commit that referenced this pull request May 13, 2015
We found in that someone just copied/pasted the boilerplate language into
their code. But the boilerplate contains 2014, not 2015. We have 2 ways
to fix this.

1) Update the boilerplate to 2015 so people would get the right one.
2) Update the boilerplate so it doesn't make sense and then warn when
people use it.

This PR takes the second option. While options #1 seems easier, it will
get wrong in 2016, 17, 18 and it's unlikely anyone remember why they
need to update the boilerplate text and the regex rewrite. So just
make the humans do a tiny bit more work now.
pweil- pushed a commit that referenced this pull request May 27, 2015
pweil- pushed a commit that referenced this pull request Sep 2, 2015
Because code.google.com is going away. Probably #1 of several.
@pweil- pweil- closed this Jan 25, 2016
pweil- pushed a commit that referenced this pull request Jun 21, 2016
Automatic merge from submit-queue

in e2e test, when kubectl exec fails to find the container to run a command, it should retry

fix kubernetes#26076 
Without retrying upon "container not found" error, `Pod Disks` test failed on the following error:
```console
[k8s.io] Pod Disks 
  should schedule a pod w/two RW PDs both mounted to one container, write to PD, verify contents, delete pod, recreate pod, verify contents, and repeat in rapid succession [Slow]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/pd.go:271
[BeforeEach] [k8s.io] Pod Disks
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:108
STEP: Creating a kubernetes client
May 23 19:18:02.254: INFO: >>> TestContext.KubeConfig: /root/.kube/config

STEP: Building a namespace api object
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [k8s.io] Pod Disks
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/pd.go:69
[It] should schedule a pod w/two RW PDs both mounted to one container, write to PD, verify contents, delete pod, recreate pod, verify contents, and repeat in rapid succession [Slow]
  /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/pd.go:271
STEP: creating PD1
May 23 19:18:06.678: INFO: Successfully created a new PD: "rootfs-e2e-11dd5f5b-211b-11e6-a3ff-b8ca3a62792c".
STEP: creating PD2
May 23 19:18:11.216: INFO: Successfully created a new PD: "rootfs-e2e-141f062d-211b-11e6-a3ff-b8ca3a62792c".
May 23 19:18:11.216: INFO: PD Read/Writer Iteration #0
STEP: submitting host0Pod to kubernetes
W0523 19:18:11.279910    4984 request.go:347] Field selector: v1 - pods - metadata.name - pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c: need to check if this is versioned correctly.
STEP: writing a file in the container
May 23 19:18:39.088: INFO: Running '/srv/dev/kubernetes/_output/dockerized/bin/linux/amd64/kubectl kubectl --server=https://130.211.199.187 --kubeconfig=/root/.kube/config exec --namespace=e2e-tests-pod-disks-3t3g8 pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '1394466581702052925' > '/testpd1/tracker0''
May 23 19:18:40.250: INFO: Wrote value: "1394466581702052925" to PD1 ("rootfs-e2e-11dd5f5b-211b-11e6-a3ff-b8ca3a62792c") from pod "pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c" container "mycontainer"
STEP: writing a file in the container
May 23 19:18:40.251: INFO: Running '/srv/dev/kubernetes/_output/dockerized/bin/linux/amd64/kubectl kubectl --server=https://130.211.199.187 --kubeconfig=/root/.kube/config exec --namespace=e2e-tests-pod-disks-3t3g8 pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '1740704063962701662' > '/testpd2/tracker0''
May 23 19:18:41.433: INFO: Wrote value: "1740704063962701662" to PD2 ("rootfs-e2e-141f062d-211b-11e6-a3ff-b8ca3a62792c") from pod "pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c" container "mycontainer"
STEP: reading a file in the container
May 23 19:18:41.433: INFO: Running '/srv/dev/kubernetes/_output/dockerized/bin/linux/amd64/kubectl kubectl --server=https://130.211.199.187 --kubeconfig=/root/.kube/config exec --namespace=e2e-tests-pod-disks-3t3g8 pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
May 23 19:18:42.585: INFO: Read file "/testpd1/tracker0" with content: 1394466581702052925

STEP: reading a file in the container
May 23 19:18:42.585: INFO: Running '/srv/dev/kubernetes/_output/dockerized/bin/linux/amd64/kubectl kubectl --server=https://130.211.199.187 --kubeconfig=/root/.kube/config exec --namespace=e2e-tests-pod-disks-3t3g8 pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
May 23 19:18:43.779: INFO: Read file "/testpd2/tracker0" with content: 1740704063962701662

STEP: deleting host0Pod
May 23 19:18:44.048: INFO: PD Read/Writer Iteration #1
STEP: submitting host0Pod to kubernetes
W0523 19:18:44.132475    4984 request.go:347] Field selector: v1 - pods - metadata.name - pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c: need to check if this is versioned correctly.
STEP: reading a file in the container
May 23 19:18:45.186: INFO: Running '/srv/dev/kubernetes/_output/dockerized/bin/linux/amd64/kubectl kubectl --server=https://130.211.199.187 --kubeconfig=/root/.kube/config exec --namespace=e2e-tests-pod-disks-3t3g8 pd-test-16d3653c-211b-11e6-a3ff-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
May 23 19:18:46.290: INFO: error running kubectl exec to read file: exit status 1
stdout=
stderr=error: error executing remote command: error executing command in container: container not found ("mycontainer")
)
May 23 19:18:46.290: INFO: Error reading file: exit status 1
May 23 19:18:46.290: INFO: Unexpected error occurred: exit status 1
```
Now I've run this fix on e2e pd test 5 times and no longer see any failure
pweil- pushed a commit that referenced this pull request Jun 21, 2016
Automatic merge from submit-queue

in each pd test, create and delete the pod for every iteration to give new pod name for exec

fix kubernetes#26141
based on chat with @ncdc

The following is a snapshot of the log. Each iteration now has a new Pod name
```text
[It] should schedule a pod w/two RW PDs both mounted to one container, write to PD, verify contents, delete pod, recreate pod, verify contents, and repeat in rapid succession [Slow] [Flaky]
  /srv/dev/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/pd.go:277
STEP: creating PD1
Jun 10 15:55:45.878: INFO: Successfully created a new PD: "rootfs-e2e-c8b82df9-2f23-11e6-a5a0-b8ca3a62792c".
STEP: creating PD2
Jun 10 15:55:49.794: INFO: Successfully created a new PD: "rootfs-e2e-cb135362-2f23-11e6-a5a0-b8ca3a62792c".
Jun 10 15:55:49.794: INFO: PD Read/Writer Iteration #0
STEP: submitting host0Pod to kubernetes
W0610 15:55:49.860308   17282 request.go:347] Field selector: v1 - pods - metadata.name - pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c: need to check if this is versioned correctly.
STEP: writing a file in the container
Jun 10 15:56:09.792: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '988876932586416926' > '/testpd1/tracker0''
Jun 10 15:56:12.003: INFO: Wrote value: "988876932586416926" to PD1 ("rootfs-e2e-c8b82df9-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: writing a file in the container
Jun 10 15:56:12.003: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '8414937992264649637' > '/testpd2/tracker0''
Jun 10 15:56:13.170: INFO: Wrote value: "8414937992264649637" to PD2 ("rootfs-e2e-cb135362-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: reading a file in the container
Jun 10 15:56:13.170: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
Jun 10 15:56:14.325: INFO: Read file "/testpd1/tracker0" with content: 988876932586416926

STEP: reading a file in the container
Jun 10 15:56:14.325: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-cd68f34b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
Jun 10 15:56:15.590: INFO: Read file "/testpd2/tracker0" with content: 8414937992264649637

STEP: deleting host0Pod
Jun 10 15:56:15.841: INFO: PD Read/Writer Iteration #1
STEP: submitting host0Pod to kubernetes
W0610 15:56:15.905485   17282 request.go:347] Field selector: v1 - pods - metadata.name - pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c: need to check if this is versioned correctly.
STEP: reading a file in the container
Jun 10 15:56:16.832: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
Jun 10 15:56:18.132: INFO: Read file "/testpd1/tracker0" with content: 988876932586416926

STEP: reading a file in the container
Jun 10 15:56:18.132: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
Jun 10 15:56:19.354: INFO: Read file "/testpd2/tracker0" with content: 8414937992264649637

STEP: writing a file in the container
Jun 10 15:56:19.354: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '7639503234625274799' > '/testpd1/tracker1''
Jun 10 15:56:20.526: INFO: Wrote value: "7639503234625274799" to PD1 ("rootfs-e2e-c8b82df9-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: writing a file in the container
Jun 10 15:56:20.526: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '7400445987108171911' > '/testpd2/tracker1''
Jun 10 15:56:21.694: INFO: Wrote value: "7400445987108171911" to PD2 ("rootfs-e2e-cb135362-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: reading a file in the container
Jun 10 15:56:21.694: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
Jun 10 15:56:22.904: INFO: Read file "/testpd1/tracker0" with content: 988876932586416926

STEP: reading a file in the container
Jun 10 15:56:22.905: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
Jun 10 15:56:24.080: INFO: Read file "/testpd2/tracker0" with content: 8414937992264649637

STEP: reading a file in the container
Jun 10 15:56:24.081: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker1'
Jun 10 15:56:25.290: INFO: Read file "/testpd1/tracker1" with content: 7639503234625274799

STEP: reading a file in the container
Jun 10 15:56:25.290: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-dcef71e1-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker1'
Jun 10 15:56:26.491: INFO: Read file "/testpd2/tracker1" with content: 7400445987108171911

STEP: deleting host0Pod
Jun 10 15:56:26.756: INFO: PD Read/Writer Iteration #2
STEP: submitting host0Pod to kubernetes
W0610 15:56:26.821828   17282 request.go:347] Field selector: v1 - pods - metadata.name - pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c: need to check if this is versioned correctly.
STEP: reading a file in the container
Jun 10 15:56:27.898: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker1'
Jun 10 15:56:29.096: INFO: Read file "/testpd1/tracker1" with content: 7639503234625274799

STEP: reading a file in the container
Jun 10 15:56:29.096: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker1'
Jun 10 15:56:30.325: INFO: Read file "/testpd2/tracker1" with content: 7400445987108171911

STEP: reading a file in the container
Jun 10 15:56:30.325: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
Jun 10 15:56:31.528: INFO: Read file "/testpd1/tracker0" with content: 988876932586416926

STEP: reading a file in the container
Jun 10 15:56:31.529: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
Jun 10 15:56:32.972: INFO: Read file "/testpd2/tracker0" with content: 8414937992264649637

STEP: writing a file in the container
Jun 10 15:56:32.972: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '1846555975530999997' > '/testpd1/tracker2''
Jun 10 15:56:34.157: INFO: Wrote value: "1846555975530999997" to PD1 ("rootfs-e2e-c8b82df9-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: writing a file in the container
Jun 10 15:56:34.157: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- /bin/sh -c echo '2775947264799611726' > '/testpd2/tracker2''
Jun 10 15:56:35.661: INFO: Wrote value: "2775947264799611726" to PD2 ("rootfs-e2e-cb135362-2f23-11e6-a5a0-b8ca3a62792c") from pod "pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c" container "mycontainer"
STEP: reading a file in the container
Jun 10 15:56:35.662: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker0'
Jun 10 15:56:36.868: INFO: Read file "/testpd1/tracker0" with content: 988876932586416926

STEP: reading a file in the container
Jun 10 15:56:36.868: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker0'
Jun 10 15:56:38.062: INFO: Read file "/testpd2/tracker0" with content: 8414937992264649637

STEP: reading a file in the container
Jun 10 15:56:38.062: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker1'
Jun 10 15:56:39.221: INFO: Read file "/testpd1/tracker1" with content: 7639503234625274799

STEP: reading a file in the container
Jun 10 15:56:39.221: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker1'
Jun 10 15:56:40.397: INFO: Read file "/testpd2/tracker1" with content: 7400445987108171911

STEP: reading a file in the container
Jun 10 15:56:40.397: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd1/tracker2'
Jun 10 15:56:41.584: INFO: Read file "/testpd1/tracker2" with content: 1846555975530999997

STEP: reading a file in the container
Jun 10 15:56:41.585: INFO: Running '/srv/dev/kubernetes/_output/local/bin/linux/amd64/kubectl exec --namespace=e2e-tests-pod-disks-2tvm2 pd-test-e370dd2b-2f23-11e6-a5a0-b8ca3a62792c -c=mycontainer -- cat /testpd2/tracker2'
Jun 10 15:56:42.800: INFO: Read file "/testpd2/tracker2" with content: 2775947264799611726

STEP: deleting host0Pod
```

@saad-ali
pweil- pushed a commit that referenced this pull request Jan 19, 2017
Automatic merge from submit-queue

Add rule for detecting exceptions to fluentd config for GKE logging (#1)

**What this PR does / why we need it**:
Add the [fluent-detect-exceptions-plugin](https://rubygems.org/gems/fluent-plugin-detect-exceptions) version 0.0.4 to the fluentd config for Kubernetes clusters running on Google Cloud. This plugin detects exception stacks in the stdout/stderr log streams of the containers that run in the cluster and makes sure that they are forwarded as a single log entry to Stackdriver Logging.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
fluentd config for GKE clusters updated: detect exceptions in container log streams and forward them as one log entry.
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants