Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include ContainerCreating in pod waiting status reasons #2063

Merged
merged 2 commits into from
Sep 19, 2018
Merged

Include ContainerCreating in pod waiting status reasons #2063

merged 2 commits into from
Sep 19, 2018

Conversation

deiwin
Copy link
Contributor

@deiwin deiwin commented Aug 16, 2018

What does this PR do?

Adds ContainerCreating to the list of whitelisted pod waiting statuses.

Motivation

I've seen different issues where the container can get stuck in the ContainerCreating status without moving to any of the other listed waiting reasons. For example, a bug in amazon-vpc-cni-k8s can cause a pod to get stuck in that status if the CNI plugin is unable to assign an IP for the pod. Whitelisting this status allows monitoring for these cases.

Review checklist

  • PR has a meaningful title or PR has the no-changelog label attached
  • Feature or bugfix has tests
  • Git history is clean
  • If PR impacts documentation, docs team has been notified or an issue has been opened on the documentation repo

Additional Notes

I originally raised the question of whitelisting this status in another PR.

With this reason added to the list, the list now contains all the possible reasons and is therefore unnecessary barring future addition of reasons. If it were up to me, I'd simply drop the whitelist check, but there seems to be some opposition to that.

I've seen different issues where the container can get stuck in the
ContainerCreating status without moving to any of the other listed waiting
reasons. For example, a bug in [amazon-vpc-cni-k8s][1] can cause a pod to get
stuck in that status if the CNI plugin is unable to assign an IP for the pod.
Whitelisting this status allows monitoring for these cases.

With this reason added to the list, the list now contains all the [possible
reasons][2] and is therefore unnecessary barring future addition of reasons. If
it were up to me, I'd simply drop the whitelist check, but there seems to be
[some opposition][3] to that.

[1]: https://github.com/aws/amazon-vpc-cni-k8s
[2]: https://github.com/kubernetes/kube-state-metrics/blob/bcb230560fe1a32810577762f0b91ad0a5f50d1a/Documentation/pod-metrics.md
[3]: #1763 (comment)
@BrianChristie
Copy link

We also have containers become hung in the ContainerCreating status for various reasons. This fix LGTM.

Is someone from DataDog able to review and merge this?

@deiwin
Copy link
Contributor Author

deiwin commented Sep 19, 2018

Please, it's a simple change.

Copy link
Contributor

@CharlyF CharlyF left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay - LGTM

@masci masci changed the title Include ContainerCreating in k8s pod waiting status reasons Include ContainerCreating in pod waiting status reasons Sep 19, 2018
@masci masci merged commit 82dc17d into DataDog:master Sep 19, 2018
@deiwin
Copy link
Contributor Author

deiwin commented Sep 20, 2018

Thank you!

@deiwin deiwin deleted the add_waiting_reason branch September 20, 2018 05:31
nmuesch pushed a commit that referenced this pull request Nov 1, 2018
Include ContainerCreating in pod waiting status reasons
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants