Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some points to use in production #174

Open
gabrieladt opened this issue May 11, 2017 · 0 comments
Open

Some points to use in production #174

gabrieladt opened this issue May 11, 2017 · 0 comments

Comments

@gabrieladt
Copy link

gabrieladt commented May 11, 2017

Hi. This is my first post , I don't know if I am doing right, sorry case not.
So I am using a fork of tack in my productions envs. I would like to share some import points for us that I found.

1 - Disable auto updates(coreos).
Today coreos and kubernetes are not synchronized this means the nodes are rebooted and you have errors (until the health checks fails and the pods are reallocated, also in ingress) or downtime.
coreos/bugs#1274
units:
- name: update-engine.service
mask: true
- name: locksmithd.service
mask: true
Or try use this guy(I need to test too)
https://github.com/coreos/container-linux-update-operator (ref on bug above)

2- Very Important, cause a lot of issues.
Is normal and happen a lot "coredumps". The problem is, the default action of systemd is
get the dump and compress, when this happen the CPU of the machine fully consumed, causing the degradation of everything that is running in the same node, this was the main cause of our first downtime on k8s,"some coredumps were generated by some bad threads, make the cpu achieve 100%, making the health check fails for all other pods and making the pods be killed, etc,etc.

So basically we disable coredump to be save to the disk (only log)
/etc/systemd/coredump.conf
Storage=none

Running

  • hyperkube-tag = "v1.5.4_coreos.0"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant