Need a reliable and low latency local cluster setup for Kubernetes

[vishh]

Derived from kubernetes/kubernetes#24106

1. A new Kubernetes user should be able to setup a cluster locally in less than a minute.
2. A Kubernetes user should be able to test their application using the local cluster
3. For the features supported in the local cluster, environment in the local cluster must match that of production.

[sebgoa]

@vishh it would be nice to explain why the current ways of starting things locally don't meet those needs.

Like the local-cluster-up.sh script or the vagrant boxes.

[davidopp]

cc/ @mikedanese

[luxas]

Do you think we should run the all-in-one-binary inside a container?

I think so, or we should do something to cAdvisor that makes kubelet compile dynamically and therefore add a hard dependency on libc, which I see as very bad.

[vishh]

Do you think we should run the all-in-one-binary inside a container?

@luxas As you already know, there are quite a few issues that needs to be resolved for getting the all-in-one binary to run inside a container, quite similar to hyperkube. It does not matter to the end user how we run the core components. They interact with kubectl primarily. We also want to support alternate runtimes in the future, rkt for example, without requiring much change. Given that we are attempting to hide all the implementation details, we can totally switch to a container based model in the future, once it works, without breaking users. Thoughts?

@Runseb kubernetes/kubernetes#24106 should provide the motivation for attempting to improve the local cluster experience.

[luxas]

@vishh With all-in-one-binary I meant ~monokube, which runs etcd, kubelet, scheduler, apiserver, controller-manager, kube-proxy and maybe dns.

I think we should run the server of minikube inside a docker container.
If it does work directly on a linux host and is statically built (no libc dep) we should consider running out of a container.

[vishh]

On Mon, Apr 18, 2016 at 12:36 PM, Lucas Käldström [email protected]
wrote:

@vishh https://github.com/vishh With all-in-one-binary I meant ~monokube,
which runs etcd, kubelet, scheduler, apiserver, controller-manager,
kube-proxy and maybe dns.

I think we should run the server of minikube inside a docker container.

We have discussed in lengths as to why running in docker directly is
problematic in kubernetes/kubernetes#24106.

If it does work directly on a linux host and is statically built (no libc
dep) we should consider running out of a container.

IIUC, the main issue with static linking is cAdvisor. If we solve that
problem, we are safe I think.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1 (comment)

[luxas]

Yes, I would really want to know why cAdvisor forces kubelet to have libc

[rata]

@luxas if no DNS is deployed in the cluster, I don't think "2." in the first report of this bug is possible, as an application uses DNS for service discovery and without it it won't be possible to test many applications.

[luxas]

@rata I didn't mean dns should be optional, I just meant that dns maybe should be in the same binary.

[rata]

@luxas: oh, sorry! My bad :)

[dlorenc]

Going to close this out for now. I believe minikube accomplishes this :)