kops: support for bare-metal

[justinsb]

Feature Description

• One-line feature description (can be used as a release note): kops supports targeting bare metal (or non-cloudprovider) machines
• Primary contact (assignee): justinsb
• Responsible SIGs: sig-cluster-lifecycle
• Design proposal link (community repo):
• Reviewer(s) - (for LGTM) recommend having 2+ reviewers (at least one from code-area OWNERS file) agreed to review. Reviewers from multiple companies preferred:
• Approver (likely from SIG/area to which feature belongs):
• Feature target (which target equals to which milestone):
  • Alpha release target 1.8
  • Beta release target 1.9
  • Stable release target 1.a

[idvoretskyi]

@justinsb @kubernetes/sig-cluster-lifecycle-feature-requests @kubernetes/sig-aws-misc any updates for 1.8? Is this feature still on track for the release?

[rtnpro]

@justinsb Any update on it? I am ready to lend a helping hand :)

[ieugen]

Hi,

First of all I think that having automatic support for bare metal provisioning is an awesome goal.
I have some questions that might spark a discussion around what features are in scope and what not.

Are you targeting a full kubernetes cluster install on bare-metal?
Could kops support a use case of adding a bare-metal worker node to an existing kubernetes AWS cluster?
I believe that this might be good way to expand an existing cluster outside AWS as servers in classic providers as Hetzner can be cheaper. I don't know if the costs will be mitigated by the data transfer costs.

If this feature is out of scope for automatic provisioning by kops, maybe the procedure can be described so it can be done manually.

[mtaylor769]

I have spent the last two to three weeks trying to get elasticsearch to run on a kubernetes cluster and have lots of R&D to offer with regards to LXD, conjure-up vs. kubeadm, namespace management, and the mish-mosh of intertwined commands between kubectl, docker and juju to get things almost running (everything says it's healthy, but can't actually get a response from ES). Let me know if you're interested.

[ieugen]

@mtaylor769 : If your message was for me, I managed to deploy single cluster node with kubeadm and it worked very well. Thanks.

The cluster works ok but I have some issues with networking. I'm trying to secure the cluster by making services listen on localhost by default. Seems like I managed. Still I do have some unknowns in that area and more documentation would be helpful.

Regarding bare-metal, I believe kubeadm is muck more helpful at this moment.

[mtaylor769]

@ieugen It was a general question, but your answer has shed some light on some of what I've been experiencing. I was trying kubeadm at first but wasn't able to get everything to play nice, so I will try that avenue again. Thanks.

[luxas]

@ieugen @mtaylor769 Just FYI, this repo is meant to be low-volume, and email notifications are expected to be feature status updates, not general discussion or support. That discussion should move to other forums, but no worries anyway.

Thanks for your interest in kubeadm and kops!

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[shapeofarchitect]

@ieugen so I have got the same thing , I just believe kubeadm is appreaing to be the best solution for BareMetal considering it's single node control plane method and scalability by just adding a tokenized string , I tried to use kubespray as well but it seems there is a drift I felt when tried to apply for new nodes.

Not sure what's the opinion but I would love to see views or experience for the same.

[KlavsKlavsen]

Where did this end up? Did you all go some other way than KOPS? using kubeadm for manual roll up on bare-metal instead, or ?

[olemarkus]

As kOps maintainer, I can at least say that there is no active project to support bare-metal anymore. And the code we once had going in that direction was obsolete and removed.
I think it is unlikely that we will support anything that doesn't come with a form of cloud-like infrastructure (e.g we do support OpenStack)

[coopbri]

@KlavsKlavsen you might already know about this project, but I've been using kubespray for multi-platform cluster deployment, including bare metal. As @olemarkus mentioned, you can instead use kops with OpenStack if you'd prefer to still use kops.

[KlavsKlavsen]

@coopbri @olemarkus I looked at openstack- but I can find absolute no documentation of how to setup openstack (except for some devel hack - that only sets everything up on your laptop) :(
It would be a fine solution, to have an openstack setup (for so few nodes - it should run just fine on 1 server - or in a few lxd's on one server).

It seems to not have gained community acceptance - since no one is writing about it.. or am just missing the right keywords to search for?

[mtaylor769]

I have a repo that I used for bare metal Kube clusters that uses the CentOS kernel for Docker. I will post a link to it here later.

…

On Sun, Mar 7, 2021 at 11:35 PM Klavs Klavsen ***@***.***> wrote: @coopbri <https://github.com/coopbri> @olemarkus <https://github.com/olemarkus> I looked at openstack- but I can find absolute no documentation of how to setup openstack (except for some devel hack - that only sets everything up on your laptop) :( It would be a fine solution, to have an openstack setup (for so few nodes - it should run just fine on 1 server - or in a few lxd's on one server). It seems to not have gained community acceptance - since no one is writing about it.. or am just missing the right keywords to search for? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#360 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHGYSYAADFSILAEMF5C3OLTCR44VANCNFSM4DUQJ4HA> .

[td-harness]

What a bummer I was very excited reading this until the end, almost sounded like the perfect solution.