Help accessing Eclipse Che with Minikube and KVM on external machine

[gnoejuan]

Summary

---

tl;dr

This is my setup, what is the proper way of using Minikube (for KVM acceleration)? Or perhaps is Minikube the wrong approach? Should I deploy a different solution for k8s? I'm open to installing a new operating system on the k8s machine (such as k3os or any other k8s focused OS). I had already backed up all my data.

---

Things I've tried

• Follow the K8s tutorial here: Exposing an External IP Address to Access an Application in a Cluster kubectl --namespace che expose deployment/che --type="LoadBalancer " --name=che-service

• Leads to an External IP always staying pending, so I followed the advice here: EXTERNAL-IP is always none and unable to reach to Service from Host machine #723 kubernetes/minikube#3966 (comment) and used `minikube tunnel'. This gave another internal IP address, although it was 10.x.x.x

• Following KVM: Creating a bridged network with NetPlan on Ubuntu bionic, I appear to now be faced with minikube being unable to resolve k8s

! Node may be unable to resolve external DNS records ! VM is unable to access k8s.gcr.io, you may need to configure a proxy or set --image-repository
chectl doesn't thow any errors, but I'm not seeing any bandwidth activity.

I was able to quickly understand Docker, but with k8s, I feel like I'm way in over my head.

Relevant information

[ibuziuk]

@tolusha will you be able to take a look at that one?

[tolusha]

That's very interesting task, but unfortunately I don't have an appropriate environment.
May be someone else have already faced that.

[ibuziuk]

@l0rd @davidfestal @benoitf maybe you can take a look at this setup and suggest smth

[gnoejuan]

At this point, KVM acceleration isn't important anymore. I believe the most productive way is to focus on a remote bare metal install. Example environments would be an AWS EC2 instance or Vultr which doesn't appear to offer a managed K8s solution.

I would still use the setup pictured above, but I hope a tutorial that applies to a hosted bare metal server (without managed k8s) would also apply to a local machine.

[skabashnyuk]

@gnoejuan would you like to try with crc(local openshift 4 ) there is a guide for it https://gist.github.com/tmckayus/8e843f90c44ac841d0673434c7de0c6a ?

[l0rd]

We use minikube running on a remote machine for our e2e tests. Maybe @rhopp @dmytro-ndp have some suggestions.

[ultrafab]

Hello @gnoejuan, i've just finish to setup something i think similar to your needs, running completely inside a private network.

This is the abstract, let me know if i can give you more informations:

• Create a KVM Virtual Machine with Ubuntu 18.04 server. (I've done cause i've had an already setup KVM host, but i think you can skip and use a bare 18.04 installation on a phisical machine).
• Installing Openshift 3.11 on the virtual (or phisical) machine (https://www.techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04/)
• Configure a dnsmasq service on another network server (in my case i've installed it on the KVM Host machine hosting the VM with Openshift), and configure it to resolve a local domain (your-che-domain.lan) pointing to the Openshift machine IP. It's important that you setup a wildcard domain, this way it will be able to resolve also the dynamic subdomains created by Che.
• THIS IS IMPORTANT (without it everything keeps failing as documented here: How to run Che on Openshift 3.11 in a local network #16417) create a static /etc/resolv.conf file insider your Openshift VM, pointing to the IP of the dnsmasq host. This way Openshift, when starting, will pass your private DNS/Domain settings to Openshift.
• Add your dnsmasq machine ip to the DNS list of every client that needs to connect to Che.

Let me know if i can provide more insight.

[gnoejuan]

@gnoejuan would you like to try with crc(local openshift 4 ) there is a guide for it gist.github.com/tmckayus/8e843f90c44ac841d0673434c7de0c6a ?

For this one, I'm stuck at AttributeError: 'NoneType' object has no attribute 'typeattributes'

But it looks like it should have been fixed here: SELinuxProject/selinux#81 (comment)

Hello @gnoejuan, i've just finish to setup something i think similar to your needs, running completely inside a private network.

This is the abstract, let me know if i can give you more informations:

* Create a KVM Virtual Machine with Ubuntu 18.04 server. (I've done cause i've had an already setup KVM host, but i think you can skip and use a bare 18.04 installation on a phisical machine).

* Installing Openshift 3.11 on the virtual (or phisical) machine ([techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04](https://www.techrepublic.com/article/how-to-install-openshift-origin-on-ubuntu-18-04/))

* Configure a dnsmasq service on another network server (in my case i've installed it on the KVM Host machine hosting the VM with Openshift), and configure it to resolve a local domain (your-che-domain.lan) pointing to the Openshift machine IP. It's important that you setup a wildcard domain, this way it will be able to resolve also the dynamic subdomains created by Che.

* THIS IS IMPORTANT (without it everything keeps failing as documented here: #16417) create a static `/etc/resolv.conf` file insider your Openshift VM, pointing to the IP of the dnsmasq host. This way Openshift, when starting, will pass your private DNS/Domain settings to Openshift.

* Add your dnsmasq machine ip to the DNS list of every client that needs to connect to Che.

Let me know if i can provide more insight.

Here, I get so close. It had stopped right at "Starting Che." It times out, and gives a directory for the logs /tmp/chexxxxxxx. When I navigate to the dir, it doesn't exist.

I'm curious if this is a problem with installing on Ubuntu 18.04.4 LTS for both solutions. I am working on downloading Ubuntu 16.04.6 LTS

[renannprado]

@gnoejuan could you get around the AttributeError: 'NoneType' object has no attribute 'typeattributes' issue?

[gnoejuan]

@gnoejuan could you get around the AttributeError: 'NoneType' object has no attribute 'typeattributes' issue?

I was unable to get around the issue. It has been awhile since I was tackling CRC method to deploy a k8s cluster. My recollection was the recommended Ubuntu version at the time was lower than 18.04, but the KVM version that came with Ubuntu 16 was too old. Researching how to upgrade KVM only returned results on installing QEMU/KVM.

Ubuntu 18.04 LTS or newer and Debian 10 or newer are not officially supported and may require manual set up of the host machine.

https://code-ready.github.io/crc/

After trying RancherOS (Supposedly due to be obsoleted rancher/os#3000 (comment)), k3OS never seemed to resolve an IP address, Fedora's CoreOS required learning another technology called Ignition. . . .

After researching Kubernetes more thoroughly, it appears the fastest/easiest path to a local Kubernetes server is to use kubectl.

Personally, I chose to use microk8s on the k8s machine pictured above.

Microk8s guide: https://microk8s.io/docs/working-with-kubectl

tl;dr: on k8s machine: microk8s config and copy and paste or microk8s config > config and find a way to retrieve the file to your client machine.

Assuming you don't have a .kube in your home directory on the client machine. I.E the machine you want to access Kubernetes from.

cd $HOME
mkdir .kube
cd .kube

put the config file from earlier in the .kube folder

chectl server:start -p k8s -a helm -b mainhub.lan

where mainhub.lan is your k8s machine.

Minikube guide: https://medium.com/@kuljeetsinghkhurana/access-minikube-using-kubectl-from-remote-machine-2b0eeefad9cb

In theory, chectl server:start -p k8s -a helm -b mainhub.lan should work here too, but I haven't tested it.

Although this experience has dragged me into the world of Kubernetes kicking and screaming, I enjoyed the learning experience.

[emircanerkul]

➜  minikube chectl server:start -p k8s -a helm -b mainhub.lan
Error: Nonexistent flags: -p, -a, -b

I'm also looking for answer;
#22989