chectl server:start with minikube is failling

[sunix]

Describe the bug

Installing che with chectl on minikube (with default option) is failling.
Environment Linux and using the minikube driver kvm2

Here the steps:

• install che in minikube
  

• got stuck at this step (will eventually fail with timeout)
  

• running the describe command
  

• showing the error
  

[sunix]

Setting blocker as it is stopping new user onbording with Che ... first time installation.

[dmytro-ndp]

In Eclipse Che nightly tests the minikube has been started in no-driver mode:
minikube start --vm-driver=none --cpus 4 --memory 13000

[sunix]

@dmytro-ndp
ok but I don't have docker and don't want to install it.
Maybe all the commands and options you are running in the tests should be in the documentation as a recommanded options ....

[gattytto]

here's a workaround:
in /usr/local/lib/chectl/templates/che-operator/crds/org_v1_che_cr.yaml
MUST set:
tlsSupport: false
ingressDomain: 'your-local-ip.nip.io'

#16545 (comment)

[l0rd]

@mmorhun are you looking at this?

[mmorhun]

@sunix could you please provide minikube version ?

[sunix]

@mmorhun

$minikube version
minikube version: v1.11.0
commit: 57e2f55f47effe9ce396cea42a1e0eb4f611ebbd

[gattytto]

this will keep happening until:

1. temporary workaround is added to chectl consisted of:
   precreate storageclasses for postgres and workspaces and set them as settings in cr
   set tlsenabled: false as default instead of current :true in cr.yaml
   ingressDomain: is also needed in .cr, it does not get autofilled by chectl like before

2. minikube upstream issue is fixed storage provisioner in v1.9.0-beta.2: Object: readObjectFieldAsBytes: expect : after object field,  kubernetes/minikube#7218

[gattytto]

#16409

[gattytto]

with mentioned workarounds in #16545 (comment):

[mmorhun]

set tlsenabled: false as default

@gattytto could you please elaborate more on this. Why is this required?

[gattytto]

set tlsenabled: false as default

@gattytto could you please elaborate more on this. Why is this required?

last time I tried with its default value, selfSignedCert: false is a default and tlsEnabled: true is default at the same time, which breaks. #16409

[gattytto]

when setting up in macOS using minikube and chectl + che-operator, there is the need of "minikube ssh" and "su", then "mkdir -p /data/wksp" and "chmod -R 777 /data", then chectl start command from macOS will work (after using the storageclass and cr.yaml workaround)

[mmorhun]

@gattytto I think problem with TLS will be gone after #16764 is fixed.

[gattytto]

timeout is also another blocker in this situation, because most of the times if no images are present in the vm, timeout won't be enough unless a very fast pc and a very fast internet is being used. Because of this first attempt of starting che will fail, making it needed to cleanup the postgres /userdata and recreating the storageclasses and volumes.

[mmorhun]

Our team is going to handle this issue in the next sprint. We already have similar issues/requests.

[mmorhun]

@gattytto thank you for deep investigations.
As about timeouts, I think that should be another issue....

[gattytto]

I think imagepulls should not be counted as part of the timeout

[gattytto]

@mmorhun it seems like this is going to be fixed very soon upstream (check this commit as mentioned in kubernetes/minikube#7218 )

[mmorhun]

@gattytto I've seen priority/important-soon label, but thanks anyway for notifying.
I am working on workaround thought. It is useful in case if someone still wants to use Minikube from 1.9 to at least 1.11

[sunix]

@mmorhun why changing the severity?

[l0rd]

@mkuznyetsov @nickboldt please wait for a fix for this issue before releasing 7.15.0.

[mmorhun]

Fixed

[l0rd]

@flacatus @rhopp I was thinking about this and was wondering if we have any test for that. There were some e2e tests for chectl at some point, are they running? Aren't we testing minikube?

[sunix]

@l0rd I think it is tested with -vm-driver=none option (using the docker engine of the host)

[gattytto]

minikube defaults to docker driver which uses docker-machine and a premade .iso.
minikube can also be run using driver=none and "--network-plugin=cni --enable-default-cni --container-runtime=cri-o" so it won't require docker and cri-o can be used, providing the cni binaries in /opt/cni/bin.