Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[helm] Eclipse Che fail to start when using self-signed-cert on k8s #16429

Closed
4 of 23 tasks
elavicount opened this issue Mar 24, 2020 · 20 comments
Closed
4 of 23 tasks

[helm] Eclipse Che fail to start when using self-signed-cert on k8s #16429

elavicount opened this issue Mar 24, 2020 · 20 comments
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/question Questions that haven't been identified as being feature requests or bugs. severity/P2 Has a minor but important impact to the usage or development of the system.

Comments

@elavicount
Copy link

Describe the bug

I'm trying to deploy Eclipse Che on a Kubernetes cluster using self-signed-cert and the helm installer, after following the doc the problem persists.

Che version

  • latest
  • nightly
  • other: please specify

Steps to reproduce

  1. Create a namespace:
    kubectl create namespace che

  2. Set environment variables:

/etc/profile.d/che.sh

export CA_CN=eclipse-che-signer
export DOMAIN="*.10.96.0.1.nip.io"
export OPENSSL_CNF=/etc/pki/tls/openssl.cnf
  1. Generate self signed certificate:
#!/bin/bash
echo "CA_CN: ${CA_CN}"
echo "Domain: ${DOMAIN}"
echo "OPENSSL_CNF: ${OPENSSL_CNF}"

openssl genrsa -out rootCA.key 4096 &&  \
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.crt -subj /CN=${CA_CN} -reqexts SAN -extensions SAN -config <(cat ${OPENSSL_CNF} <(printf '[SAN]\nbasicConstraints=critical, CA:TRUE\nkeyUsage=keyCertSign, cRLSign, digitalSignature, keyEncipherment')) && \
openssl genrsa -out domain.key 2048 && \
openssl req -new -sha256 -key domain.key -subj "/O=EclipseChe/CN=${DOMAIN}" -reqexts SAN -config <(cat ${OPENSSL_CNF} <(printf "\n[SAN]\nsubjectAltName=DNS:${DOMAIN}\nbasicConstraints=critical, CA:FALSE\nkeyUsage=keyCertSign, digitalSignature, keyEncipherment\nextendedKeyUsage=serverAuth")) -out domain.csr && \
openssl x509 -req -sha256 -extfile <(printf "subjectAltName=DNS:${DOMAIN}\nbasicConstraints=critical, CA:FALSE\nkeyUsage=keyCertSign, digitalSignature, keyEncipherment\nextendedKeyUsage=serverAuth") -days 365 -in domain.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out domain.crt
cp rootCA.crt ca.crt
  1. Create the secret key:
#!/bin/bash
kubectl delete secret che-tls -n che
kubectl create secret tls che-tls --key=domain.key --cert=domain.crt -n che
kubectl create secret generic self-signed-certificate --from-file=ca.crt -n che
  1. Execute the following command in a kubernetes cluster:

chectl server:start --platform=k8s --multiuser --domain=10.96.0.1.nip.io --installer=helm --postgres-pvc-storage-class-name=local-storage --chenamespace=che --tls --self-signed-cert

  1. Wait until che pod fails, this cause a CrashLoopBackOff

Expected behavior

Eclipse che should be deployed correctly

Runtime

  • kubernetes

Kubectl version

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
  • Openshift (include output of oc version)
  • minikube (include output of minikube version and kubectl version)
  • minishift (include output of minishift version and oc version)
  • docker-desktop + K8S (include output of docker version and kubectl version)
  • other: (please specify)

Screenshots

Installation method

  • chectl 
chectl version
chectl/7.10.0 linux-x64 node-v10.19.0

helm version
version.BuildInfo{Version:"v3.1.1", GitCommit:"afe70585407b420d0097d07b21c47dc511525ac8", GitTreeState:"clean", GoVersion:"go1.13.8"}
  • che-operator
  • minishift-addon
  • I don't know

Environment

  • my computer
    • Windows
    • Linux
    • macOS
  • Cloud
    • Amazon
    • Azure
    • GCE
    • other (please specify)
  • other: Centos 8

Eclipse Che Logs

Caused by: java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: https://keycloak-che.10.96.0.1.nip.io/auth/realms/che/.well-known/openid-configuration
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:104)
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings$$FastClassByGuice$$e0d0786b.newInstance(<generated>)
        at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.RealMapBinder$RealMapProvider.doProvision(RealMapBinder.java:796)
        at com.google.inject.internal.RealMapBinder$RealMapProvider.doProvision(RealMapBinder.java:733)
        at com.google.inject.internal.InternalProviderInstanceBindingImpl$Factory.get(InternalProviderInstanceBindingImpl.java:113)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211)
        at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
        at com.google.inject.Guice.createInjector(Guice.java:87)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:140)
        at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:45)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:85)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching keycloak-che.10.96.0.1.nip.io found.
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
        at java.net.URL.openStream(URL.java:1067)
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:97)
        ... 86 more

Additional context

kubectl get all -n che -o wide
NAME                                    READY   STATUS             RESTARTS   AGE   IP            NODE     NOMINATED NODE   READINESS GATES
pod/che-6595df56b8-rx72n                0/1     CrashLoopBackOff   9          37m   10.244.2.54   node-2   <none>           <none>
pod/devfile-registry-696845fc9f-2zhrb   1/1     Running            0          37m   10.244.2.53   node-2   <none>           <none>
pod/keycloak-5cdcb5c9f6-pq2pg           1/1     Running            0          37m   10.244.1.79   node-1   <none>           <none>
pod/plugin-registry-7b94d46db7-xzrh8    1/1     Running            0          37m   10.244.2.55   node-2   <none>           <none>
pod/postgres-7f7bbd9c6d-qdxf4           1/1     Running            0          37m   10.244.1.80   node-1   <none>           <none>

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE   SELECTOR
service/che-host           ClusterIP   10.109.12.206    <none>        8080/TCP,8087/TCP   37m   app=che,component=che
service/devfile-registry   ClusterIP   10.104.173.178   <none>        8080/TCP            37m   app=che,component=devfile-registry
service/keycloak           ClusterIP   10.97.216.36     <none>        5050/TCP            37m   io.kompose.service=keycloak
service/plugin-registry    ClusterIP   10.109.123.99    <none>        8080/TCP            37m   app=che,component=plugin-registry
service/postgres           ClusterIP   10.110.91.235    <none>        5432/TCP            37m   io.kompose.service=postgres

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS             IMAGES                                        SELECTOR
deployment.apps/che                0/1     1            0           37m   che                    quay.io/eclipse/che-server:7.10.0             app=che
deployment.apps/devfile-registry   1/1     1            1           37m   che-devfile-registry   quay.io/eclipse/che-devfile-registry:7.10.0   app=che,component=devfile-registry
deployment.apps/keycloak           1/1     1            1           37m   keycloak               quay.io/eclipse/che-keycloak:7.10.0           app=che,component=keycloak
deployment.apps/plugin-registry    1/1     1            1           37m   che-plugin-registry    quay.io/eclipse/che-plugin-registry:7.10.0    app=che,component=plugin-registry
deployment.apps/postgres           1/1     1            1           37m   postgres               quay.io/eclipse/che-postgres:7.10.0           app=che,component=postgres

NAME                                          DESIRED   CURRENT   READY   AGE   CONTAINERS             IMAGES                                        SELECTOR
replicaset.apps/che-6595df56b8                1         1         0       37m   che                    quay.io/eclipse/che-server:7.10.0             app=che,pod-template-hash=6595df56b8
replicaset.apps/devfile-registry-696845fc9f   1         1         1       37m   che-devfile-registry   quay.io/eclipse/che-devfile-registry:7.10.0   app=che,component=devfile-registry,pod-template-hash=696845fc9f
replicaset.apps/keycloak-5cdcb5c9f6           1         1         1       37m   keycloak               quay.io/eclipse/che-keycloak:7.10.0           app=che,component=keycloak,pod-template-hash=5cdcb5c9f6
replicaset.apps/plugin-registry-7b94d46db7    1         1         1       37m   che-plugin-registry    quay.io/eclipse/che-plugin-registry:7.10.0    app=che,component=plugin-registry,pod-template-hash=7b94d46db7
replicaset.apps/postgres-7f7bbd9c6d           1         1         1       37m   postgres               quay.io/eclipse/che-postgres:7.10.0           app=che,component=postgres,pod-template-hash=7f7bbd9c6d

 kubectl get all
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   3d23h

kubectl get ingress -n che
NAME               HOSTS                                   ADDRESS   PORTS     AGE
che-ingress        che-che.10.96.0.1.nip.io                          80, 443   45m
devfile-registry   devfile-registry-che.10.96.0.1.nip.io             80, 443   45m
keycloak-ingress   keycloak-che.10.96.0.1.nip.io                     80, 443   45m
plugin-registry    plugin-registry-che.10.96.0.1.nip.io              80, 443   45m

Possible duplicates #16396, #16280

Also could you please tell if there is any way to use a non nightly version, I need a stable version for production enviroment.
@elavicount elavicount added the kind/bug Outline of a bug - must adhere to the bug report template. label Mar 24, 2020
@che-bot che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Mar 24, 2020
@sleshchenko sleshchenko added the area/install Issues related to installation, including offline/air gap and initial setup label Mar 24, 2020
@tolusha
Copy link
Contributor

tolusha commented Mar 24, 2020

@eder-santos
The following doc [1] is relevant when you use operator installer.
In case of helm there are no manual steps. Just run the command below. tls will be enabled by default and self-sign certificate will be generated.

chectl server:start --platform=k8s --multiuser --domain=10.96.0.1.nip.io --installer=helm --postgres-pvc-storage-class-name=local-storage --chenamespace=che

[1] https://www.eclipse.org/che/docs/che-7/setup-che-in-tls-mode-with-self-signed-certificate/#deploy-che-with-self-signed-tls-on-kubernetes_setup-che-in-tls-mode-with-self-signed-certificate

@tolusha tolusha added kind/question Questions that haven't been identified as being feature requests or bugs. and removed kind/bug Outline of a bug - must adhere to the bug report template. status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. labels Mar 24, 2020
@elavicount
Copy link
Author

Hi @tolusha
I delete the whole namespace in order to destroy all resources and try again as with the command that you provide me, but unfortunately I got the same exception.

Caused by: java.lang.RuntimeException: Exception while retrieving OpenId configuration from endpoint: https://keycloak-che.10.96.0.1.nip.io/auth/realms/che/.well-known/openid-configuration
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:104)
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings$$FastClassByGuice$$e0d0786b.newInstance(<generated>)
        at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.SingleParameterInjector.inject(SingleParameterInjector.java:42)
        at com.google.inject.internal.SingleParameterInjector.getAll(SingleParameterInjector.java:65)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:113)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.FactoryProxy.get(FactoryProxy.java:62)
        at com.google.inject.internal.SingleFieldInjector.inject(SingleFieldInjector.java:52)
        at com.google.inject.internal.MembersInjectorImpl.injectMembers(MembersInjectorImpl.java:147)
        at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:124)
        at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
        at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
        at com.google.inject.internal.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:40)
        at com.google.inject.internal.SingletonScope$1.get(SingletonScope.java:168)
        at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
        at com.google.inject.internal.InternalInjectorCreator.loadEagerSingletons(InternalInjectorCreator.java:211)
        at com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:182)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:109)
        at com.google.inject.Guice.createInjector(Guice.java:87)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:140)
        at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:45)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:85)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4699)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5165)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:743)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:719)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:970)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1841)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative DNS name matching keycloak-che.10.96.0.1.nip.io found.
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268)
        at java.net.URL.openStream(URL.java:1067)
        at org.eclipse.che.multiuser.keycloak.server.KeycloakSettings.<init>(KeycloakSettings.java:97)
        ... 50 more
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching keycloak-che.10.96.0.1.nip.io found.
        at sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:214)
        at sun.security.util.HostnameChecker.match(HostnameChecker.java:96)
        at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:462)
        at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:428)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:209)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
        ... 64 more

These are the secrets generated by helm:

[root@master-node scripts]# kubectl get secret -n che
NAME                        TYPE                                  DATA   AGE
che-keycloak-token-gdwmf    kubernetes.io/service-account-token   3      5m46s
che-tls                     kubernetes.io/tls                     3      5m56s
che-token-dgxzl             kubernetes.io/service-account-token   3      5m46s
default-token-wpbzh         kubernetes.io/service-account-token   3      6m35s
sh.helm.release.v1.che.v1   helm.sh/release.v1                    1      5m46s
[root@master-node scripts]# kubectl describe secret che-tls -n che
Name:         che-tls
Namespace:    che
Labels:       <none>
Annotations:  cert-manager.io/alt-names: 10.96.0.1.nip.io,*.10.96.0.1.nip.io
              cert-manager.io/certificate-name: che-certificate
              cert-manager.io/common-name: *.10.96.0.1.nip.io
              cert-manager.io/ip-sans:
              cert-manager.io/issuer-kind: ClusterIssuer
              cert-manager.io/issuer-name: che-cluster-issuer
              cert-manager.io/uri-sans:

Type:  kubernetes.io/tls

Data
====
ca.crt:   1854 bytes
tls.crt:  1586 bytes
tls.key:  1679 bytes
[root@master-node scripts]# kubectl describe secret che-keycloak-token-gdwmf -n che
Name:         che-keycloak-token-gdwmf
Namespace:    che
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: che-keycloak
              kubernetes.io/service-account.uid: 3d802c5a-8b48-4a2f-bfc6-5227b67a8bbe

Type:  kubernetes.io/service-account-token

Data
====
namespace:  3 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IkhpbllIM2VTN1VFVUQzaE1uYzhoQTRHXzcwemtaVWZBWWgzV1NteFVFSTAifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJjaGUiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiY2hlLWtleWNsb2FrLXRva2VuLWdkd21mIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImNoZS1rZXljbG9hayIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjNkODAyYzVhLThiNDgtNGEyZi1iZmM2LTUyMjdiNjdhOGJiZSIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpjaGU6Y2hlLWtleWNsb2FrIn0.nuSKpA091q-YHVpwqnTFu56BtMbKCuRK1HcrO9jwCsgTmZ9r5JluNU3Wcabd9Aiqv1Z3dU6zpfodMdu_LCJeZeep2aEm1Dj-98K1JFBclbDkUG-NjgAnUme0NXlj_vT4zQQvF0XGaWcsdT3T8edbYnIijDplb1at6Qx32yWvN4WfSsXlY7UyAWzWlul9Qh1n09HDGvAmfVjhS6b_E4TCVneK5C725CXGHfLruQLx3zbhEp0mZKwwL-amlTeHK0kIiGr0utat2olcZodrHZuu7szaHXZf8elhJqJ-lWNOdjvvdPtar3atK5L8hkJwA-NpfBm4_7QcNXtZ3nyqdXhTng
ca.crt:     1025 bytes

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020

@eder-santos
kubectl get configmap che -n che -o=yaml | grep CHE_INFRA_KUBERNETES_TLS__SECRET

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020 •
edited
Loading

Also
kubectl describe ingress keycloak-ingress -n che

@tolusha tolusha added the status/info-needed More information is needed before the issue can move into the “analyzing” state for engineering. label Mar 25, 2020
@elavicount
Copy link
Author

Thanks @tolusha
Here is the additional data, let me know if you need more information. Thanks again

[root@master-node chectl-logs]# kubectl get configmap che -n che -o=yaml | grep CHE_INFRA_KUBERNETES_TLS__SECRET
  CHE_INFRA_KUBERNETES_TLS__SECRET: che-tls

[root@master-node chectl-logs]# kubectl describe ingress keycloak-ingress -n che
Name:             keycloak-ingress
Namespace:        che
Address:
Default backend:  default-http-backend:80 (<none>)
TLS:
  che-tls terminates keycloak-che.10.96.0.1.nip.io
Rules:
  Host                           Path  Backends
  ----                           ----  --------
  keycloak-che.10.96.0.1.nip.io
                                 /   keycloak:5050 (10.244.1.99:8080)
Annotations:
  kubernetes.io/ingress.class:                        nginx
  nginx.ingress.kubernetes.io/proxy-connect-timeout:  3600
  nginx.ingress.kubernetes.io/proxy-read-timeout:     3600
  nginx.ingress.kubernetes.io/ssl-redirect:           true
Events:

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020

kubectl get deployment che -n che -o=yaml

@elavicount
Copy link
Author

Thanks @tolusha, feel free to ask me any question.

kubectl get deployment che -n che -o=yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-03-25T15:09:29Z"
  generation: 1
  labels:
    app: che
    component: che
  name: che
  namespace: che
  resourceVersion: "1296122"
  selfLink: /apis/apps/v1/namespaces/che/deployments/che
  uid: 7445efd4-241c-4fab-a041-9ba898db8cb2
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: che
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: che
        component: che
    spec:
      containers:
      - env:
        - name: OPENSHIFT_KUBE_PING_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: CHE_INFRA_KUBERNETES_TLS__CERT
          valueFrom:
            secretKeyRef:
              key: tls.crt
              name: che-tls
              optional: false
        - name: CHE_INFRA_KUBERNETES_TLS__KEY
          valueFrom:
            secretKeyRef:
              key: tls.key
              name: che-tls
              optional: false
        envFrom:
        - configMapRef:
            name: che
        image: quay.io/eclipse/che-server:7.10.0
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /api/system/state
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 120
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 10
        name: che
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        - containerPort: 8000
          name: http-debug
          protocol: TCP
        - containerPort: 8888
          name: jgroups-ping
          protocol: TCP
        - containerPort: 8087
          name: http-metrics
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /api/system/state
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 60
        resources:
          limits:
            memory: 600Mi
          requests:
            memory: 256Mi
        securityContext:
          runAsUser: 1724
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      initContainers:
      - env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: ENDPOINT
          value: postgres
        image: quay.io/eclipse/che-endpoint-watcher:7.10.0
        imagePullPolicy: IfNotPresent
        name: wait-for-postgres
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: ENDPOINT
          value: keycloak
        image: quay.io/eclipse/che-endpoint-watcher:7.10.0
        imagePullPolicy: IfNotPresent
        name: wait-for-keycloak
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 1724
      serviceAccount: che
      serviceAccountName: che
      terminationGracePeriodSeconds: 30
status:
  conditions:
  - lastTransitionTime: "2020-03-25T15:09:29Z"
    lastUpdateTime: "2020-03-25T15:09:29Z"
    message: Deployment does not have minimum availability.
    reason: MinimumReplicasUnavailable
    status: "False"
    type: Available
  - lastTransitionTime: "2020-03-25T15:09:29Z"
    lastUpdateTime: "2020-03-25T15:09:29Z"
    message: ReplicaSet "che-8bf974698" is progressing.
    reason: ReplicaSetUpdated
    status: "True"
    type: Progressing
  observedGeneration: 1
  replicas: 1
  unavailableReplicas: 1
  updatedReplicas: 1

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020

I see
you don't have env var. But how you get it

 - name: CHE_SELF__SIGNED__CERT
          valueFrom:
            secretKeyRef:
              key: ca.crt
              name: che-tls
              optional: false

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020

Did you use --self-signed-cert flag ?

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020

So root cause is that before deploying Eclipse Che the che-tls secret existed in namespace che.

@tolusha
Copy link
Contributor

tolusha commented Mar 25, 2020 •
edited
Loading

I assume that che namespace was not vanished properly before second installation

@elavicount
Copy link
Author

Hi @tolusha
I remove the whole namespace the following way, but I'm still getting the same error.

kubectl delete secret che-tls -n che

kubectl delete secret self-signed-certificate -n che

kubectl delete secret self-signed-cert -n che

kubectl get pvc -n che | awk 'NR > 1 { print $1 }' | xargs -L 1 kubectl delete pvc $1 -n che

kubectl get pv -n che | awk 'NR > 1 { print $1 }' | xargs -L 1 kubectl delete pv $1 -n che

kubectl get storageclass -n che | awk 'NR > 1 { print $1 }' | xargs -L 1 kubectl delete storageclass $1 -n che

kubectl delete all --all -n che

kubectl delete namespace che

and then create again the namespace
kubectl create namespace che

I'm not using the --self-signed-cert option, but I notice the env var is created even without that option. This is my start command

chectl server:start --platform=k8s --multiuser --domain=10.96.0.1.nip.io --installer=helm --postgres-pvc-storage-class-name=local-storage --chenamespace=che

Let me know if you need more information, thank you

@tolusha
Copy link
Contributor

tolusha commented Mar 26, 2020

@eder-santos
I will try to reproduce locally.

@tolusha tolusha added severity/P2 Has a minor but important impact to the usage or development of the system. and removed status/info-needed More information is needed before the issue can move into the “analyzing” state for engineering. labels Mar 26, 2020
@tolusha
Copy link
Contributor

tolusha commented Mar 26, 2020

Meanwhile you can try to deploy on minikube [1] cluster.
[1] https://kubernetes.io/docs/tasks/tools/install-minikube/

@tolusha tolusha changed the title Eclipse Che fail to start when using self-signed-cert [helm] Eclipse Che fail to start when using self-signed-cert on k8s Mar 26, 2020
@elavicount
Copy link
Author

Thank you @tolusha I will try it

@mmorhun
Copy link
Contributor

mmorhun commented Apr 3, 2020

@eder-santos have you succeeded with any of the installation methods?

@elavicount
Copy link
Author

Hi @mmorhun
I try to run it with minikube but I'm unable to make it run properly in a CentOS Virtual Machine as I can't use the Kubernetes Cluster to install minikube. We are currently evaluating Eclipse Che as PaaS as we are unable to solve this problem running it on the cluster.

@mmorhun
Copy link
Contributor

mmorhun commented Apr 6, 2020

@eder-santos Minikube installation is designed to try it on single machine or for development purposes. The same I may say about CRC, but there are some tries to make it accessible outside. You may take a look at this one.

Can you try to deploy Che as you described in the issue description but using --installer=operator ?

@tolusha tolusha mentioned this issue Apr 13, 2020
Closed
17 tasks
@tolusha tolusha mentioned this issue Apr 21, 2020
Closed
50 tasks
@tolusha
Copy link
Contributor

tolusha commented Apr 24, 2020

@eder-santos
chectl version 7.10.0 was used.
It contains several bugs related to deploying che with tls.
pls update it and retry again
chectl update stable

@tolusha
Copy link
Contributor

tolusha commented Apr 28, 2020

@eder-santos
I close this issue since it was cause outdated version of chectl.

@tolusha tolusha closed this as completed Apr 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/install Issues related to installation, including offline/air gap and initial setup kind/question Questions that haven't been identified as being feature requests or bugs. severity/P2 Has a minor but important impact to the usage or development of the system.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@tolusha @sleshchenko @elavicount @mmorhun @che-bot