Implemented recovery functionality for Kubernetes/OpenShift infrastructures #9301

sleshchenko · 2018-03-30T12:21:54Z

What does this PR do?

Implements recovery functionality for Kubernetes/OpenShift infrastructures.

It's done by usage of persistent cache for storing Kuberentes/OpenShift runtimes states including machine and servers information.

Jpa is used as persistent cache backend. To make it possible to have two Running Che Server (like during Rolling Update) it is needed to synchronize JPA L2 cache with JGroups, it is done in #8982.

What issues does this PR fix or reference?

#5919

Release Notes

Implemented recovery functionality for Kubernetes/OpenShift infrastructures.

Docs PR

N/A

skabashnyuk · 2018-03-30T13:15:24Z

infrastructures/kubernetes/pom.xml

+        <dependency>
+            <groupId>org.eclipse.persistence</groupId>
+            <artifactId>javax.persistence</artifactId>
+            <scope>provided</scope>


remove provied if not needed.

skabashnyuk · 2018-03-30T13:19:44Z

wsmaster/che-core-sql-schema/src/main/resources/che-schema/6.4.0/2__add_k8s_metadata.sql

+    PRIMARY KEY (workspace_id)
+);
+-- indexes
+CREATE UNIQUE INDEX index_k8s_runtime_workspace_id ON che_k8s_runtime (workspace_id);


You already have an index on workspace_id. Instead, you need foreign kay on workspace table.

skabashnyuk · 2018-03-30T13:19:56Z

wsmaster/che-core-sql-schema/src/main/resources/che-schema/6.4.0/2__add_k8s_metadata.sql

+
+-- Runtime ---------------------------------------------------------------------
+CREATE TABLE che_k8s_runtime (
+    workspace_id    VARCHAR(255)    NOT NULL,


workspace_id uniuque

It is a primary key. Are you sure that I must specify unique here? I checked other places like workspace, stack tables, id field is not marked as unique.

skabashnyuk · 2018-03-30T13:21:31Z

wsmaster/che-core-sql-schema/src/main/resources/che-schema/6.4.0/2__add_k8s_metadata.sql

+    machine_name    VARCHAR(255),
+    attribute_key   VARCHAR(255),
+    attribute       VARCHAR(255)
+);


index on workspace_id, machine_name

I planned to do it a bit later ;)

skabashnyuk · 2018-03-30T13:22:31Z

wsmaster/che-core-sql-schema/src/main/resources/che-schema/6.4.0/2__add_k8s_metadata.sql

+    server_name     VARCHAR(255),
+    attribute_key   VARCHAR(255),
+    attribute       VARCHAR(255)
+);


index workspace_id, machine_name, server_name

sleshchenko · 2018-04-04T14:33:38Z

ci-test

codenvy-ci · 2018-04-04T14:38:37Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:9301
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

codenvy-ci · 2018-04-06T16:08:25Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:9301
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

skabashnyuk · 2018-04-10T13:56:33Z

...clipse/che/workspace/infrastructure/kubernetes/cache/jpa/JpaKubernetesRuntimeStateCache.java

+    try {
+      return managerProvider
+          .get()
+          .createQuery("SELECT r FROM KubernetesRuntime r", KubernetesRuntimeState.class)


can we use named query here?

Good catch. Replaced with a named query.

skabashnyuk · 2018-04-10T13:56:42Z

...org/eclipse/che/workspace/infrastructure/kubernetes/cache/jpa/JpaKubernetesMachineCache.java

+      return managerProvider
+          .get()
+          .createQuery(
+              "SELECT m FROM KubernetesMachine m WHERE m.machineId.workspaceId = :workspaceId",


can we use named query here?

sleshchenko · 2018-04-11T13:48:27Z

ci-test

akorneta

good job 👍

garagatyi

From what I understand looks good. But PR is too big to provide a really thoughtful review.
BTW I was not sure about adding WS servers readiness probes on each Che master but @akorneta said that it is not supposed to be used on a two working Che masters, but rather during the swap of Che master instances.

garagatyi · 2018-04-11T13:55:51Z

...c/main/java/org/eclipse/che/workspace/infrastructure/openshift/project/OpenShiftProject.java

+  }
+
+  public OpenShiftProject(
+      OpenShiftClientFactory clientFactory, String name, String workspaceId, boolean doPrepare)


Can you add a description to the doPrepare parameter and constructors?

Replaced doPrepare parameter with package private method that should be invoked by project factory.

garagatyi · 2018-04-11T13:56:39Z

...java/org/eclipse/che/workspace/infrastructure/openshift/project/OpenShiftProjectFactory.java

@@ -44,4 +44,9 @@ public OpenShiftProject create(String workspaceId) throws InfrastructureExceptio
    final String projectName = isNullOrEmpty(this.projectName) ? workspaceId : this.projectName;
    return new OpenShiftProject(clientFactory, projectName, workspaceId);
  }
+
+  public OpenShiftProject create(String workspaceId, String projectName)


Can you add javadocs with clarification why 2 methods exist?

codenvy-ci · 2018-04-11T18:59:58Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:9301
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

sleshchenko · 2018-04-12T08:53:41Z

BTW I was not sure about adding WS servers readiness probes on each Che master but @akorneta said that it is not supposed to be used on a two working Che masters, but rather during the swap of Che master instances.

@garagatyi I have thought about avoiding WS servers readiness probes on each Che master and there is not an easy way to do that. Since original issue that should be solved now is Hot Update, I decided that it's OK for now to leave it. So, when we will have to solve the issue with multiple Che Master (replications) then this issue definitely should be investigated more.

sleshchenko · 2018-04-12T09:30:31Z

ci-test

…caches

… caches

codenvy-ci · 2018-04-12T15:33:07Z

ci-test build report:
Build details
Test report
selenium tests report data
docker image: eclipseche/che-server:9301
https://github.com/orgs/eclipse/teams/eclipse-che-qa please check this report.

sleshchenko added kind/enhancement A feature request - must adhere to the feature request template. status/in-progress This issue has been taken by an engineer and is under active development. labels Mar 30, 2018

sleshchenko self-assigned this Mar 30, 2018

sleshchenko requested a review from a user March 30, 2018 12:21

sleshchenko requested review from garagatyi, l0rd, riuvshin and skabashnyuk as code owners March 30, 2018 12:21

sleshchenko force-pushed the openshiftRuntimesCache branch 2 times, most recently from aca3dc1 to de5f41f Compare March 30, 2018 12:40

skabashnyuk reviewed Mar 30, 2018

View reviewed changes

sleshchenko force-pushed the openshiftRuntimesCache branch 14 times, most recently from 422cf79 to e6e35cf Compare April 4, 2018 08:44

sleshchenko requested a review from vparfonov as a code owner April 4, 2018 11:23

sleshchenko mentioned this pull request Apr 5, 2018

Improve abstract InternalRuntime to be more flexible for recovery functionality #9345

Merged

sleshchenko force-pushed the openshiftRuntimesCache branch from b23fc8a to 27a3dca Compare April 5, 2018 14:35

sleshchenko force-pushed the openshiftRuntimesCache branch from 989af77 to 7f46175 Compare April 10, 2018 08:02

sleshchenko changed the title ~~[WIP] Implemented recovery functionality for Kubernetes/OpenShift infrastructures~~ Implemented recovery functionality for Kubernetes/OpenShift infrastructures Apr 10, 2018

sleshchenko added status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. and removed status/in-progress This issue has been taken by an engineer and is under active development. labels Apr 10, 2018

sleshchenko force-pushed the openshiftRuntimesCache branch 2 times, most recently from 756253b to 276e1d5 Compare April 10, 2018 13:40

skabashnyuk reviewed Apr 10, 2018

View reviewed changes

sleshchenko force-pushed the openshiftRuntimesCache branch from b415c03 to 08bc586 Compare April 11, 2018 07:06

skabashnyuk approved these changes Apr 11, 2018

View reviewed changes

sleshchenko force-pushed the openshiftRuntimesCache branch 4 times, most recently from 48d8874 to 25a8bb7 Compare April 11, 2018 13:31

akorneta approved these changes Apr 11, 2018

View reviewed changes

garagatyi approved these changes Apr 11, 2018

View reviewed changes

sleshchenko force-pushed the openshiftRuntimesCache branch from 25a8bb7 to 9c82289 Compare April 12, 2018 07:34

sleshchenko added 3 commits April 12, 2018 17:23

CHE-5919 Add Kubernetes/OpenShift runtime state cache

7627ea4

CHE-5919 Add JPA Implementations for K8s/OpenShift Runtimes/Machines …

5f3c135

…caches

CHE-5919 Fix existing and add new tests for changes related to K8s/OS…

8bcef4e

… caches

sleshchenko force-pushed the openshiftRuntimesCache branch from 048dd86 to 8bcef4e Compare April 12, 2018 14:23

sleshchenko merged commit 7b4adc6 into eclipse-che:master Apr 12, 2018

sleshchenko deleted the openshiftRuntimesCache branch April 12, 2018 15:35

sleshchenko removed the status/code-review This issue has a pull request posted for it and is awaiting code review completion by the community. label Apr 12, 2018

benoitf added this to the 6.4.0 milestone Apr 12, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implemented recovery functionality for Kubernetes/OpenShift infrastructures #9301

Implemented recovery functionality for Kubernetes/OpenShift infrastructures #9301

sleshchenko commented Mar 30, 2018 •

edited

Loading

skabashnyuk Mar 30, 2018

skabashnyuk Mar 30, 2018

skabashnyuk Mar 30, 2018

sleshchenko Mar 30, 2018

skabashnyuk Apr 2, 2018

skabashnyuk Mar 30, 2018

sleshchenko Mar 30, 2018 •

edited

Loading

sleshchenko Mar 30, 2018

skabashnyuk Mar 30, 2018

sleshchenko Mar 30, 2018

sleshchenko commented Apr 4, 2018

codenvy-ci commented Apr 4, 2018

codenvy-ci commented Apr 6, 2018

skabashnyuk Apr 10, 2018

sleshchenko Apr 10, 2018

skabashnyuk Apr 10, 2018

sleshchenko commented Apr 11, 2018

akorneta left a comment

garagatyi left a comment

garagatyi Apr 11, 2018

sleshchenko Apr 12, 2018

garagatyi Apr 11, 2018

sleshchenko Apr 12, 2018

codenvy-ci commented Apr 11, 2018

sleshchenko commented Apr 12, 2018

sleshchenko commented Apr 12, 2018

codenvy-ci commented Apr 12, 2018

Implemented recovery functionality for Kubernetes/OpenShift infrastructures #9301

Implemented recovery functionality for Kubernetes/OpenShift infrastructures #9301

Conversation

sleshchenko commented Mar 30, 2018 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

Release Notes

Docs PR

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sleshchenko Mar 30, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sleshchenko commented Apr 4, 2018

codenvy-ci commented Apr 4, 2018

codenvy-ci commented Apr 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sleshchenko commented Apr 11, 2018

akorneta left a comment

Choose a reason for hiding this comment

garagatyi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codenvy-ci commented Apr 11, 2018

sleshchenko commented Apr 12, 2018

sleshchenko commented Apr 12, 2018

codenvy-ci commented Apr 12, 2018

sleshchenko commented Mar 30, 2018 •

edited

Loading

sleshchenko Mar 30, 2018 •

edited

Loading