-
Notifications
You must be signed in to change notification settings - Fork 28
7. Kubernetes
- Kubectl CLI installed
- Be connected to a running Kubernetes instance
- Public container registry where to push/pull images
- Kind using LoadBalancer: https://kind.sigs.k8s.io/docs/user/loadbalancer/ (the framework will expose services using LoadBalancer. This is not configurable yet.)
Required Maven dependency:
<dependency>
<groupId>io.quarkus.qe</groupId>
<artifactId>quarkus-test-kubernetes</artifactId>
<scope>test</scope>
</dependency>And now, we can write also scenarios to be run in Kubernetes by adding the @KubernetesScenario annotation:
@KubernetesScenario
public class KubernetesPingPongResourceIT {
@QuarkusApplication
static final RestService app = new RestService();
@Test
public void shouldPingPongWorks() {
app.given().get("/ping").then().statusCode(HttpStatus.SC_OK).body(is("ping"));
app.given().get("/pong").then().statusCode(HttpStatus.SC_OK).body(is("pong"));
}
}We can reuse all the scenarios to be run on Baremetal and Kubernetes by extending scenarios, for example having written the scenario on baremetal:
@QuarkusScenario
public class PingPongResourceIT {
@QuarkusApplication
static final RestService pingPongApp = new RestService();
@Test
public void shouldPingPongWorks() {
pingpong.given().get("/ping").then().statusCode(HttpStatus.SC_OK).body(is("ping"));
pingpong.given().get("/pong").then().statusCode(HttpStatus.SC_OK).body(is("pong"));
}
}We can create a new scenario called KubernetesPingPongResourceIT as:
@KubernetesScenario
public class KubernetesPingPongResourceIT extends PingPongResourceIT {
}Kubernetes needs a container registry where to push and pull images, so we need to provide a property like:
mvn clean verify -Dts.container.registry-url=quay.io/<your username>
The container registry must automatically expose the containers publicly.
These tests can be disabled if the above system property is not set using the @DisabledIfNotContainerRegistry annotation.
The same applies for all the deployment strategies available:
This strategy will build the image locally and push it to an intermediary container registry (provided by a system property). Then, the image will be pulled from the container registry into Kubernetes.
@KubernetesScenario(deployment = KubernetesDeploymentStrategy.UsingContainerRegistry)
public class KubernetesPingPongResourceIT {
// ...
}This strategy will delegate the deployment into the Quarkus Kubernetes extension, so it will trigger a Maven command to run it.
Example:
@KubernetesScenario(deployment = KubernetesDeploymentStrategy.UsingKubernetesExtension)
public class KubernetesPingPongResourceIT {
// ...
}In order to use this strategy, you need to add this Maven profile into the pom.xml:
<profile>
<id>deploy-to-kubernetes-using-extension</id>
<dependencies>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-container-image-jib</artifactId>
</dependency>
</dependencies>
</profile>| Important note: This strategy does not support custom sources to be selected, this means that the whole Maven module will be deployed. Therefore, if we have:
Take into account that we have used the quarkus-container-image-jib dependency to build the Quarkus image, but we can use any from https://quarkus.io/guides/container-image.
@KubernetesScenario(deployment = KubernetesDeploymentStrategy.UsingKubernetesExtension)
public class KubernetesPingPongResourceIT {
@QuarkusApplication(classes = PingResource.class)
static final RestService pingPongApp = new RestService();
// ...
}The test case will fail saying that this is not supported using the Using Kubernetes extension strategy.
We can inject the Kubectl client to interact with Kubernetes. This can be useful to cope with more complex scenarios like scale up/down services.
import io.quarkus.test.bootstrap.inject.KubectlClient;
import io.quarkus.test.scenarios.KubernetesScenario;
@KubernetesScenario
public class KubernetesGreetingResourceIT extends GreetingResourceIT {
@Test
public void shouldInjectKubectlClient(KubectlClient client) {
// ...
client.scaleTo(app, 2);
}
}Another option is by injecting the client directly to the test class using the @Inject annotation:
import io.quarkus.test.bootstrap.inject.KubectlClient;
import io.quarkus.test.scenarios.KubernetesScenario;
@KubernetesScenario
public class KubernetesGreetingResourceIT extends GreetingResourceIT {
@Inject
static KubectlClient client;
// ...
}| Note that the injection is only supported to static fields.
The framework will create a namespace for every scenario and delete it once the scenario is finished. However, sometimes due to cluster restrictions we're not allowed to create projects. In those cases, you can disable ephemeral namespaces and run all your tests in your current namespace.
-Dts.kubernetes.ephemeral.namespaces.enabled=false
Sometimes deploying a third party into Kubernetes involves some complex configuration that is not required when deploying on bare metal. For these scenarios, we allow to provide a custom template via test.properties:
ts.consul.kubernetes.template=/yourtemplate.yaml
| And the custom template must contain ONLY ONE deployment (for kubernetes).
Moreover, if the service that is exposing the port we want to target is named differently to our service, we can provide the service name via:
ts.consul.kubernetes.service=consul-http-service
What about if we want to use the internal service as route (not the exposed route), we can set this behaviour by enabling the property ts.<MY_SERVICE>.kubernetes.use-internal-service-as-url:
ts.consul.kubernetes.use-internal-service-as-url=true
By default, the framework will always delete the Kubernetes namespace and, sometimes, it's useful to not delete the Kubernetes namespace on failures to troubleshooting purposes. For disabling the deletion on failures, we need to run the test using:
mvn clean verify -Dts.kubernetes.delete.namespace.after.all=false