Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify and Document how to customize the "wait-for" image when using init tasks on kubernetes #35455

Closed
2 of 5 tasks
cescoffier opened this issue Aug 22, 2023 · 2 comments · Fixed by #35877 or #35918
Closed
2 of 5 tasks
Assignees
Labels
area/jbang Issues related to when using jbang.dev with Quarkus area/kubernetes kind/epic Large issue with links to sub-issues
Milestone

Comments

@cescoffier
Copy link
Member

cescoffier commented Aug 22, 2023

Context

When using an init task (Liquibase, Flyway, or soon Kafka and AMQP), Quarkus generates multiple resources:

  • a pod containing two containers: the application and an init-container waiting for the init job to run
  • a job running the application container using specific parameters to run the init tasks.

Note that each init task runs in a separate job and introduces another init container in the application pod.

By default, Quarkus init-container uses the https://github.com/groundnuty/k8s-wait-for/ image to wait for the job execution.
The init container runs that image with the following command line argument: job <name of the job>.
Also, Quarkus generates a service account, adding the view-jobs roles (batch -> job -> get) to allow the init container to check the job execution status.

Problem

The image used by default is not following our image recommendation and, at the moment, contains two critical CVEs and 11 high CVEs (and a myriad of minors and lows).

Fortunately, it's possible to customize the image. However, the process is a bit cumbersome.
First, you need to provide an image following the exact same "way" to run and not requiring any more permission than "batch->jobs->GET" (so you cannot list the jobs). Then, you need to configure for each init task:

quarkus.kubernetes.init-tasks.coffeeshop-service-kafka-topic-init.image=quay.io/cescoffi/wait-for-job

The coffeeshop-service-kafka-topic-init mixes the task name and the application name. As reported in #35436, an end user can't guess this name (without reading the Quarkus code).

Also, you need to comply with the launch parameter of the container and the granted permissions.

Some parts of the solution

I came up with a containerized JBang script replacing the "wait-for" image:

///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS info.picocli:picocli:4.6.3
//DEPS io.fabric8:kubernetes-client:6.8.1

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Parameters;

import java.util.concurrent.Callable;

import io.fabric8.kubernetes.api.model.batch.v1.Job;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClientBuilder;

@Command(name = "WaitForJob", mixinStandardHelpOptions = true, version = "WaitForJob 0.1", description = "WaitForJob made with jbang")
class WaitForJob implements Callable<Integer> {

    @Parameters(index = "0", description = "The `job` string")
    private String ignored;

    @Parameters(index = "1", description = "The job name (mandatory)")
    private String name;

    public static void main(String... args) {
        int exitCode = new CommandLine(new WaitForJob()).execute(args);
        System.exit(exitCode);
    }

    @Override
    public Integer call() throws Exception {
        if (name == null || name.isBlank()) {
            throw new IllegalArgumentException("The job name must be set");
        }

        KubernetesClient client = new KubernetesClientBuilder().build();

        long begin = System.currentTimeMillis();
        // Wait for 10 min before given up
        while (System.currentTimeMillis() - begin < 10 * 60 * 1000) {
            Job job = client.batch().v1().jobs().withName(name).get();            
            if (job == null) {
                System.out.println("Job " + name + " not found");
            } else {                
                if (job.getStatus().getActive() != null && job.getStatus().getActive() == 1) {
                    System.out.println("Job " + name + " found: active: " + job.getStatus().getActive() + ", succeeded: "
                        + job.getStatus().getSucceeded() + ", failed: " + job.getStatus().getFailed());
                    return 0;
                }
                System.out.println("Job " + name + " found: active: " + job.getStatus().getActive() + ", succeeded: "
                        + job.getStatus().getSucceeded() + ", failed: " + job.getStatus().getFailed());
            }
            Thread.sleep(2000);
        }
        System.out.println("Job still not executed correctly - exiting");
        return -1;
    }
}
# syntax=docker/dockerfile:1.4

# Step 1 - build a jbang application
FROM jbangdev/jbang-action as builder
WORKDIR /

COPY ./WaitForJob.java /

RUN chmod +x WaitForJob.java && mkdir out
RUN jbang export portable -O output/waitForJob.jar WaitForJob.java

# Step 2 - build the image that will wait for the job to be executed.
FROM registry.access.redhat.com/ubi9/openjdk-17-runtime:1.15

COPY --from=builder /out /out
ENTRYPOINT ["java", "-jar", "out/waitForJob.jar"]

Once built and deployed, you can replace the init container image.

Pending issues

@cescoffier cescoffier added the kind/epic Large issue with links to sub-issues label Aug 22, 2023
@quarkus-bot quarkus-bot bot added area/jbang Issues related to when using jbang.dev with Quarkus area/kubernetes labels Aug 22, 2023
@quarkus-bot
Copy link

quarkus-bot bot commented Aug 22, 2023

/cc @Sgitario (kubernetes), @geoand (kubernetes), @iocanel (kubernetes), @maxandersen (jbang), @quarkusio/devtools (jbang)

@cescoffier
Copy link
Member Author

The PR is a good start but it only fixes some things.
The current documentation does not explain how the image is launched, what are the permissions granted, and so on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/jbang Issues related to when using jbang.dev with Quarkus area/kubernetes kind/epic Large issue with links to sub-issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants