[OPENJDK-2735] Jlink Stage 3 template

[jhuttana]

As the COPY --from=ubi9-jlinked-image are not working fine directly in the template (PR #461) I just tried this approach to get our phase-3 done.
This is working fine.

$ oc get builds
NAME                       TYPE     FROM          STATUS     STARTED         DURATION
multistage-buildconfig-1   Docker   Git@5576f04   Complete   7 minutes ago   1m40s

[jmtd]

Nice!

This works because the multi-stage build is not split between two buildConfigs, so the COPY --from= instructions can refer to the image in the first stage within the same Dockerfile.

You've moved to an external Dockerfile, but that is not what has made this work: this example commit brings the Dockerfile back into the template and removes the external git reference: jmtd@5cfd7d5 ; this performs the same.

I think it is preferable to have the Dockerfile in-line in the template, because then the template is self-contained, and does not rely on fetching an external git repository during the build (and avoids issues of keeping that in sync etc)

The next step is to remove the hard-coded quay.io/jdowland/jlink:ubi9-jlinked-image to prepare for receiving the input from Phase 2. That will be via an ImageStream. Please extend this template with a placeholder ImageStream, which points at quay.io/jdowland/jlink:ubi9-jlinked-image for now. You could start by doing this

oc tag quay.io/jdowland/jlink:ubi9-jlinked-image intermediate:latest
oc get -o yaml is/intermediate

and then edit/simplify the result to add into the template. In particular, I think we should try changing lookupPolicy.local to true: I think this will help us to map between ImageStream names and the URI to include in the Dockerfile. From https://docs.openshift.com/container-platform/4.15/rest_api/image_apis/imagestream-image-openshift-io-v1.html#spec-lookuppolicy :

local will change the docker short image references (like "mysql" or "php:latest") on objects in this namespace to the image ID whenever they match this image stream, instead of reaching out to a remote registry. The name will be fully qualified to an image ID if found. The tag’s referencePolicy is taken into account on the replaced value. Only works within the current namespace.

In other words I think creating the above ImageStream and changing the Dockerfile to have FROM intermediate:latest should work. Please try! I haven't got this to work yet.

[jmtd]

There might be another approach we can take here if we take a step back. The multistage docker build was a solution to a problem we had: wanting to copy a bunch of artefacts from one image into another. There are other ways to achieve that in OpenShift, see e.g. https://docs.openshift.com/container-platform/4.15/cicd/builds/creating-build-inputs.html

If we temporarily ignore the first stage of the multi-stage build (which installs grep and gawk), and focus on the second part: we have the COPY --from=... instructions. Instead of copying from the first stage, we could have OpenShift provide the paths we want as build inputs using .source.images[].from,.source.images[].paths. Something like

      images:
      - from: 
          kind: ImageStreamTag
          name: intermediate:latest
        paths:
        - sourcePath:     /deployments
          destinationDir: deployments

[jmtd]

copying the artefacts using the buildConfig (instead of multi-stage build) looks really promising. This is what I managed so far:
jmtd@08d950e

I also created a Deployment from the lightweight-image ImageStream. It fails at start-up due to the missing grep:

/opt/jboss/container/util/pathfinder/pathfinder.sh: line 33: grep: command not found

but this at least confirms that the run script was successfully copied in; as was java:

...
Usage: java [options] <mainclass> [args...]
...

To resolve the missing grep (and awk) we could introduce yet another buildConfig (that did the first stage of the multistage Dockerfile) between the input intermediate, here, or, we could add those steps to the buildConfig in phase 1, and assume that the incoming intermediate imagestream has /tmp/jrootfs in place. I think I favour the latter.

We will also need the relevant triggers and probably wrapping ubi-micro up in an ImageStream.

[jhuttana]

copying the artefacts using the buildConfig (instead of multi-stage build) looks really promising. This is what I managed so far: jmtd@08d950e

I also created a Deployment from the lightweight-image ImageStream. It fails at start-up due to the missing grep:

/opt/jboss/container/util/pathfinder/pathfinder.sh: line 33: grep: command not found

but this at least confirms that the run script was successfully copied in; as was java:

...
Usage: java [options] <mainclass> [args...]
...

To resolve the missing grep (and awk) we could introduce yet another buildConfig (that did the first stage of the multistage Dockerfile) between the input intermediate, here, or, we could add those steps to the buildConfig in phase 1, and assume that the incoming intermediate imagestream has /tmp/jrootfs in place. I think I favour the latter.

Thanks Jom for making these changes and getting it to this stage.
And +1 for pushing /tmp/jrootfs to phase-1.
But I have question here, our intermediate imagestream is an output of phase-2 where we do an s2i build using the output of phase-1 (which is BUILDER_IMAEG eg: ubi9-openjdk-17-jlink:latest) so basically everything that is there in phase-1 will be propagated to phase-2 output through just an s2i build ? or we will have to have some specific copy instructions to copy /tmp/jrootfs?

We will also need the relevant triggers and probably wrapping ubi-micro up in an ImageStream.

+1

[jhuttana]

I also tried to create a deployment but its failing with ImagePullBackOff

$ oc create deployment lightweightimage --image=lightweight-image:latest
deployment.apps/lightweightimage created

$ oc get pods
NAME                                READY   STATUS             RESTARTS   AGE
lightweightimage-57968679c7-cksmn   0/1     ImagePullBackOff   0          2m4s
multistage-buildconfig-2-build      0/1     Init:Error         0          62m
multistage-buildconfig-3-build      0/1     Completed          0          46m

$ oc get all
Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
NAME                                    READY   STATUS             RESTARTS   AGE
pod/lightweightimage-57968679c7-cksmn   0/1     ImagePullBackOff   0          30s
pod/multistage-buildconfig-2-build      0/1     Init:Error         0          60m
pod/multistage-buildconfig-3-build      0/1     Completed          0          44m

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/lightweightimage   0/1     1            0           30s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/lightweightimage-57968679c7   1         1         0       30s

NAME                                                    TYPE     FROM         LATEST
buildconfig.build.openshift.io/multistage-buildconfig   Docker   Dockerfile   3

NAME                                                TYPE     FROM         STATUS                     STARTED          DURATION
build.build.openshift.io/multistage-buildconfig-2   Docker   Dockerfile   Failed (BuildPodEvicted)   56 minutes ago   12s
build.build.openshift.io/multistage-buildconfig-3   Docker   Dockerfile   Complete                   44 minutes ago   34s

NAME                                               IMAGE REPOSITORY                                                                    TAGS     UPDATED
imagestream.image.openshift.io/intermediate        default-route-openshift-image-registry.apps-crc.testing/phase-4/intermediate        latest   About an hour ago
imagestream.image.openshift.io/lightweight-image   default-route-openshift-image-registry.apps-crc.testing/phase-4/lightweight-image   latest   44 minutes ago

Am I doing anything wrong while creating the deployment ?

[jmtd]

this is putting the result at /jrootfs in the final image. We need to adjust it somehow so it unpacks over / instead.