[OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image #461

jhuttana · 2024-03-14T19:49:32Z

This is still WIP and an initial attempt to write a BuildConfig template.

jmtd · 2024-03-15T11:17:22Z

This is looking good! I'll leave some comments.

templates/multistage-buildconfig.yaml

jmtd · 2024-03-15T11:22:38Z

templates/multistage-buildconfig.yaml

+    dockerfile: |
+	#Stage-1: ubi9-jlinked-image is builder image + application + jlinked JVM
+	# install additional system dependencies (for ubi-micro) to /mnt/jrootfs
+	FROM ubi9-jlinked-image AS ubi9-jlinked-image  # Replace this with appropriate base image


We wil want OpenShift to substitute FROM ubi9-jlinked-image with the correct imagestream name/URI for the specific instance of this template once deployed. To achieve that, change this line to FROM - AS ubi9-jlinked-image; and then we also need to add some more metadata elsewhere in the buildConfig, a dockerStrategy key underneath strategy:, which will describe the ImageStream to pull from. See https://docs.openshift.com/container-platform/4.15/cicd/builds/build-strategies.html#builds-strategy-docker-from-image_build-strategies

Aha, quoting that URI, we may have a problem: "If the Dockerfile uses multi-stage builds, the image in the last FROM instruction will be replaced.", so in this case, that would be the ubi-micro below, not what we want.

We might have to switch from a mutistage dockerfile to two separate, linked OpenShift buildConfigs instead.

jmtd · 2024-03-19T14:05:59Z

I don't think switching to dockerfilePath will address the problem that OpenShift will substitute the last FROM instruction in the multi-stage build, rather than the first: in other words, whether the Dockerfile is embedded or not, OpenShift will (optionally) replace FROM registry.access.redhat.com/ubi9/ubi-micro, but not FROM ubi9-jlinked-image, which is the one we need substituted.

Two approaches that might work are moving the first stage work into Phase 1, or wrapping the whole thing up in an OpenShift template, and using the template parameters to substitute the FROM. We are going to be using OpenShift templates for other aspects, so it would be worth looking at that now.

jmtd · 2024-03-20T16:08:53Z

templates/multistage-buildconfig.yaml

+spec:
+  source:
+    dockerfile: |
+	#Stage-1: ubi9-jlinked-image is builder image + application + jlinked JVM


error parsing multistage-buildconfig.yaml: error converting YAML to JSON: yaml: line 8: found a tab character where an indentation space is expected

jmtd · 2024-03-20T16:10:44Z

templates/multistage-buildconfig.yaml

+metadata:
+  generation: 1
+  name: jlinked-image
+  namespace: default


remove this line

templates/multistage-buildconfig.yaml

jmtd · 2024-03-20T16:11:45Z

templates/multistage-buildconfig.yaml

+    dockerfile: |
+	#Stage-1: ubi9-jlinked-image is builder image + application + jlinked JVM
+	# install additional system dependencies (for ubi-micro) to /mnt/jrootfs
+	FROM - AS ubi9-jlinked-image  # Replace this with appropriate base image


since we cannot rely on OpeNShift to substitute "-" here, for dev purposes we could use FROM quay.io/jdowland/jlink:ubi9-jlinked-image as ubi9-jlinked-image

… for jlinked image Signed-off-by: Jayashree Huttanagoudar <[email protected]>

… a new file multistage-buildconfig1.yaml with another strategy Signed-off-by: Jayashree Huttanagoudar <[email protected]>

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

…stage-1 and stage-2 in the multistage Dockerfile. Also use proper BuildConfig objects to replace FROM, ENV and ARG in the Dockerfile. Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana · 2024-03-22T07:31:24Z

On my local crc setup I tried to create the template as follows:

$ oc create -f templates/multistage-buildconfig.yaml
template.template.openshift.io/jlinked-image-template created

$ oc get template
NAME                     DESCRIPTION                                                              PARAMETERS    OBJECTS
jlinked-image-template   Template to produce an imagestream and buildconfig for a Jlinked image   0 (all set)   3

Further I tried to process the template and end-up with some errors :)

$ oc process jlinked-image-template | oc create -f -
imagestream.image.openshift.io/jlinked-image created
Error from server (Invalid): error when creating "STDIN": BuildConfig.build.openshift.io "ubi9-jlinked-image-buildconfig" is invalid: spec.strategy: Invalid value: build.BuildStrategy{DockerStrategy:(*build.DockerBuildStrategy)(nil), SourceStrategy:(*build.SourceBuildStrategy)(nil), CustomStrategy:(*build.CustomBuildStrategy)(nil), JenkinsPipelineStrategy:(*build.JenkinsPipelineBuildStrategy)(nil)}: must provide a value for exactly one of sourceStrategy, customStrategy, dockerStrategy, or jenkinsPipelineStrategy
Error from server (Invalid): error when creating "STDIN": BuildConfig.build.openshift.io "lean-runtime-buildconfig" is invalid: spec.strategy: Invalid value: build.BuildStrategy{DockerStrategy:(*build.DockerBuildStrategy)(nil), SourceStrategy:(*build.SourceBuildStrategy)(nil), CustomStrategy:(*build.CustomBuildStrategy)(nil), JenkinsPipelineStrategy:(*build.JenkinsPipelineBuildStrategy)(nil)}: must provide a value for exactly one of sourceStrategy, customStrategy, dockerStrategy, or jenkinsPipelineStrategy

Looks like some wrong stuff is fed for build strategy !.
Which I need to figure out.

jmtd · 2024-03-22T10:39:51Z

These are just indentation errors I think: It's saying the buildconfigs don't have .spec.strategy keys. At the moment the strategy keys are indented such that they're under spec.source.

jmtd · 2024-03-22T10:42:29Z

templates/multistage-buildconfig.yaml

+        dockerStrategy:
+          from:
+            kind: ImageStreamTag
+            name: quay.io/repository/jdowland/jlink:ubi9-jlinked-image


This is a registry URI , but it needs to be an ImageStreamTag. The eventually-correct value for this will depend upon Josh's Phase 2 template. for now you could put jlinked-image which definitely exists (your template creates it) as a placeholder. I'd put a comment after # placeholder value

jmtd · 2024-03-22T10:43:03Z

templates/multistage-buildconfig.yaml

+      dockerfile: |
+        #Stage-1: ubi9-jlinked-image is builder image + application + jlinked JVM
+        # install additional system dependencies (for ubi-micro) to /mnt/jrootfs
+        FROM -


until the dockerStategy.from.name below is the valid image stream, I'd put the test image URI here: FROM quay.io/repository/jdowland/jlink:ubi9-jlinked-image

jmtd · 2024-03-22T10:44:12Z

templates/multistage-buildconfig.yaml

+          imageChange:
+            from:
+              kind: ImageStreamTag
+              name: quay.io/repository/jdowland/jlink:ubi9-jlinked-image  # ImageStreamTag to trigger the build (stage-1 in Dockerfile)


likewise this is not a valid ImageStreamTag. Put a placeholder value in until we know a real one further along development

jmtd · 2024-03-22T10:44:39Z

templates/multistage-buildconfig.yaml

+        dockerStrategy:
+          from:
+            kind: ImageStreamTag
+            name: registry.access.redhat.com/ubi9/ubi-micro


not a valid ImageStream tag. use a placeholder

jmtd · 2024-03-22T10:45:23Z

templates/multistage-buildconfig.yaml

+    source:
+      dockerfile: |
+        #Stage-2:copy application JAR and jlinked JRE to runtime image
+        FROM -


this doesn't need to be parameterized I don't think. use FROM registry.access.redhat.com/ubi9/ubi-micro. (I wonder if we should actually define an ImageStream for ubi-micro or not.)

jmtd · 2024-03-22T10:46:51Z

templates/multistage-buildconfig.yaml

+        #Stage-2:copy application JAR and jlinked JRE to runtime image
+        FROM -
+
+        COPY --from=ubi9-jlinked-image /mnt/jrootfs/ /


these instructions will only work in a Dockerfile multi-stage build, since there's nothing labelled "ubi9-jlinked-image" in these instructions now

If we replace FROM instruction with quay image directly as FROM quay.io/repository/jdowland/jlink:ubi9-jlinked-image then the output.to.name is ubi9-jlinked-image. My understanding is that this is made available by the first buildConfig that is ubi9-jlinked-image-buildconfig

Correct me if I am wrong

jmtd · 2024-03-22T10:48:01Z

templates/multistage-buildconfig.yaml

+          imageChange:
+            from:
+              kind: ImageStreamTag
+              name: registry.access.redhat.com/ubi9/ubi-micro  # ImageStreamTag for the final output image (stage-2 in Dockerfile)


this isn't a valid ImageStreamTag. If we wan't to trigger a rebuild when ubi-micro is changed, we will need to define a ubi-micro ImageStream. Perhaps we should

jmtd · 2024-03-22T10:49:20Z

templates/multistage-buildconfig.yaml

+            --setopt=varsdir=/etc/dnf/vars \
+            grep gawk
+        RUN rm -rf /mnt/jrootfs/var/cache/* /mnt/jrootfs/var/lib/rpm /mnt/jrootfs/var/lib/dnf
+      strategy:


this key belongs under spec not spec.source

jmtd · 2024-03-22T10:49:30Z

templates/multistage-buildconfig.yaml

+
+        USER 185
+        CMD /opt/jboss/container/java/run/run-java.sh
+      strategy:


this key belongs under spec not spec.source

jmtd · 2024-03-22T10:52:14Z

This is looking great. I'm not 100% sure that we should split the build into two buildConfigs here or not; it might be that we can use an openshift template parameter for the FROM in a dockerfile multi-stage build, if we could determine the correct value based on template parameters. (I'm not sure if we can or not!) But if we move ahead with two buildConfigs, as per the current status, we need to do something to fix up the COPY --from lines.

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana · 2024-03-22T14:06:00Z

This is looking great. I'm not 100% sure that we should split the build into two buildConfigs here or not; it might be that we can use an openshift template parameter for the FROM in a dockerfile multi-stage build, if we could determine the correct value based on template parameters. (I'm not sure if we can or not!) But if we move ahead with two buildConfigs, as per the current status, we need to do something to fix up the COPY --from lines.

Thanks for the review.
I will try to brainstorm and experiment few options and update my findings :)

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana · 2024-03-22T18:24:39Z

With the recent commit able to create two separate BuildConfigs for stage-1 and stage-2 in our multi-stage Dockerfile

$ oc create -f templates/multistage-buildconfig.yaml
template.template.openshift.io/jlinked-image-template created

$ oc get templates
NAME                     DESCRIPTION                                                              PARAMETERS    OBJECTS
jlinked-image-template   Template to produce an imagestream and buildconfig for a Jlinked image   0 (all set)   3

$ oc process jlinked-image-template | oc create -f -
imagestream.image.openshift.io/jlinked-image created
buildconfig.build.openshift.io/ubi9-jlinked-image-buildconfig created
buildconfig.build.openshift.io/lean-runtime-buildconfig created

jmtd · 2024-03-26T11:46:55Z

Some notes on the current status:

The first buildConfig cannot be run because of a dependency issue

oc start-build ubi9-jlinked-image-buildconfig
The ImageStreamTag "jlinked-image:latest" is invalid: from: Error resolving ImageStreamTag jlinked-image:latest in namespace jlink: unable to find latest tagged image

The ImageStream exists, but there is no :latest tag, because no builds have run yet. The buildConfig outputs to that ImageStream, so we cannot also depend upon it. Removing dockerStrategy.from resolves that problem.

Next problem, builds from that build Config still fail, " Output image could not be resolved.". The output is described as ubi9-jlinked-image:latest which does not exist. The imageStream defined here is jlinked-image instead. Changing that and I can start builds

The builds fail because quay.io/repository/jdowland/jlink:ubi9-jlinked-image cannot be pulled. the correct name is quay.io/jdowland/jlink:ubi9-jlinked-image (no "repository"). Fixing that, and the build can fetch the image OK, and the build succeeds!

The next problem is, after the build succeeds, it updates the jlinked-image imagestream, which causes the trigger for the buildconfig to fire, so it builds again... a potentially infinite loop. The imageChange trigger for this buildConfig should be removed.

jmtd · 2024-03-26T11:52:44Z

On to the second buildConfig, lean-runtime-buildconfig. This gets triggered by the imagestream jlinked-image getting updated (which the first buildConfig does), great.

This second buildConfig is defined to build from, and also write to, jlinked-image imageStream, which cannot work. We need a separate imagStream for each buildConfig to read or write to.

builds from the second buildConfig fail at the COPY --from=ubi9-jlinked-image instructions: it does not know what ubi9-jlinked-image is. I tried to change it to COPY --from=jlinked-image, matching the input imageStream, but that doesn't work either yet.

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana · 2024-03-27T10:56:45Z

With the recent changes FirstBuildConfig completed successfully.
The secondBuildConfig when I see on the web console as developer its state is still new but as Administrator -> Build -> Phase-3 -> Activity -> view evernts

Generated from buildconfig-controller
14 times in the last 13 minutes
error instantiating Build from BuildConfig phase-3/lean-runtime-buildconfig (0): Error resolving ImageStreamTag ubi9-jlinked-image:latest in namespace phase-3: unable to find latest tagged image

But the FirstBuildConfig has successully created ubi9-jlinked-image:latest

$ oc get is
NAME                 IMAGE REPOSITORY                                                                     TAGS     UPDATED
ubi9-jlinked-image   default-route-openshift-image-registry.apps-crc.testing/phase-3/ubi9-jlinked-image   latest   5 minutes ago

…to resolve the second BuildConfig indefinite pending status Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana · 2024-03-27T19:41:38Z

I am not sure whether custom build strategy is useful to handle COPY instructions in the secondBuildConfig.
If yes, then we have to use script. Instead the better way would be to use strategy.dockerStrategy.dockerfilePath as used in #471

jhuttana · 2024-04-04T08:42:59Z

Closing this PR as we are focusing on #471

jhuttana requested review from jmtd and Josh-Matsuoka March 14, 2024 19:49

jmtd reviewed Mar 15, 2024

View reviewed changes

templates/multistage-buildconfig.yaml Outdated Show resolved Hide resolved

jmtd reviewed Mar 15, 2024

View reviewed changes

jmtd reviewed Mar 20, 2024

View reviewed changes

templates/multistage-buildconfig.yaml Outdated

metadata:

generation: 1

name: jlinked-image

namespace: default

Copy link

Member

jmtd Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this line

jmtd reviewed Mar 20, 2024

View reviewed changes

templates/multistage-buildconfig.yaml Outdated Show resolved Hide resolved

jmtd reviewed Mar 20, 2024

View reviewed changes

templates/multistage-buildconfig.yaml Outdated Show resolved Hide resolved

jmtd reviewed Mar 20, 2024

View reviewed changes

jhuttana added 3 commits March 21, 2024 23:03

BuildConfig for multistage build to align with 3-phase build strategy…

204a3c6

… for jlinked image Signed-off-by: Jayashree Huttanagoudar <[email protected]>

Address review comments for multistage-buildconfig.yaml and also push…

6a1cb7b

… a new file multistage-buildconfig1.yaml with another strategy Signed-off-by: Jayashree Huttanagoudar <[email protected]>

Address review comments for syntatic errors and base image

78a7ac8

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana force-pushed the 3-phase-template branch from ddaa113 to 78a7ac8 Compare March 21, 2024 18:04

Attempt to update the buildconfig to use two linked buildconfigs for …

d0ddc0e

…stage-1 and stage-2 in the multistage Dockerfile. Also use proper BuildConfig objects to replace FROM, ENV and ARG in the Dockerfile. Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jmtd reviewed Mar 22, 2024

View reviewed changes

Address review comments for FROM instruction and some indentation

51be413

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

Fix buildStrategy related issues

724d3e9

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana changed the title ~~[OPENJDK-2735] BuildConfig for multistage build to align with 3-phase build strategy for jlinked image~~ [OPENJDK-2735] Phase-3 BuildConfig for multistage build for jlinked image Mar 25, 2024

jhuttana changed the title ~~[OPENJDK-2735] Phase-3 BuildConfig for multistage build for jlinked image~~ [OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image Mar 25, 2024

Address review comments for docketStrategy.from and few more

f6f6d81

Signed-off-by: Jayashree Huttanagoudar <[email protected]>

Commented out COPY instructions and added final image stream section …

f263384

…to resolve the second BuildConfig indefinite pending status Signed-off-by: Jayashree Huttanagoudar <[email protected]>

jhuttana mentioned this pull request Mar 27, 2024

[OPENJDK-2735] Jlink Stage 3 template #471

Merged

jhuttana closed this Apr 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image #461

[OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image #461

jhuttana commented Mar 14, 2024

jmtd commented Mar 15, 2024

jmtd Mar 15, 2024

jmtd Mar 15, 2024

jmtd commented Mar 19, 2024

jmtd Mar 20, 2024

jmtd Mar 20, 2024

jmtd Mar 20, 2024

jhuttana commented Mar 22, 2024 •

edited

Loading

jmtd commented Mar 22, 2024

jmtd Mar 22, 2024 •

edited

Loading

jmtd Mar 22, 2024 •

edited

Loading

jmtd Mar 22, 2024

jmtd Mar 22, 2024

jmtd Mar 22, 2024

jmtd Mar 22, 2024

jhuttana Mar 22, 2024 •

edited

Loading

jmtd Mar 22, 2024

jmtd Mar 22, 2024

jmtd Mar 22, 2024

jmtd commented Mar 22, 2024

jhuttana commented Mar 22, 2024

jhuttana commented Mar 22, 2024

jmtd commented Mar 26, 2024

jmtd commented Mar 26, 2024

jhuttana commented Mar 27, 2024

jhuttana commented Mar 27, 2024

jhuttana commented Apr 4, 2024

[OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image #461

[OPENJDK-2735] Phase-3 BuildConfig template for multistage build for jlinked image #461

Conversation

jhuttana commented Mar 14, 2024

jmtd commented Mar 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmtd commented Mar 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhuttana commented Mar 22, 2024 • edited Loading

jmtd commented Mar 22, 2024

jmtd Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

jmtd Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhuttana Mar 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmtd commented Mar 22, 2024

jhuttana commented Mar 22, 2024

jhuttana commented Mar 22, 2024

jmtd commented Mar 26, 2024

jmtd commented Mar 26, 2024

jhuttana commented Mar 27, 2024

jhuttana commented Mar 27, 2024

jhuttana commented Apr 4, 2024

jhuttana commented Mar 22, 2024 •

edited

Loading

jmtd Mar 22, 2024 •

edited

Loading

jmtd Mar 22, 2024 •

edited

Loading

jhuttana Mar 22, 2024 •

edited

Loading