Ability to detect podman when creating container output

[ajhajj]

I have noticed that:

./mvnw package -Pnative -Dquarkus.native.container-build=true

will detect either docker or podman and use accordingly. However:

./mvnw package -Pnative -Dquarkus.native.container-build=true -Dquarkus.container-image.build=true

requires the presence of docker unless I explicitly specify '-Dquarkus.native.container-runtime=podman'. This feels like inconsistent behavior. Can we see about detecting podman for the container image build as we do for the native build?

[geoand]

Can you explain which part of the build requires docker?

Some logs for the second case would be helpful

[ajhajj]

It seems that if I want to create a container output as documented in https://quarkus.io/guides/building-native-image#creating-a-container. There it mentions a dependency on container-image extensions (https://quarkus.io/guides/container-image). There are three as I understand it. But for usecases outside of OpenShift just jib and docker. Unfortunately jib also requires docker in some cases as indicated by:
"In situations where all that is needed to build a container image and no push to a registry is necessary (essentially by having set quarkus.container-image.build=true and left quarkus.container-image.push unset - it defaults to false), then this extension creates a container image and registers it with the Docker daemon. This means that although Docker isn’t used to build the image, it is nevertheless necessary. Also note that using this mode, the built container image will show up when executing docker images."

With the following command even though I am running podman locally and have explicitly indicated that I want to use podman, it will not try to put the image in the local podman repo. It wants to go to docker.io instead.

./mvnw package -Pnative -Dquarkus.native.container-build=true -Dquarkus.container-image.build=true -Dquarkus.native.container-runtime=podman -Dquarkus.container-image.push=true

The output of the run is below:

[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------------< com.redhat:jsp >---------------------------
[INFO] Building jsp 1.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- quarkus-maven-plugin:1.12.2.Final:generate-code (default) @ jsp ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ jsp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 10 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ jsp ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- quarkus-maven-plugin:1.12.2.Final:generate-code-tests (default) @ jsp ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ jsp ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/ajhajj/dev/quarkus_servlet/workspace/insult/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ jsp ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-test) @ jsp ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ jsp ---
[INFO] Building jar: /home/ajhajj/dev/quarkus_servlet/workspace/insult/target/jsp-1.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- quarkus-maven-plugin:1.12.2.Final:build (default) @ jsp ---
[INFO] [org.jboss.threads] JBoss Threads version 3.2.0.Final
[INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: /home/ajhajj/dev/quarkus_servlet/workspace/insult/target/jsp-1.0.0-SNAPSHOT-native-image-source-jar/jsp-1.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from /home/ajhajj/dev/quarkus_servlet/workspace/insult/target/jsp-1.0.0-SNAPSHOT-native-image-source-jar/jsp-1.0.0-SNAPSHOT-runner.jar
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Using podman to run the native image builder
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Checking image status quay.io/quarkus/ubi-quarkus-native-image:21.0.0-java11
Trying to pull quay.io/quarkus/ubi-quarkus-native-image:21.0.0-java11...
Getting image source signatures
Copying blob 57de4da701b5 skipped: already exists
Copying blob 0f7eddd60d0a skipped: already exists
Copying blob cf0f3ebe9f53 [--------------------------------------] 0.0b / 0.0b
Copying config 31ccea2b17 done
Writing manifest to image destination
Storing signatures
31ccea2b17aeb3f2f1d2ab187da9c78b7e7f1ab135a44bff982f0baef86026ab
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GraalVM Version 21.0.0 (Java Version 11.0.10+8-jvmci-21.0-b06)
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=1 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --initialize-at-build-time= -H:InitialCollectionPolicy=com.oracle.svm.core.genscavenge.CollectionPolicy$BySpaceAndTime -H:+JNI -H:+AllowFoldMethods -jar jsp-1.0.0-SNAPSHOT-runner.jar -H:FallbackThreshold=0 -H:+ReportExceptionStackTraces -H:-AddAllCharsets -H:EnableURLProtocols=http --no-server -H:-UseServiceLoaderFeature -H:+StackTrace jsp-1.0.0-SNAPSHOT-runner
[jsp-1.0.0-SNAPSHOT-runner:20] classlist: 2,210.20 ms, 0.96 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (cap): 587.32 ms, 0.96 GB
[jsp-1.0.0-SNAPSHOT-runner:20] setup: 1,873.20 ms, 0.96 GB
18:34:27,922 INFO [org.jbo.threads] JBoss Threads version 3.2.0.Final
[jsp-1.0.0-SNAPSHOT-runner:20] (clinit): 451.45 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (typeflow): 10,902.18 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (objects): 9,749.35 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (features): 434.92 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] analysis: 22,416.41 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] universe: 1,066.85 ms, 2.30 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (parse): 2,636.97 ms, 3.15 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (inline): 4,478.33 ms, 3.80 GB
[jsp-1.0.0-SNAPSHOT-runner:20] (compile): 18,705.34 ms, 4.77 GB
[jsp-1.0.0-SNAPSHOT-runner:20] compile: 27,232.37 ms, 4.77 GB
[jsp-1.0.0-SNAPSHOT-runner:20] image: 2,467.70 ms, 4.77 GB
[jsp-1.0.0-SNAPSHOT-runner:20] write: 412.37 ms, 4.77 GB
[jsp-1.0.0-SNAPSHOT-runner:20] [total]: 57,914.68 ms, 4.77 GB
[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] No container image registry was set, so 'docker.io' will be used
[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Starting container image build
[WARNING] [io.quarkus.container.image.jib.deployment.JibProcessor] Base image 'registry.access.redhat.com/ubi8/ubi-minimal' does not use a specific image digest - build may not be reproducible
[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Using base image with digest: sha256:e72e188c6b20281e241fb3cf6f8fc974dec4cc6ed0c9d8f2d5460c30c35893b3
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] I/O error for image [registry-1.docker.io/ajhajj/jsp]:
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] I/O error for image [registry-1.docker.io/ajhajj/jsp]:
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] javax.net.ssl.SSLException
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] javax.net.ssl.SSLException
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] Socket closed
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] Socket closed
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:08 min
[INFO] Finished at: 2021-04-08T14:35:17-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.quarkus:quarkus-maven-plugin:1.12.2.Final:build (default) on project jsp: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
[ERROR] [error]: Build step io.quarkus.container.image.jib.deployment.JibProcessor#buildFromNative threw an exception: java.lang.RuntimeException: Unable to create container image
[ERROR] at io.quarkus.container.image.jib.deployment.JibProcessor.containerize(JibProcessor.java:191)
[ERROR] at io.quarkus.container.image.jib.deployment.JibProcessor.buildFromNative(JibProcessor.java:168)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:920)
[ERROR] at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2415)
[ERROR] at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1452)
[ERROR] at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR] at org.jboss.threads.JBossThread.run(JBossThread.java:501)
[ERROR] Caused by: com.google.cloud.tools.jib.api.RegistryUnauthorizedException: Unauthorized for registry-1.docker.io/ajhajj/jsp
[ERROR] at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:164)
[ERROR] at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:115)
[ERROR] at com.google.cloud.tools.jib.registry.RegistryClient.callRegistryEndpoint(RegistryClient.java:624)
[ERROR] at com.google.cloud.tools.jib.registry.RegistryClient.checkBlob(RegistryClient.java:476)
[ERROR] at com.google.cloud.tools.jib.builder.steps.PushBlobStep.call(PushBlobStep.java:74)
[ERROR] at com.google.cloud.tools.jib.builder.steps.PushLayerStep.call(PushLayerStep.java:93)
[ERROR] at com.google.cloud.tools.jib.builder.steps.PushLayerStep.call(PushLayerStep.java:33)
[ERROR] at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
[ERROR] at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:69)
[ERROR] at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
[ERROR] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[ERROR] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[ERROR] at java.base/java.lang.Thread.run(Thread.java:834)
[ERROR] Caused by: com.google.cloud.tools.jib.http.ResponseException: 401 Unauthorized
[ERROR] HEAD https://registry-1.docker.io/v2/ajhajj/jsp/blobs/sha256:cc607d978888bd7eb276c3a582e6c6e00425b47804d18ee767504a63196479bb
[ERROR] at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:329)
[ERROR] at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:250)
[ERROR] at com.google.cloud.tools.jib.registry.RegistryEndpointCaller.call(RegistryEndpointCaller.java:139)
[ERROR] ... 12 more
[ERROR] Caused by: com.google.api.client.http.HttpResponseException: 401 Unauthorized
[ERROR] HEAD https://registry-1.docker.io/v2/ajhajj/jsp/blobs/sha256:cc607d978888bd7eb276c3a582e6c6e00425b47804d18ee767504a63196479bb
[ERROR] at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1113)
[ERROR] at com.google.cloud.tools.jib.http.FailoverHttpClient.call(FailoverHttpClient.java:323)
[ERROR] ... 14 more
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[ajhajj]

I updated my previous comment as I had forgotten to enable the jib extension on my first run output.

[ajhajj]

The fact that it will automatically register with the local docker registry if a remote registry is not provided but not do the same for podman is the issue I am trying to describe really.

[geoand]

OK, thanks for the information.

As implied by the names of the different properties, (quarkus.native.container-runtime and quarkus.native.container-build) these are two different things.
So although I understand you would like to use podman throughout, these are two different things, so you would need to configure the registry (-Dquarkus.container-image.registry=....) to do what you want.

[ajhajj]

I agree that they are different. My concern was the overall consistency of user experience. Maybe that is something we can consider down the road some time.

[geoand]

Yes, that is a good point.

Maybe we could introduce some global setting.

WDYT @zakkak, @gsmet ?

[zakkak]

+1 being able to define the container engine globally (or even better detect it automatically) would be nice.

I have only tried podman with container-image-docker though, so I am not sure how it would work with JIB and S2I.

[andrewazores]

There is also the dockerClient.executable config property that Jib supports which I think might be relevant here. If it is not then I think that a separate issue could be opened to add support for this to the quarkus-container-image-jib build plugin as quarkus.jib.dockerClient.executable=/usr/bin/podman. I use this in Cryostat to default the image builder to podman, but allow it to be overridden if some developer needs that to be docker (or probably buildah too - haven't tried that yet).

[geoand]

Best open a new issue for this.

[geoand]

I am going to close this because #24238 will introduce the same detection method we have for the native binary using container build into our Jib support