jib builder

[lburgazzoli]

Evaluate to use jib [1] as a builder strategy and eventually leverage its plug-in extension mechanism [2]

[1] https://github.com/GoogleContainerTools/jib
[2] https://cloud.google.com/blog/products/application-development/using-jib-to-containerize-java-apps

[lburgazzoli]

/cc @astefanutti

[github-actions]

This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!

[lburgazzoli]

I think it would be nice to consolidate on jib (except of s2i probably) and get rid of the other build strategies mentioned here: #3336

[squakez]

Would it make sense to target for a 2.0 version? It looks like we won't be able to maintain compatibility (at least, not that easily) if we switch to something new.

[lburgazzoli]

Yes for 2.x

[squakez]

Something else we want to consider is to support multi-architecture via JIB: see #1238

[gansheer]

I spent some time getting to know the existing JIB plugins available.

The ones I found and tested are:

• Quarkus Container Extension Jib (camel-k quarkus version)
• Jkube Kubernetes Plugin (version 1.10.1)
• Jib maven plugin (version 3.3.1)
• Fabric8 docker plugin (version 0.41.0)

I did some build test using the following requirements :

• build a container image containing the content from the /tmp/kit-xxxxxx-yyyyy/context instead of the default target folder ( /tmp/kit-xxxxxx-yyyyy/maven/target)
• build incrementally the container images between multiple integration kits by having some influence on layer build
• build the container image for different architectures (linux/am64, linux/arm64, etc)
• producing a container image the most similar than the one produced by spectrum

The native build still needs to be tested but in the context of container image build is it also about a runner being made available in /tmp/kit-xxxxxx-yyyyy/context after the build phase, so it should be covered but the requirements.

I put all my data here : https://github.com/gansheer/jib-work

Only two of the tested plugins validate all the requirements : Jib maven plugin and Fabric8 docker plugin. Seeing the Fabric8 docker plugin's version is 0.41.0, I feel the Jib maven plugin is the best choice.

The Jkube Kubernetes Plugin is still a good candidate if it can add the multi-architecture support. I found it offers a lot of flexibility and also come with a Jkube Openshift Plugin built on top of Kubernetes Plugin that has s2i features.

What I see as an implementation roadmap for the addition of a jib publish strategy would be:

• implement a first version using the Jib maven plugin while discussing with the jkube projet to see if the multi-architecture can be added
• if (and hopefully when) the Jkube Kubernetes Plugin provides the multi-architecure feature, plan a migration to the JKube plugin

Last notes:

• Java native build still need to be tested, particularly the java native build PR.
• I didn't see anything incompatible with the build decoupling discussions

Please don't hesitate to tell me your opinions on this and any additional inputs.

cc @lburgazzoli @astefanutti @oscerd @claudio4j @christophd @tadayosi

[squakez]

Great analysis. I think that eventually JIB and JKube may coexist if we manage to have enough resources to support both. For the time being we could start with JIB implementation IMO.

[gansheer]

I am trying to understand the reason behind some code designs. This is partially related to the #4024 discussion.

From what I can see the effective build code consist of 2 Tasks: a BuildTask and a PublishTask. The BuildTask itself contains many Steps separated in 5 phases:

• InitPhase
• ProjectGenerationPhase
• ProjectBuildPhase
• ApplicationPackagePhase
• ApplicationPublishPhase

I want to find a right way to add the maven command for the Jib build in the new Publish strategy. From the current code, I don't get the information about the build maven command already generated by the build task (passed through the steps in the builderContext), that leaves me 3 choices (that I can see):

• generate it again as it was done in build code
• create a BuilderTask Step instead of a publish task
• find a way to communicate it from the build task to the publish task

Before making any design choice I would like to understand the following points:

• Why has the code been separated in 2 Tasks ? Is there a technical reason why the PublishTask code is not a BuilderTask Step ?
• Is there a purpose to the presence of the ApplicationPublishPhase or is it some dead code ? There is no step associated to it.
• What are the possible way to pass information from a Task to another (here the BuilderTask to the PublishTask) ? There might some possibility in the CRs, but I can't see it.

I would really like your opinions and any input that could help me avoid any known pitfall.

cc @squakez @claudio4j @christophd @lburgazzoli @astefanutti @oscerd @tadayosi

[squakez]

@gansheer I don't have the full historical context to answer, but from what I've experimented in the past, the best approach is to go in iterations and make live past and new approaches for a short period of time. IMO, you can keep the "build" part the way it is now (we may rework this part in #4024). The "publish" part with the JIB strategy may be overlapping what we're already doing in the "build". What I think you could do is not to worry at this stage. I think you should consider the input coming from the build part (ie, the Maven project in a given directory), then, you should "decorate" it with the configuration required by JIB (ie, adding plugin and registry configuration). Finally you can run, again, a maven command but through the JIB plugin this time. Hope it helps.

[gansheer]

Thank for you input @squakez .

To explain the technical code problem. The maven command expect the following informations:

• GlobalSettings: KO computed by BuildTask
• UserSettings: KO computed by BuildTask
• SettingsSecurity: KO computed by BuildTask
• ExtraMavenOpts: KO computed by BuildTask (mainly Truststore)
• LocalRepository: OK since available from ip through localRepository
• AdditionalArguments : Partially available from ip through cliOptions
• AdditionalEntries: can't see any code using this one.

camel-k/pkg/util/maven/maven_command.go

Lines 157 to 166 in 4b5b5d3

	type Context struct {
	Path string
	ExtraMavenOpts []string
	GlobalSettings []byte
	UserSettings []byte
	SettingsSecurity []byte
	AdditionalArguments []string
	AdditionalEntries map[string]interface{}
	LocalRepository string
	}

Just to be clear : your advice is to only take what is in the /tmps/kit-xxxxxx directory by principle and ignore what is computed in the build task ?

[squakez]

So, what you're trying to do is to unmarshal what the "build" has provided before. IMO we could avoid that step and instead change the configuration structure on the directory files. Basically it would be a very low level "publisher" whose goal is to parse the pom xml, add some configuration with some raw editor, save it back and run the mvn compile accordingly.

[squakez]

Closed by #4680