Let Camel framework manage Kamelets

[squakez]

Kamelets resolution is still done in the old fashioned way:

[1] 2023-07-27 07:24:31,641 INFO  [org.apa.cam.k.lis.SourcesConfigurer] (main) Loading routes from: SourceDefinition{name='camel-k-embedded-flow', language='yaml', type='source', location='file:/etc/camel/sources/camel-k-embedded-flow.yaml', }
[1] 2023-07-27 07:24:31,678 INFO  [org.apa.cam.k.lis.SourcesConfigurer] (main) Loading routes from: SourceDefinition{name='log-sink', language='yaml', type='source', location='file:/etc/camel/sources/log-sink.yaml', }
[1] 2023-07-27 07:24:31,683 INFO  [org.apa.cam.k.lis.SourcesConfigurer] (main) Loading routes from: SourceDefinition{name='timer-source', language='yaml', type='source', location='file:/etc/camel/sources/timer-source.yaml', }

The runtime takes care to do the work which supposedly has to be done by the framework. This was fine in the early days of Kamelets, but since now Kamelets are a Camel thing, we need to let it manage them accordingly.

[lburgazzoli]

can you add more detail about then issue here ?
like what happen today under the hoods and what we expect to happen ?

[squakez]

Sure. Right now, the operator is not deploying a Kamelet, but it transform the Kamelet into a route template. For example:

k get cm timer-source-to-log-sink-kamelet-log-sink-template -o yaml
apiVersion: v1
data:
  content: |
    - routeTemplate:
        from:
          steps:
          - to:
              parameters:
                level: '{{?level}}'
                logMask: '{{?logMask}}'
                marker: '{{?marker}}'
                multiline: '{{?multiline}}'
                showAllProperties: '{{?showAllProperties}}'
                showBody: '{{?showBody}}'
                showBodyType: '{{?showBodyType}}'
                showCachedStreams: '{{?showCachedStreams}}'
                showExchangePattern: '{{?showExchangePattern}}'
                showHeaders: '{{?showHeaders}}'
                showProperties: '{{?showProperties}}'
                showStreams: '{{?showStreams}}'
              uri: log:{{loggerName}}
          uri: kamelet:source
        id: log-sink
kind: ConfigMap

This is working, but I think that we need instead to deploy the Kamelet spec and let the runtime to handle such Kamelet with the mechanism expected by the framework.

Hope it clarifies.

[lburgazzoli]

this can be done, however it would mean copying the entire kamelet inside a configmap ? not that as far as I recall, the camel-k operator also sets a number of properties

[squakez]

Yes, I'd expect something like that. Not sure about the implementations details, but ideally we should provide the same mechanism offered by the framework out of the box. We can try to work on it and revert if we discover this is not feasible for any reason.

[lburgazzoli]

Eventually the kamelet can get copied to the image during the build phase, this may also help achieving a truly immutable container image (which is something we must talk about anyway)

[squakez]

#4797 PR should solve this issue by moving the logic from a custom logic to something that is expected by the Camel runtime.

@lburgazzoli please have a look and also I'd like to share with you the results of some experiments I did before ending up with this solution (which may be temporary while we address in a proper way).

I tried to bundle the Kamelet spec as a resource to be included to the project at build time. However this is not actually feasible with the design we have in place. Basically we have a strong separation between the resource we manage at Integration level and the ones we manage at IntegrationKit and Build level. The IntegrationKit was designed to be Integration agnostic and there is no easy way to slip Integration resources into the Build. We did a minimal exception when we introduced the possibility to include sources for native builds (which I am reluctant to extend on other things): native are almost always meant to be built from scratch.

I think this separation of concerns was (and possibly still is) a good thing. If we want however to move in a different direction and be able to still have an incremental image driven by the kit, we could think on an additional step where the Integration have an additional container image which contains resource (such as kamelets or others like in #4639). Definitely not a top priority for now, but I'd like to have some opinion for the future.