Camel K could go offline

[squakez]

Ref #1721

I've done some experiment to try a Camel K installation entirely offline. I've developed a script that download the entire Camel Catalog set of dependencies, which would allow Camel K to work completely closed from the Internet. The build requires around half an hour to download the wide set of dependencies. It has to be tested, but the outcome is that we could produce a docker image of about 5 GB:

apache/camel-k                                                             2.0.0-SNAPSHOT          5ace9561ac83   2 minutes ago   4.56GB

This "fat" installation could be delivered beside the "normal" installation. Draft work available in ea2974d

[so-dewy]

Hello, at my work I need to install Camel K on cluster without internet and I'm trying to build the image locally to try out the work you've done. What should be done to make the kamel use this image at installation time?

[squakez]

If you use that work, you just need to make images. This command will push the image to the registry where your local docker daemon is connected (ie, you can docker image ls right after you made the images). You can run the operator if you follow the procedure provided in https://camel.apache.org/camel-k/next/contributing/local-development.html

[so-dewy]

Thanks for pointing to the make images, I was able to build the image locally with pre-downloaded dependencies. And install it to the Minikube by loading the image inside and then turning off internet:

 minikube image load docker.io/apache/camel-k:2.0.0-SNAPSHOT

But I wasn't able to create an integration because during it's creation it couldn't resolve parent POM org.apache:apache:pom:29 for org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT

[ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT: Could not transfer artifact org.apache:apache:pom:29 from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/org/apache/apache/29/apache-29.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 22, column 13
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-direct:jar is missing. @ line 23, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-http:jar is missing. @ line 27, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-jackson:jar is missing. @ line 31, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-json-validator:jar is missing. @ line 35, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-rabbitmq:jar is missing. @ line 39, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.k:camel-k-runtime:jar is missing. @ line 43, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-java-joor-dsl:jar is missing. @ line 47, column 17

I took a look around the pre-downloaded dependencies and noticed that there were org.apache:apache:pom lying around but just up to 28, so I changed 29 into 28 inside the dowloaded org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT.pom manually. It made some progress but still was trying to download more missing dependencies, there was quite a lot of them and it was too tedious to figure them out one by one, although I think eventually I would be able to make it work.

[ERROR] Some problems were encountered while processing the POMs:
[ERROR] Non-resolvable import POM: Could not transfer artifact io.quarkus.platform:quarkus-camel-bom:pom:2.16.0.Final from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/io/quarkus/platform/quarkus-camel-bom/2.16.0.Final/quarkus-camel-bom-2.16.0.Final.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 100, column 25
[ERROR] Non-resolvable import POM: Could not transfer artifact io.quarkus.platform:quarkus-bom:pom:2.16.0.Final from/to central (https://repo.maven.apache.org/maven2): transfer failed for https://repo.maven.apache.org/maven2/io/quarkus/platform/quarkus-bom/2.16.0.Final/quarkus-bom-2.16.0.Final.pom @ org.apache.camel.k:camel-k-runtime-bom:1.17.1-SNAPSHOT, /tmp/artifacts/m2/org/apache/camel/k/camel-k-runtime-bom/1.17.1-SNAPSHOT/camel-k-runtime-bom-1.17.1-SNAPSHOT.pom, line 107, column 25
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-direct:jar is missing. @ line 23, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-http:jar is missing. @ line 27, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-jackson:jar is missing. @ line 31, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-json-validator:jar is missing. @ line 35, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-rabbitmq:jar is missing. @ line 39, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.k:camel-k-runtime:jar is missing. @ line 43, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-java-joor-dsl:jar is missing. @ line 47, column 17

So what I decided to do is use a messy hack:

1. Use cluster with internet as staging ground for developing and testing integrations using the stock releases of camel-k, not my locally modified one.
2. Inside the camel-k-operator it stores all the dependencies needed for creating integrations in /tmp/artifacts.
3. After developing and creating integrations. Copy the whole directory from inside the pod to the host, it should contain dependencies required by now

kubectl cp default/CAMEL_K_OPERATOR_POD_NAME:/tmp/artifacts /your/host/path

4. Install the camel-k-operator inside the air-gapped cluster without internet, you need to point to your docker registry that is installed inside your cluster (I'm using nexus docker registry) and put docker.io/apache/camel-k:RELEASE_VERSION image inside it so that kamel can resolve it without internet. Also you need to put 'adoptopenjdk/openjdk11:slim' there too as it is used for creating integrations

kamel install --registry YOUR_REGISTRY:5000  --registry-auth-username USERNAME --registry-auth-password
PASSWORD --registry-insecure true --base-image YOUR_REGISTRY:5000/adoptopenjdk/openjdk11:slim

5. Move the directory to the air-gapped cluster and copy it inside the installed camel-k-operator pod at this location /tmp/artifcats.
6. After all the steps I was able to create integrations without internet!

As I understand /tmp/artifacts is a default location for the local maven repository if you don't provide one when doing kamel install. If you do provide it I think you could do the same thing by copying from staging maven repository to air-gapped one.

This feels hacky but at least I was able to make it work

[squakez]

Yeah, that procedure was a quick experiment to see how to fill the docker image and how much it would require in term of space. We need to fine tune in order for the operator to have the dependencies in the right directory. It is also advisable to run the build with mvn -o in order to make sure it does not go out looking for external dependencies. Your hack looks fine, so, while we make the feature available I guess you can make use of it.

[squakez]

BTW, feel free to work on the official feature if you want to. We're happy to receive any contribution!

[github-actions]

This issue has been automatically marked as stale due to 90 days of inactivity.
It will be closed if no further activity occurs within 15 days.
If you think that’s incorrect or the issue should never stale, please simply write any comment.
Thanks for your contributions!

[squakez]

I'm trying to refine the experiment script mentioned in the issue description, but I'm hitting the Maven issue https://issues.apache.org/jira/browse/MDEP-82 when we try to run the integration:

I'm using a base Camel K runtime project in order to try to go offline, but the maven procedure is not really getting all the dependencies expected:

$ mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:go-offline -Dmaven.repo.local=repo
...

$ mvn verify -o -Dmaven.repo.local=repo
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------
[INFO] Building camel-k-integration 2.0.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-resources-plugin:3.3.0:resources (default-resources) @ camel-k-integration ---
[WARNING] The POM for org.codehaus.plexus:plexus-interpolation:jar:1.26 is missing, no dependency information available
[WARNING] The POM for org.apache.maven.shared:maven-filtering:jar:3.3.0 is missing, no dependency information available
[WARNING] The POM for commons-io:commons-io:jar:2.11.0 is missing, no dependency information available
[WARNING] The POM for org.apache.commons:commons-lang3:jar:3.12.0 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.101 s
[INFO] Finished at: 2023-08-08T16:45:48+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources (default-resources) on project camel-k-integration: Execution default-resources of goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources failed: Plugin org.apache.maven.plugins:maven-resources-plugin:3.3.0 or one of its dependencies could not be resolved: Cannot access central (https://repo.maven.apache.org/maven2) in offline mode and the artifact org.codehaus.plexus:plexus-utils:jar:1.1 has not been downloaded from it before. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException