Inspect all transitive dependencies of an integration #1777

doru1004 · 2020-10-22T15:35:42Z

Release Note

kind/feature

This patch adds more functionality to the kamel inspect subcommand. The default behavior of this command consists of outputting the top level dependencies. In this patch we enable the saving of the transitive dependencies for an integration in a specific location on the file system local to where the kamel inspect command was invoked. The transitive dependencies consists of all the jar files required for successfully running the integration.

lburgazzoli

added some notes

pkg/cmd/inspect.go

pkg/builder/builder_steps.go

doru1004 · 2020-10-28T15:01:41Z

@lburgazzoli I think I have addressed all the comments. Please let me know what you think. I've made some changes to where the functions that compute the integration dependencies and their sanitization are defined. Let me know if you agree with the move.

lburgazzoli · 2020-10-29T11:52:50Z

pkg/cmd/inspect.go

+		Long: `Output dependencies for a list of integration files. By default this command returns the
+top level dependencies only. When --all-dependencies is enabled, the transitive dependencies
+will be generated by calling Maven and then copied into the directory pointed to by the
+--dependencies-directory flag.`,


I'm not very sure about this. Would be useful to just get the list of dependencies without having to copy them on a folder ?

@nicolaferraro what do you think ?

doru1004 · 2020-11-02T17:34:14Z

@lburgazzoli I have modified the patch to only print transitive dependencies. Please let me know if you have any further comments.

nicolaferraro · 2020-11-04T16:44:00Z

pkg/util/defaults/defaults.go

@@ -42,4 +42,7 @@ const (

 	// ImageName --
 	ImageName = "docker.io/apache/camel-k"
+
+	// CamelVersion --
+	CamelVersion = "3.6.0"


This is a readonly file, which is generated from Makefile options. We've removed the need to specify the Camel version, because a runtime version implies only one Camel version and one Quarkus version.

Instead of hardcoding it here, you can load the CamelCatalog for the current runtime version (it's embedded in the /deploy virtual file system) and retrieve it from the metadata section (

camel-k/deploy/camel-catalog-1.5.0-quarkus.yaml

Line 34 in c79dc9b

camel.version: 3.5.0

).

Done, thanks for the feedback.

nicolaferraro

That's cool @doru1004 .

The only thing strange I see is that the output looks like:

[nferraro@localhost camel-k]$ kamel inspect examples/simple.groovy --all-dependencies --output=yaml
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------
[INFO] Building camel-k-integration 1.3.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- camel-k-maven-plugin:1.5.0:generate-dependency-list (default-cli) @ camel-k-integration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.856 s
[INFO] Finished at: 2020-11-05T15:24:46+01:00
[INFO] ------------------------------------------------------------------------
dependencies:
- /home/nferraro/.m2/repository/org/apache/camel/k/camel-k-loader-groovy/1.5.0/camel-k-loader-groovy-1.5.0.jar
- /home/nferraro/.m2/repository/org/apache/camel/k/camel-k-runtime-core/1.5.0/camel-k-runtime-core-1.5.0.jar

I.e. the maven part should be probably removed from the output. Wdyt?

nicolaferraro · 2020-11-05T14:28:06Z

That's cool @doru1004 .

The only thing strange I see is that the output looks like:

[nferraro@localhost camel-k]$ kamel inspect examples/simple.groovy --all-dependencies --output=yaml
[INFO] Scanning for projects...
[INFO] 
[INFO] ---------< org.apache.camel.k.integration:camel-k-integration >---------
[INFO] Building camel-k-integration 1.3.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- camel-k-maven-plugin:1.5.0:generate-dependency-list (default-cli) @ camel-k-integration ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.856 s
[INFO] Finished at: 2020-11-05T15:24:46+01:00
[INFO] ------------------------------------------------------------------------
dependencies:
- /home/nferraro/.m2/repository/org/apache/camel/k/camel-k-loader-groovy/1.5.0/camel-k-loader-groovy-1.5.0.jar
- /home/nferraro/.m2/repository/org/apache/camel/k/camel-k-runtime-core/1.5.0/camel-k-runtime-core-1.5.0.jar

I.e. the maven part should be probably removed from the output. Wdyt?

Or at least shown only when there's an error

doru1004 · 2020-11-05T14:31:21Z

I.e. the maven part should be probably removed from the output. Wdyt?

The maven needs to be run to fetch the dependencies. I'm guessing your concern is regarding whether the output of that command is shown or not. Is that correct?

pkg/cmd/inspect.go

nicolaferraro · 2020-11-05T14:36:43Z

I.e. the maven part should be probably removed from the output. Wdyt?

The maven needs to be run to fetch the dependencies. I'm guessing your concern is regarding whether the output of that command is shown or not. Is that correct?

Yes, it's only about the output. I imagine that external tools that are going to process the output of that command may get confused.

doru1004 · 2020-11-05T14:45:15Z

After changing to use quarkus I'm getting this type errors when inspecting a simple integration:

[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-core:jar is missing. @ line 37, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-kafka:jar is missing. @ line 41, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-slack:jar is missing. @ line 45, column 17

Any thoughts?

nicolaferraro · 2020-11-05T15:08:31Z

After changing to use quarkus I'm getting this type errors when inspecting a simple integration:

[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-core:jar is missing. @ line 37, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-kafka:jar is missing. @ line 41, column 17
[ERROR] 'dependencies.dependency.version' for org.apache.camel.quarkus:camel-quarkus-slack:jar is missing. @ line 45, column 17

Any thoughts?

I think you need to add the quarkus bom as it's done in the builder

doru1004 · 2020-11-05T16:49:42Z

@nicolaferraro I resolved the error. The change involves moving to running with Quarkus specific project building and dependency resolution routines.

doru1004 · 2020-11-05T22:02:56Z

@nicolaferraro I briefly had another error where a warning would be printed when running the maven command. It turned out to be an error on my side as I was using the wrong runtime dependency. Can confirm that no other output than the list of dependencies is being printed.

doru1004 mentioned this pull request Oct 22, 2020

Add a cli subcommand to inspect dependencies required by an Integration #1738

Closed

lburgazzoli requested changes Oct 26, 2020

View reviewed changes

pkg/cmd/inspect.go Outdated Show resolved Hide resolved

pkg/cmd/inspect.go Outdated Show resolved Hide resolved

pkg/cmd/inspect.go Outdated Show resolved Hide resolved

pkg/builder/builder_steps.go Outdated Show resolved Hide resolved

doru1004 force-pushed the inspect-all branch from 1257222 to da007e7 Compare October 27, 2020 18:52

lburgazzoli reviewed Oct 29, 2020

View reviewed changes

doru1004 force-pushed the inspect-all branch from f94f485 to 543fa81 Compare November 2, 2020 17:32

doru1004 added 14 commits November 3, 2020 10:30

Support computing and outputting of transitive dependencies.

b6b03cf

Move dependencies to dependencies folder.

19fe055

Clean-up.

3d06a4f

Remove double printing of top level dependencies.

198c114

Fix default versions.

c0f4a31

Add capability to process several additional dependencies flags.

70df78d

Only print top-level dependencies if they are requested.

70639c6

Fix after rebase.

eca2201

Update Camel version.

bd0a41f

Remove workspace directory. Make maven directory temporary.

4f178c4

Refactor output of dependencies.

3a6c182

Move builder independent functions to utils.

ede3c9e

Only print transitive dependencies.

c2b01a1

Update comment.

934df79

doru1004 force-pushed the inspect-all branch from 70d4af9 to 934df79 Compare November 3, 2020 15:30

nicolaferraro requested changes Nov 4, 2020

View reviewed changes

Remove hardcoded camel runtime version.

29bf6d5

nicolaferraro approved these changes Nov 5, 2020

View reviewed changes

nicolaferraro reviewed Nov 5, 2020

View reviewed changes

pkg/cmd/inspect.go Outdated Show resolved Hide resolved

nicolaferraro reviewed Nov 5, 2020

View reviewed changes

pkg/cmd/inspect.go Outdated Show resolved Hide resolved

Use default and quarkus runtime.

4f16866

Move to quarkus.

8ef2178

doru1004 added 2 commits November 5, 2020 14:25

Make output of maven quiet for local runs.

aaed184

Remove comment.

33fd698

nicolaferraro approved these changes Nov 5, 2020

View reviewed changes

nicolaferraro merged commit 04d7909 into apache:master Nov 5, 2020

nicolaferraro mentioned this pull request Dec 22, 2020

Release 1.3.0 #1867

Closed

christophd mentioned this pull request Jan 19, 2024

fix(#5047): Update Quarkus service discovery setting for data type transformer #5055

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inspect all transitive dependencies of an integration #1777

Inspect all transitive dependencies of an integration #1777

doru1004 commented Oct 22, 2020

lburgazzoli left a comment

doru1004 commented Oct 28, 2020

lburgazzoli Oct 29, 2020

doru1004 commented Nov 2, 2020

nicolaferraro Nov 4, 2020

doru1004 Nov 4, 2020

nicolaferraro left a comment

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

doru1004 commented Nov 5, 2020

Inspect all transitive dependencies of an integration #1777

Inspect all transitive dependencies of an integration #1777

Conversation

doru1004 commented Oct 22, 2020

lburgazzoli left a comment

Choose a reason for hiding this comment

doru1004 commented Oct 28, 2020

lburgazzoli Oct 29, 2020

Choose a reason for hiding this comment

doru1004 commented Nov 2, 2020

nicolaferraro Nov 4, 2020

Choose a reason for hiding this comment

doru1004 Nov 4, 2020

Choose a reason for hiding this comment

nicolaferraro left a comment

Choose a reason for hiding this comment

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

nicolaferraro commented Nov 5, 2020

doru1004 commented Nov 5, 2020

doru1004 commented Nov 5, 2020