Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven #781

LorenzoBettini · 2022-03-16T18:01:49Z

I've just found a problem in my Maven artifacts released to Central with Tycho 2.27 (maybe it's due to the new https://github.com/eclipse/tycho/blob/master/RELEASE_NOTES.md#mixed-reactor-build-support ?)

For my projects (Xtext projects, but I think that has nothing to do with my problem), I've always released with Tycho both p2 sites and some Maven artifacts to Maven central (they are Eclipse bundles but deployed to Maven central). For such projects, I've always repeated in the POM the Maven dependencies, since, when consumed by pure Maven, the MANIFEST dependencies are not taken into consideration.

This is an example of an artifact that is a bundle and it's deployed to Maven central as well

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <parent>
    <groupId>io.github.lorenzobettini.edelta</groupId>
    <artifactId>edelta.parent</artifactId>
    <version>2.9.0-SNAPSHOT</version>
  </parent>
  <artifactId>edelta.lib</artifactId>
  <packaging>eclipse-plugin</packaging>

  <dependencies>
    <dependency>
      <groupId>log4j</groupId>
      <artifactId>log4j</artifactId>
    </dependency>
    <dependency>
      <groupId>org.eclipse.xtext</groupId>
      <artifactId>org.eclipse.xtext.xbase.lib</artifactId>
    </dependency>
    <dependency>
      <groupId>org.eclipse.emf</groupId>
      <artifactId>org.eclipse.emf.ecore.xmi</artifactId>
    </dependency>
    <dependency>
      <groupId>org.eclipse.emf</groupId>
      <artifactId>org.eclipse.emf.ecore</artifactId>
    </dependency>
    <dependency>
      <groupId>org.eclipse.emf</groupId>
      <artifactId>org.eclipse.emf.common</artifactId>
    </dependency>
  </dependencies>
</project>

the version numbers are configured in the dependency management section of the parent.

This is the POM of 2.8.0 released with Tycho 2.26: https://repo1.maven.org/maven2/io/github/lorenzobettini/edelta/edelta.lib/2.8.0/edelta.lib-2.8.0.pom which looks fine.

If consumed in a pure Maven project, transitive dependencies are correctly resolved (see that from Eclipse):

After I switched to Tycho 2.27, things do not seem to work anymore...

For example, this is the POM of 2.9.0 released with Tycho 2.27:
https://repo1.maven.org/maven2/io/github/lorenzobettini/edelta/edelta.lib/2.9.0/edelta.lib-2.9.0.pom

This contains the compile scope dependencies, but then it contains lots of other stuff (generated by Tycho I guess?).

The problem is that if I try to consume this artifact version from Maven, the artifact is resolved, but not the transitive dependencies (which of course it's likely to generate lots of compilation errors, see the error marker in the Java code, even if I run Maven from the command line):

Is that related to the new https://github.com/eclipse/tycho/blob/master/RELEASE_NOTES.md#mixed-reactor-build-support ?

Should I change some configurations in my POMs?

I had a look at https://github.com/eclipse/tycho/blob/master/tycho-its/projects/mixed.reactor/pom.xml and I see the use of "consider" (which I never had to use in the past) and also this https://github.com/eclipse/tycho/blob/master/tycho-its/projects/mixed.reactor/pom.xml#L68 configuration of the tycho-p2-plugin...

Thanks in advance

LorenzoBettini · 2022-03-16T18:04:08Z

Sorry, I forgot to mention that of course the problem can be reproduced also by installing the artifact locally with install and then trying to resolve it from another project.

laeubi · 2022-03-17T04:46:20Z

This has nothing to do with mixed reactors but with the new generation of the consumer pom what should hopefully no longer require to manage the pom manually.

If you think something is generated wrong please let us know
If you still want to manage your pom manually (as before) you can set skipPomGeneration in the tycho-packaging:update-consumer-pom

LorenzoBettini · 2022-03-17T17:05:18Z

@laeubi, thank you for the hint! Setting skipPomGeneration to true has fixed the problem!

But...

This new goal has not been documented at all, has it?

I seem to understand that this new goal should avoid having to set dependencies in a POM manually? But from my experiments, it does not help... remember that I'm talking about POMs of bundles that should also be consumed as Maven artifacts. That's why I've always had to specify the dependencies in the POM (basically duplicating what's inside MANIFEST.MF). This new goal does not help with that respect...

This new goal (enabled by default) completely breaks the POMs that should be consumed as Maven artifacts. As I detailed in the initial issue, Maven can resolve the artifact itself, but not its transitive dependencies.

Another symptom of the problem I initially reported is that after you installed your artifacts to the local repository, performing a standard Maven build (no Tycho) leads to

[WARNING] The POM for io.github.lorenzobettini.edelta:edelta.lib:jar:2.9.0-SNAPSHOT is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details

This speaks for itself: the produced POM is not valid, and in fact, transitive dependencies are not available.

Summarizing skipPomGeneration solved the problem, but the goal does seem to break a few things.

Or am I missing something?

laeubi · 2022-03-17T17:26:13Z

This new goal has not been documented at all, has it?

It is documented here https://www.eclipse.org/tycho/sitedocs/tycho-packaging-plugin/update-consumer-pom-mojo.html

But from my experiments, it does not help... remember that I'm talking about POMs of bundles that should also be consumed as Maven artifacts.

That's what it is meant for, all required dependencies are already computed by Tycho and added to the maven model but not included in the resulting artifact (until Tycho 2.7.0).

This new goal does not help with that respect...

While development it was tested with only a small subset but for that cases it worked like designed

This speaks for itself: the produced POM is not valid

Please tell what is "not valid" and we can fix this, its hard to guess from the provided error/warning whats wrong here.

LorenzoBettini · 2022-03-17T17:56:34Z

This new goal has not been documented at all, has it?

It is documented here https://www.eclipse.org/tycho/sitedocs/tycho-packaging-plugin/update-consumer-pom-mojo.html

Sorry, I meant it wasn't announced in the release...

But from my experiments, it does not help... remember that I'm talking about POMs of bundles that should also be consumed as Maven artifacts.

That's what it is meant for, all required dependencies are already computed by Tycho and added to the maven model but not included in the resulting artifact (until Tycho 2.7.0).

Dependencies like required bundles in MANIFEST? That doesn't seem to be the case in my first experiments, but I can look further

This new goal does not help with that respect...

While development it was tested with only a small subset but for that cases it worked like designed

By tested did you:

installed the resulting artifacts and POMs in the local repo
tried to consume the artifacts as plain Maven artifacts?

This speaks for itself: the produced POM is not valid

Please tell what is "not valid" and we can fix this, its hard to guess from the provided error/warning whats wrong here.

That was the real problem for me: I have no clue why Maven considers the POM invalid... that's only the message I get during the Maven build...

laeubi · 2022-03-17T18:49:32Z

Dependencies like required bundles in MANIFEST?

Everything that is also shown by dependency:list

That doesn't seem to be the case in my first experiments, but I can look further

Please do so and report any issue back without feedback we can't improve this feature.

By tested did you

It was tested by some user-stories including installing in the local repository and consuming it e.g. in the maven-target location what uses standard maven resolving. The issue here is that the pom is considered invalid (for what we don't know the reason until now).

So if you think there is a general problem, the best would be to provide an integration-test to demonstrate the issue, then we can fix it and you are safe that it does not break in the next version of Tycho.

I have no clue why Maven considers the POM invalid..

I have no idea either, maybe running with -X reveals some details. Inserting the pom inside a real project the only thing that seems where maven complains is

<dependency>
    <groupId>p2.p2.installable.unit</groupId>
    <artifactId>org.eclipse.rcp_root</artifactId>
    <version>4.22.0.v20211124-1800</version>
    <type>p2-installable-unit</type>
    <scope>system</scope>
    <optional>false</optional>
  </dependency>

as it is marked as system without a path, if this is why maven considers the whole pom as invalid I have no clue.

LorenzoBettini · 2022-03-17T19:55:52Z

@laeubi ok, I'll try to create a small reproducing problem.

Can this somehow related to the attach artifact of the other tycho plugin also? Those goals are not documented IIRC.

laeubi · 2022-03-18T05:14:08Z

@LorenzoBettini just to make sure what maven version are you using for tests?

laeubi · 2022-03-18T05:31:58Z

enable debug logging for more details

Have you enabled -X for further details as suggested by the warning?

laeubi · 2022-03-18T05:36:54Z

I also have created https://issues.apache.org/jira/browse/MNG-7436 maybe the behavior could be improved.

Signed-off-by: Christoph Läubrich <[email protected]>

LorenzoBettini · 2022-03-18T07:59:35Z

@laeubi I still haven't worked on this, but a few updates:

I'm using both 3.8.1 embedded in m2e and 3.8.4 (through the Maven wrapper)

Could you please tell me the link to the integration test of the user story you were telling me about? I might use that as a starting point.

I'm still wondering how Tycho can insert a Maven dependency in the POM starting from a required bundle: how does it determine the GAV coordinates of the Maven artifact corresponding to the required bundle?

laeubi · 2022-03-18T09:00:53Z

I'm still wondering how Tycho can insert a Maven dependency in the POM starting from a required bundle: how does it determine the GAV coordinates of the Maven artifact corresponding to the required bundle?

If the artifact is consumed directly from maven it uses the coordinates there, if it is a reactor project, it uses the reactor coordinates, if the P2 matadata contains the maven GAV it is determined from there and in all other case it adds an (optional) dependency derived from the type and IU id.

I have submitted a patch that hopefully fixes the "invalid pom" warning in tycho 3.0.0-SNAPSHOT.

LorenzoBettini · 2022-03-18T09:17:03Z

but why did you close this issue?
Is this considered fixed?

laeubi · 2022-03-18T09:52:16Z

As soon as the build is finished you can try out the current tycho snapshot build. If there are still issues we can reopen, or create more specific issues.

laeubi · 2022-03-18T10:09:18Z

I'm still wondering how Tycho can insert a Maven dependency in the POM starting from a required bundle: how does it determine the GAV coordinates of the Maven artifact corresponding to the required bundle?

By the way if you are interested in the details you can take a look here:

LorenzoBettini · 2022-03-18T17:49:49Z

As soon as the build is finished you can try out the current tycho snapshot build. If there are still issues we can reopen, or create more specific issues.

OK, I'll try, but just to make sure I understand: there is an issue in version 2.7.0 that basically breaks transitive dependencies in Maven artifacts unless skipPomGeneration is used, is that right?

laeubi · 2022-03-19T05:25:21Z

Not in general but:

If you include the pom in the output jar -and-
Your bundle either contains a "p2 installable unit" like in your example -or-
Your bundle references another one that has an embedded jar

this should be now fixed in 3.0.0-SNAPSHOT.

Signed-off-by: Christoph Läubrich <[email protected]>

SimonCockx · 2022-10-05T11:07:11Z

I also encountered this issue while upgrading from Tycho 2.5.0 to 2.7.5. Building the project worked fine, but plain Maven projects that relied on this project suddenly failed to find some transitive dependencies.

laeubi · 2022-10-05T11:12:11Z

Please try out Tycho 3.0

LorenzoBettini · 2022-10-05T11:13:38Z

@laeubi Oh, so that is not the default behavior anymore?

laeubi · 2022-10-05T11:16:25Z

Tycho 3.0 has some improvements here, and Tycho 3.1.0-SNAPSHOT has even a way to validate the pom just in case you care about your maven consumers getting complete dependency info:

https://github.com/eclipse-tycho/tycho/blob/master/RELEASE_NOTES.md#new-maven-dependency-consistency-check

SimonCockx · 2022-10-05T11:51:13Z

@laeubi For now I'm tied to Java 11, but I'll try it out once I get to Java 17, thanks.

laeubi · 2022-10-05T11:52:51Z

You only need to run with Java 17 you still can compile with/for java 11

SimonCockx · 2022-10-05T11:54:01Z

Aha, that's a misunderstanding from my side. I'll look into it. Cheers!

SimonCockx · 2022-10-05T16:02:10Z

Same issue in 3.0.0. Any specific info you might need? I'll try to make a reproducer tomorrow.

laeubi mentioned this issue Mar 17, 2022

Duplicate p2 prefix for p2-installable-unit #784

Open

LorenzoBettini mentioned this issue Mar 17, 2022

Fix deployed Maven artifacts LorenzoBettini/edelta#408

Closed

laeubi mentioned this issue Mar 18, 2022

Bump maven-version from 3.8.4 to 3.8.5 #789

Merged

laeubi added a commit that referenced this issue Mar 18, 2022

Fix #781 - all p2 items and embedded jars must be considered optional

3907dd6

Signed-off-by: Christoph Läubrich <[email protected]>

laeubi closed this as completed in bb105b0 Mar 18, 2022

LorenzoBettini changed the title ~~Maven artifacts deployed with Tycho 2.27 are resolved without transitive dependencies by Maven~~ Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven Mar 18, 2022

LorenzoBettini mentioned this issue Mar 18, 2022

Error when using Xsemantics in maven/tycho build eclipse/xsemantics#156

Closed

LorenzoBettini mentioned this issue Mar 21, 2022

Fix deployed Xklaim Maven artifacts LorenzoBettini/xklaim#39

Closed

laeubi added this to the 2.7.1 milestone Mar 30, 2022

laeubi added a commit that referenced this issue Mar 30, 2022

Fix #781 - all p2 items and embedded jars must be considered optional

da34d2f

Signed-off-by: Christoph Läubrich <[email protected]>

LorenzoBettini mentioned this issue Jun 21, 2022

FJ: xtext-maven-plugin fails in Xtext 2.27.0 eclipse/xsemantics#158

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven #781

Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven #781

LorenzoBettini commented Mar 16, 2022

LorenzoBettini commented Mar 16, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 19, 2022

SimonCockx commented Oct 5, 2022

laeubi commented Oct 5, 2022

LorenzoBettini commented Oct 5, 2022

laeubi commented Oct 5, 2022

SimonCockx commented Oct 5, 2022

laeubi commented Oct 5, 2022

SimonCockx commented Oct 5, 2022

SimonCockx commented Oct 5, 2022

Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven #781

Maven artifacts deployed with Tycho 2.7 are resolved without transitive dependencies by Maven #781

Comments

LorenzoBettini commented Mar 16, 2022

LorenzoBettini commented Mar 16, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 17, 2022

LorenzoBettini commented Mar 17, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 18, 2022

laeubi commented Mar 18, 2022

LorenzoBettini commented Mar 18, 2022

laeubi commented Mar 19, 2022

SimonCockx commented Oct 5, 2022

laeubi commented Oct 5, 2022

LorenzoBettini commented Oct 5, 2022

laeubi commented Oct 5, 2022

SimonCockx commented Oct 5, 2022

laeubi commented Oct 5, 2022

SimonCockx commented Oct 5, 2022

SimonCockx commented Oct 5, 2022