Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong version of emf.common during exec-maven-plugin and xtext-maven-plugin #2397

Closed
LorenzoBettini opened this issue Nov 3, 2022 · 140 comments

Comments

@LorenzoBettini
Copy link
Contributor

I'm trying to build one of my (involved) DSL with Xtext 2.29.0.M2. I've updated the TP and the POM (I'm using the xtext-dev-bom).

During the execution of exec-maven-plugin (for running MWE2) I get this error:

0    [org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main()] ERROR mf.mwe2.launch.runtime.Mwe2Launcher  - IS_OSGI_RUNNING
java.lang.NoSuchFieldError: IS_OSGI_RUNNING
	at org.eclipse.emf.ecore.impl.EPackageRegistryImpl.createGlobalRegistry(EPackageRegistryImpl.java:55)
	at org.eclipse.emf.ecore.EPackage$Registry.<clinit>(EPackage.java:75)
	at org.eclipse.emf.mwe2.language.Mwe2StandaloneSetupGenerated.createInjectorAndDoEMFRegistration(Mwe2StandaloneSetupGenerated.java:38)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:75)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:36)
	at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:254)
	at java.base/java.lang.Thread.run(Thread.java:829)

Similarly, when running xtext-maven-plugin:

Execution default of goal org.eclipse.xtext:xtext-maven-plugin:2.29.0.M2:generate
failed: An API incompatibility was encountered while executing
org.eclipse.xtext:xtext-maven-plugin:2.29.0.M2:generate:
java.lang.NoSuchFieldError: IS_OSGI_RUNNING

From what I understand, a new version of emf.ecore is used but an older version of emf.common, where that static field is not yet available.

In fact, version 2.20 of emf.ecore used to execute

  public static EPackage.Registry createGlobalRegistry()
  {
    try
    {
      String className = System.getProperty("org.eclipse.emf.ecore.EPackage.Registry.INSTANCE");
      if (className == null)
      {
        if (EcorePlugin.getDefaultRegistryImplementation() != null)
        {
          return EcorePlugin.getDefaultRegistryImplementation();
        }
        else if (!EMFPlugin.IS_ECLIPSE_RUNNING)
...

while the version 2.28 of emf.ecore (which is the one being used) does

  public static EPackage.Registry createGlobalRegistry()
  {
    try
    {
      String className = System.getProperty("org.eclipse.emf.ecore.EPackage.Registry.INSTANCE");
      if (className == null)
      {
        if (EcorePlugin.getDefaultRegistryImplementation() != null)
        {
          return EcorePlugin.getDefaultRegistryImplementation();
        }
        else if (!EMFPlugin.IS_OSGI_RUNNING)
...

The xtext-dev-bom specifies 2.17.0 for emf.common, which is extremely old. It looks like that version of emf.common is used together with emf.ecore 2.28 and BANG.

I'm not doing strange thing with the TP nor with the POM (hopefully).

If I redefine in my parent's POM dependencyManagement section the version of emf.common

			<dependency>
				<groupId>org.eclipse.emf</groupId>
				<artifactId>org.eclipse.emf.common</artifactId>
				<version>2.26.0</version>
			</dependency>

then mwe2 runs fine.

Similarly, I have to make this dependency explicit in the pluginManagement configuration of xtext-maven-plugin

				<plugin>
					<groupId>org.eclipse.xtext</groupId>
					<artifactId>xtext-maven-plugin</artifactId>
					<version>${xtextVersion}</version>
					<dependencies>
						<dependency>
							<groupId>org.eclipse.emf</groupId>
							<artifactId>org.eclipse.emf.common</artifactId>
							<version>2.26.0</version>
						</dependency>
					</dependencies>
				</plugin>

then also this plugin runs fine.

Why is the version of emf.common is so old in the BOM? Shouldn't the versions of emf in the BOM be updated?

@szarnekow
Copy link
Contributor

Shouldn't the versions of emf in the BOM be updated?

Yes, please do.

@LorenzoBettini
Copy link
Contributor Author

@szarnekow OK, I'll do that ASAP, using the latest and greatest versions of EMF, is that OK? Or is there a minimal version we'd like to support? In any case, at least for mwe2 and xtext-maven-plugin we should get to the latest versions (or force emf.ecore to be a lower version?)

I've just also realized that the BOM is in xtext-lib, not in xtext-core... we can check whether we can move this issue there.

@cdietrich
Copy link
Member

cdietrich commented Nov 4, 2022

The problem is also that a ton of ecore files and genmodels and also code needs to be updated and I don’t have the time to bring this through until M3

I wonder how zoo end up with a newer emf code when we use 2.20 as default everywhere

@LorenzoBettini
Copy link
Contributor Author

@cdietrich so should I update the EMF versions in the BOM? the ecore and genmodel files should be OK like that anyway, shouldn't they?

@szarnekow
Copy link
Contributor

ecore files and genmodels and also code needs to be updated

These indeed might contain EMF versions in there, but that's merely for the code generation. The generated code itself is backwards / forwards compatible. I think we are good to "just" make the EMF versions consistent in the BOMs, as @LorenzoBettini mentioned 28 seconds ago

@cdietrich
Copy link
Member

This is not enough
All the ecore/be models and also a lot of stuff needs update
I am also not use if this version you want to update to is in eclipse 2022-03 which is the minimal target platform

i still wonder how zog end up with bad code in the first place
Does emf not support the compatibility for the version speechified in ecore/genmodel

@LorenzoBettini
Copy link
Contributor Author

Just for curiosity: has no one else experienced such a problem?

@cdietrich
Copy link
Member

I don’t understand. The emf versions in the bom are consistent, aren’t they ?

@LorenzoBettini
Copy link
Contributor Author

they are, but too old; somehow during mwe2 and xtext-maven-plugin a newer version of emf.ecore is used, which relies on a new version of emf.common (but the old version of emf.common is dragged in, and I get the problem I described).

@cdietrich
Copy link
Member

This is what I understood and so we would need a consistent bump not only in the bom but also in the rest of the code.

so the question is why does emf 2.28 not produce 2.20 compatible code or does your ecore/genmodel not specify emf 2.20
(Can’t check on my iPad)

@szarnekow
Copy link
Contributor

they are, but too old; somehow during mwe2 and xtext-maven-plugin a newer version of emf.ecore is used

Doesn't this indicate that they aren't? Or do you have managed EMF dependencies somewhere outside the bom?

@szarnekow
Copy link
Contributor

so the question is why does emf 2.28 not produce 2.20 compatible code

It does. Problem appears to be that emf.ecore has a dependency on emf.common but doesn't have exact lower version bounds in the manifest or the consumed emf pom has wrong lower version bounds or the xtext bom overrides these.

@cdietrich
Copy link
Member

But where is the problem
Our bom lists all emf 2.20 stuff, including emf.common

@cdietrich
Copy link
Member

cdietrich commented Nov 4, 2022

And I don’t know why problem is not visible
In https://github.com/xtext/maven-xtext-example
Which is the only real world example of Xtext maven plugin I know of

@LorenzoBettini
Copy link
Contributor Author

@cdietrich @szarnekow I'll have to investigate further, but I'm pretty sure the problem is, like it happened in the past, the xtext-maven-plugin used in a reactor with Tycho. As it happened in the past, the problems with xtext-maven-plugin did not take place in a pure Maven build, but only in a Maven/Tycho build (due to the mixture of p2 bundles and Maven JARs). I would say that the xtext-maven-plugin alters the final classpath generated by Tycho (by using a recent version of emf.ecore without a recent version of emf.common) and also breaks mwe2 when executed by exec-maven-plugin.

I'm not blaming xtext-maven-plugin of course O:)

@cdietrich
Copy link
Member

Then the question would be why tycho does not pull in new versions of both plugins

@LorenzoBettini
Copy link
Contributor Author

By the way, in the repo you mention, the example project using the xtext-maven-plugin is built with pure Maven, Tycho is not involved. Also in the past, when I spotted problems with the plugin, I had problems in a Tycho reactor, not with pure Maven. Later I'll try to investigate further.

@LorenzoBettini
Copy link
Contributor Author

But the example project (where the xtext-maven-plugin is used), https://github.com/xtext/maven-xtext-example/tree/master/example-project, is not part of the reactor of the parent project. It's not built with Tycho, but with pure Maven. In fact, in the build.sh you first install all the bundles of the parent, and then you build the example-project alone.

@cdietrich
Copy link
Member

But you use tycho. Ok

@LorenzoBettini
Copy link
Contributor Author

No, to build the example-project (the one using xtext-maven-plugin) you don't use Tycho at all

@LorenzoBettini
Copy link
Contributor Author

Some updates: xtext-maven-plugin doesn't seem the culprit here because, even if I disable it in my reactor, the mwe2 step fails with the same error... I enabled debug information in the Tycho build and it looks, again, a problem with mixture of target platform and Maven bundles: these are the interesting parts of the reported "collected artifacts" in the order of collection:

org.eclipse.emf:org.eclipse.emf.common:jar:2.17.0:compile (OLD VERSION)

then

p2.eclipse.plugin:org.eclipse.emf.common:eclipse-plugin:2.26.0.v20220817-1401:system (NEW VERSION)

then

p2.eclipse.plugin:org.eclipse.emf.ecore:eclipse-plugin:2.28.0.v20220817-1401:system (NEW VERSION)

then

org.eclipse.emf:org.eclipse.emf.ecore:jar:2.20.0:compile (OLD VERSION)

So as you see, first the old version of emf.common is collected and then the newer version of emf.ecore is collected (before the old one), and that's the problem: the newer version of emf.ecore tries to access a new member of emf.common and it can't find it in the old version of emf.common.

@cdietrich
Copy link
Member

are deps in build.properties or inn manifest?
which ones are explicitely mentioned in the exec call?

@LorenzoBettini
Copy link
Contributor Author

That's the build.properties:

source.. = src/,\
           src-gen/,\
           xtend-gen/
bin.includes = model/generated/,\
               .,\
               META-INF/,\
               plugin.xml
bin.excludes = **/*.mwe2,\
               **/*.xtend
additional.bundles = org.eclipse.xtext.xbase,\
                     org.eclipse.xtext.common.types,\
                     org.eclipse.xtext.xtext.generator,\
                     org.eclipse.emf.codegen.ecore,\
                     org.eclipse.emf.mwe.utils,\
                     org.eclipse.emf.mwe2.launch,\
                     org.eclipse.emf.mwe2.lib,\
                     org.objectweb.asm,\
                     org.apache.commons.logging,\
                     org.apache.log4j

and this is the MANIFEST.MF:

Manifest-Version: 1.0
Automatic-Module-Name: io.github.lorenzobettini.tychotestlanguage
Bundle-ManifestVersion: 2
Bundle-Name: io.github.lorenzobettini.tychotestlanguage
Bundle-Vendor: My Company
Bundle-Version: 1.0.0.qualifier
Bundle-SymbolicName: io.github.lorenzobettini.tychotestlanguage; singleton:=true
Bundle-ActivationPolicy: lazy
Require-Bundle: org.eclipse.xtext,
 org.eclipse.xtext.xbase,
 org.eclipse.equinox.common;bundle-version="3.5.0",
 org.eclipse.emf.ecore,
 org.eclipse.xtext.xbase.lib;bundle-version="2.14.0",
 org.eclipse.xtext.util,
 org.eclipse.emf.common,
 org.antlr.runtime;bundle-version="[3.2.0,3.2.1)"
Bundle-RequiredExecutionEnvironment: JavaSE-1.8
Export-Package: io.github.lorenzobettini.tychotestlanguage.services,
 io.github.lorenzobettini.tychotestlanguage.tychotestlanguage.impl,
 io.github.lorenzobettini.tychotestlanguage.tychotestlanguage,
 io.github.lorenzobettini.tychotestlanguage.validation,
 io.github.lorenzobettini.tychotestlanguage,
 io.github.lorenzobettini.tychotestlanguage.serializer,
 io.github.lorenzobettini.tychotestlanguage.parser.antlr,
 io.github.lorenzobettini.tychotestlanguage.parser.antlr.internal,
 io.github.lorenzobettini.tychotestlanguage.tychotestlanguage.util,
 io.github.lorenzobettini.tychotestlanguage.scoping,
 io.github.lorenzobettini.tychotestlanguage.generator
Import-Package: org.apache.log4j

So it's the same as what the Xtext wizard creates. The difference is that this DSL is based on a parent POM. (the idea is to extract common configurations in a parent POM; something that I'm using for my DSLs and in the future I might propose as an Xtext feature).

I can only guess that due some ordering of plugins, Tycho takes dependencies in a different order and the EMF Maven artifacts with version ranges (instead of strict versions) do the rest...

As I said in the original post, I know how to fix that. However, I wondered if we (Xtext) can prevent these problems upstream.

@LorenzoBettini
Copy link
Contributor Author

Another suspect: https://github.com/eclipse/xtext-core/blob/master/org.eclipse.xtext.xtext.generator/build.gradle

I pinned all the platform dependencies in the past, which gave me headaches in a similar environment. I did not pin the EMF Maven dependencies because I did not realize they had version ranges. By looking at the p2 bundle of emf.codegen.ecore I see

image

so it requires a recent version of emf.ecore but not of emf.common.

The POM of the version of emf.codegen.ecore https://search.maven.org/artifact/org.eclipse.emf/org.eclipse.emf.codegen.ecore/2.20.0/jar does something similar:

    <dependency>
      <groupId>org.eclipse.emf</groupId>
      <artifactId>org.eclipse.emf.ecore</artifactId>
      <version>[2.20.0,3.0.0)</version>
    </dependency>

without an explicit pin in build.gradle for emf.ecore, a newer version will be taken, which will be mixed up with an older version of emf.common.

That's just a suspect. I'd have to patch xtext.xtext.generator, install it locally and see what happens.

@LorenzoBettini
Copy link
Contributor Author

@cdietrich @szarnekow some updated.

I'll try to explain my impression of the problem as clearly as possible, but there are few things involved.

I made another experiment (slightly different from forcing emf.common version in two places as I said in the original post #2397).

I added in the exec-maven-plugin configuration an explicit dependency on xtext.xtext.generator:

<dependencies>
	<dependency>
		<groupId>org.eclipse.xtext</groupId>
		<artifactId>org.eclipse.xtext.xtext.generator</artifactId>
		<version>${xtextVersion}</version>
	</dependency>
</dependencies>

This makes mwe2 run succeed. However, as I suspected, since the xtext.generator does NOT pin the emf.ecore and emf.common dependencies it drags newer versions:

Downloading from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.common/2.26.0/org.eclipse.emf.common-2.26.0.jar
Downloading from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.ecore/2.28.0/org.eclipse.emf.ecore-2.28.0.jar

But at least, it works because, besides the newer version of emf.ecore it also takes the newer version of emf.common.

The xtext-maven-plugin still fails. By the way, xtext-maven-plugin depends on xtext.xtext.generator of course.

So I pinned manually the dependencies to emf.ecore, emf.common and emf.ecore.xmi (yes, also the latter must be pinned) in the pom of xtext.xtext.generator in the .m2 cache. I know it's a dirty hack but it allows me to do such experiments without building a new version of xtext-core.

With this manual hack, xtext-maven-plugin works as well! And I can see that when exec plugin executes mwe2 it takes the pinned version.

This means that these 3 emf dependencies must be pinned in https://github.com/eclipse/xtext-core/blob/master/org.eclipse.xtext.xtext.generator/build.gradle

So I though that pinning the versions of emf.ecore in xtext.generator would be enough, and I tried to remove the explicit dependency to xtext.generator from the exec plugin configuration and... BANG! It goes back to the original problem! Why? The emf dependencies are now pinned, so it shouldn't happen again.

Another strange thing I was seeing during the maven/tycho run was that, even with all pinned dependencies in my hacked xtext.xtext.generator pom, maven metadata were still downloaded (without downloading any new jars), e.g., something like:

Downloaded from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.codegen/maven-metadata.xml (899 B at 28 kB/s)
Downloading from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.ecore.xmi/maven-metadata.xml
Downloaded from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.ecore.xmi/maven-metadata.xml (975 B at 30 kB/s)
Downloading from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.mwe.core/maven-metadata.xml
Downloaded from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.mwe.core/maven-metadata.xml (1.2 kB at 35 kB/s)
Downloading from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.mwe2.runtime/maven-metadata.xml
Downloaded from : https://repo.maven.apache.org/maven2/org/eclipse/emf/org.eclipse.emf.mwe2.runtime/maven-metadata.xml (1.2 kB at 35 kB/s)

In my experience in fixing dependency problems, when I see something like that, it means that "some one" does not pin dependencies. With my hacked version of xtext.generator, it cannot be xtext.generator. Since this happens only when running exec plugin (and not xtext maven plugin) I think the culprit is the only other participant: mwe2! By looking at its poms, I see that mwe2.runtime (on which mwe2.language depends) has emf.ecore as a required bundle in the MANIFEST.MF (from an OSGI point of view) but it has NO Maven dependency to emf in the POM. I guess that's the source of the problem.

Summarizing, I think we have to fix two things:

  • pin the additional emf dependencies in xtext.xtext.generator
  • pin emf dependencies in mwe2 poms

That's my impression.
What do you think?

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

Because the EMF updatesite was not produced from deployed maven artifacts.

@cdietrich
Copy link
Member

so its done only if maven-type="jar"?

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

so its done only if maven-type="jar"?

No it is done if <property name='maven-repository' value='id of repository'/> is given in the metadata.

@cdietrich
Copy link
Member

ok, and as it looks like this one gets inherited if an upstream p2 repo is consumed downstream. so there is no option to get rid of it.

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

ok, and as it looks like this one gets inherited if an upstream p2 repo is consumed downstream. so there is no option to get rid of it.

That's the purpose, and actually there is no point in "get rid of it" ... because it allows Tycho to use the real maven artifact instead of a (misused) system scoped dependency with generated maven coordinates.

The problem only arises because you want to mix released M3 artifacts with SNAPSHOT artifacts and even newer versions like used in the M3 build from a different library that you have pinned in your M3 release to an older version.

So from maven POV, either your M3 release is obviously not compatible with newer EMF releases using different techniques, or the emf dependency should not be a compile but a provided dependency.

And that's why pinning dependencies to fix downstream maven versionranges is bad and why I asked platform to fix that instead of trying to fix that in Tycho ;-)

@cdietrich
Copy link
Member

cdietrich commented Nov 16, 2022

but a provided dependency wont work in pure gradle or maven projects where there is no target platform

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

but a provided dependency wont work in pure gradle or maven projects where there is no target platform

That's just not true. It does work as specified by the maven inclusion rules that is that provided dependencies are not inherited (and that's it purpose), so a consumer simply need to require xtext and emf in the version they desire and think "should work"...

@cdietrich
Copy link
Member

but this requires 1000s of clients to be spoiled with exclude and include rules

@cdietrich
Copy link
Member

maybe we should abandon p2 and tycho for the exec part alltogether and have a own run workflow plugin to avoid the problem

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

Whatever suites... the usual maven way is to provide a bom for the usual lazy maven user that just only want everything so it works ...

@cdietrich
Copy link
Member

cdietrich commented Nov 16, 2022

we have a bom. but it does not work here...
https://search.maven.org/artifact/org.eclipse.xtext/xtext-dev-bom/2.29.0.M3/pom
the bom does not know anything about p2 maven coordinates

@laeubi
Copy link
Member

laeubi commented Nov 16, 2022

The bom is for the maven users where " provided dependency wont work" ... you can add emf there as a compile/provided/... dependency and other projects can import it.

@cdietrich
Copy link
Member

I am not firm in maven enhough to understand the proposal @LorenzoBettini cab you help out

@LorenzoBettini
Copy link
Contributor Author

Hi everyone! I haven't run away after opening Pandora's box ;)
it's just that I'm busy today.
I'll be back tomorrow to catch up on the conversation.

@LorenzoBettini
Copy link
Contributor Author

@LorenzoBettini adding the emf dependencies before the xtext ones in manifest and build.properties and have explicit p2 dependencies for both emf plugins in manifest and build.properties seems to help on my machine

I confirm that just putting emf.common on top of the required bundles fixes the problem: https://github.com/LorenzoBettini/issue-2000-example/pull/1/files

This looks like a kind of hack to me, though

LorenzoBettini referenced this issue in eclipse/xtext-core Nov 18, 2022

See #2000
This is the same but for org.eclipse.xtext

Signed-off-by: Lorenzo Bettini [email protected]
@LorenzoBettini
Copy link
Contributor Author

I don't know if it's because of eclipse/xtext-core#2021 but with the released Xtext 2.29.0, the problem went away in the example project LorenzoBettini/issue-2000-example#3

@LorenzoBettini
Copy link
Contributor Author

I think we can close this one now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants