Gradle metadata missing version information

[brockj]

After upgrading gradle to 6.0 and the new metadata publishing we have noticed that dependencies aren't resolving due to missing version numbers.

Given the following build file

plugins {
    id 'java'
    id "maven-publish"
    id "io.spring.dependency-management" version "1.0.8.RELEASE"
}

group 'org.example'
version '1.0-SNAPSHOT'

sourceCompatibility = JavaVersion.VERSION_13

repositories {
    mavenCentral()
}

dependencies {
    implementation group: "org.apache.logging.log4j", name: "log4j-api"
}

dependencyManagement {

//    imports {
//        mavenBom "org.springframework.boot:spring-boot-dependencies:2.2.1.RELEASE"
//    }

    dependencies {
        dependency group: "org.apache.logging.log4j", name: "log4j-api", version: "2.12.1"
    }
}

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}

When generating the metadata file it produces a dependency section like this:

"dependencies": [
        {
          "group": "org.apache.logging.log4j",
          "module": "log4j-api"
        }
      ],

When we depend on this artifact from another artifact it fails to resolve the correct version of log4j-api unless we directly set a version number for it in the consuming build file. That is, we aren't getting the transitive version

The generated POM is also missing the version number, but there is a dependency management section in it that is used to define the version, so it works ok.

The only workaround we have so far is to not publish gradle metadata to our nexus and consume the POM instead.

We have found that if we specify the version directly (ie. don't use dependency management plugin) it produces a dependency section like this:

      "dependencies": [
        {
          "group": "org.apache.logging.log4j",
          "module": "log4j-api",
          "version": {
            "requires": "2.12.1"
          }
        }
      ],

Note that it includes the required version number and we are able to consume the artifact correctly.

[wilkinsona]

Thanks for the report. I'm not sure if this is an enhancement that needs to be implemented here, or something that Gradle should pick up automatically based on what this plugin's resolution strategy has done. What's your take on this please, @melix?

If you're publishing Gradle's own metadata then I'd strongly encourage you to look at Gradle's platform and enforced platform support as a replacement for this plugin.

[brockj]

Thanks for the tip @wilkinsona
It looks like platform dependencies should do what we want

[jjohannes]

Answering for @melix. :)
From Gradle's point of view the behavior is not surprising.

As @wilkinsona pointed out, you can basically do what you want with Gradle's built-in features now. The sample could look like this:

plugins {
    id 'java'
    id "maven-publish"
}

group 'org.example'
version '1.0-SNAPSHOT'

sourceCompatibility = JavaVersion.VERSION_13

repositories {
    mavenCentral()
}

dependencies {
    implementation group: "org.apache.logging.log4j", name: "log4j-api"

    // implementation platform("org.springframework.boot:spring-boot-dependencies:2.2.1.RELEASE")
    constraints {
        implementation group: "org.apache.logging.log4j", name: "log4j-api", version: "2.12.1"
    }
}

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
        }
    }
}

This will also publish the dependency constraint in the metadata. If you un-comment the platform dependency to the BOM, that will also be published and the BOM will automatically be pulled in by consumers as well.

Another option you have if you want versions to be published on the dependencies directly, is to publish the resolved dependency versions. I think that should also work for the original sample with the Spring dependency management plugin.

[brockj]

After looking at platforms it doesn't look like it fully meets what we want yet, or don't understand how to use it correctly.

For now we will go with disabling consumption of gradle metadata from our hosted repositories by adding a metatdataSources block to our repositories.

            metadataSources {
                mavenPom()
                ignoreGradleMetadataRedirection()
            }

Hopefully someone will in time provide examples of multi project builds using spring boot and gradle platforms

[wilkinsona]

Hopefully someone will in time provide examples of multi project builds using spring boot and gradle platforms

I don't think Platforms have any special requirements in a multi-project build. You should declare a dependency on the Platform in every project where you want it to be used and for every configuration that you want to use it. You can do that in a subprojects block, for example.

I've made a note in #211 about publication of the resolved dependency versions. Given that there are no other changes needed here, I am going to close this issue.

[jjohannes]

@brockj I am really curious what is blocking you from using Gradle Module Metadata and I am happy to point you at, or provide you with, additional examples. Unfortunately I am not sure what you mean by this:

Hopefully someone will in time provide examples of multi project builds using spring boot and gradle platforms

There should be nothing special to multi-projects nor to spring boot BOM that should require some special setup beyond what exist and what is documented.

Could you elaborate which issue you are still running in and/or which error you are facing that you can work around by disabling consumption of module metadata? Thanks a lot.

[brockj]

@jjohannes The issue we had was that it was publishing with the unresolved version numbers (ie. blank)

We use the dependency management plugin because that is what is in every spring boot example i have seen. It helps us manage groups of dependencies that do not offer a BOM (we currently don't produce BOMs for our own artifacts either)

I will setup a repo tomorrow to reproduce the issue we are having. I am pretty sure we can work around it with the gradle platforms, but it would mean a fair bit of rework on our projects

[brockj]

@jjohannes I have pushed a sample repo to
https://github.com/brockj/gradle-metata-versions

It is only a single project as it should be enough to show the issue.
We are using the dependencySet to share versions between related artifacts and then declaring them as dependencies. In our real projects, we declare the dependency set in a parent project so we don't have to re-declare the version in the sub projects. Adding version mapping fixed the publishing of the version, but i didn't test what would happen with snapshots. I assume you end up publishing the resolved version, not SNAPSHOT.

Still happy to leave the issue closed as we have workarounds and will slowly migrate our structure to use more standard Gradle features.

[jjohannes]

@brockj thank you for sharing the sample. Very much appreciated. I think in the discussion two new Gradle core concepts got mixed up platforms and dependency constraints.

• When you want to use the spring-boot-dependencies BOM, you declare a platform dependency implementation platform("..."). The platform provides dependency constraints. This replaces the "BOM import":

// Spring DM plugin
dependencyManagement {
  imports {
    mavenBom ...
  }
}

// Gradle core
dependencies {
  implementation platform(...)
}

• When you want to declare versions for which you don't have a BOM, you do not need to define your own platform, but you can use dependency constraints directly dependencies { constraints { ... } }.

// Spring DM plugin
dependencyManagement {
  dependencies {
    ...
  }
} 

// Gradle core
dependencies {
  constraints {
     ...
  }
}

I created a PR for your example repo to show it for that example. The resulting POM should be the same as before but the module file now contains dependencyConstraints { } blocks which are in a way the equivalent to the <dependencyManagement> block in the POM.

@wilkinsona You could investigate if you can wrap (parts of) your implementation around these new Gradle APIs and possibly get rid of a good chunk of your own POM-writing implementation in PomDependencyManagementConfigurer. Everything that's handled by Gradle natively and not by the plugin modifying the POM directly should work fine for both POM and Gradle metadata.

[JasonMing]

• you declare a platform dependency implementation platform("..."). The platform provides dependency constraints. This replaces the "BOM import":

@jjohannes Could you add a demo show how to implement import bom with overriding properties

dependencyManagement {
  imports {
    mavenBom("xxx") {
      bomProperty("xxx.version", "1.2.3")
    }
  }
}

[jjohannes]

@JasonMing Unfortunately, this is not (yet) supported by Gradle natively (gradle/gradle#9160). If you can provide a complete example of your use case, I can check if there is a good alternative solution or how you could best combine the Spring DM plugin with Gradle's features to get the best of both worlds.

[xomidar]

annotationProcessor("org.projectlombok:lombok")
not picking version if I use custom bom

    dependencies {
        implementation platform(com.custom:java-dependencies:${version})
    }
//Generates Error

   dependencyManagement.imports {
        mavenBom "com.custom:java-dependencies:${version}"
   }
//Works Fine

is there any way to make annotationProcessor("org.projectlombok:lombok") work using platform ?

[melix]

A platform is not applied to the annotation processor classpath by default. You need to declare it explicitly:

dependencies {
        annotationProcessor platform(com.custom:java-dependencies:${version})
    }

[xomidar]

A platform is not applied to the annotation processor classpath by default. You need to declare it explicitly:

dependencies {
        annotationProcessor platform(com.custom:java-dependencies:${version})
    }

Thank you :)

[alexei-osipov]

It seems that version metadata still missing in Gradle module metadata (checked with Gradle 6.7.1). @wilkinsona Is there a dedicated ticket for that issue? #211 does not have mentions of the missing version metadata in Gradle-s modules.

This works as a temporary fix but it would be nice to get the issue fixed.

metadataSources {
                mavenPom()
                ignoreGradleMetadataRedirection()
}

[wilkinsona]

@alexei-osipov Ideally, if you're interested in Gradle's module metadata you should use Gradle's native platform support rather than this plugin. If that's not possible then you may be able to configure Gradle to use the resolution result when determining the versions in the metadata. That's what the comment made on #211 is proposing to document.

[ddoubleday]

I'm pretty confused by all this. In the above, when I make the replacement suggested:

// Spring DM plugin
dependencyManagement {
imports {
mavenBom ...
}
}

// Gradle core
dependencies {
implementation platform(...)
}

EVERYTHING in Spring Boot Platform is in my compile/runtime classpath, which I don't want, and which was not true when I imported the mavenBom.

EDIT: hmmm....perhaps I'm wrong and there was always more in my classpath than I realized, coming in transitively.

[wilkinsona]

EVERYTHING in Spring Boot Platform is in my compile/runtime classpath

I assume you're looking at the output from Gradle's dependencies task? I thought the same when I first used a Platform. What it's actually showing you is the Platform's version constraints. The modules themselves are not present on the classpath unless there's a standard dependency upon them.

[alexei-osipov]

@wilkinsona I'm not interested in Gradle's module metadata. The problem that "this plugin + Gradle 6.7" results in broken project.
That happens because Gradle 6 publishes own metadata by default but it's unable to consume that metadata because it's not complete.

I think that this plugin should do one of these options:

1. Update documentation to mention issue with Gradle 6 and possible options (like disabling Gradle metadata format)
2. Modify metadata that Gradle publishes to make it valid (and make it actually consumable by other Gradle projects)
3. Implicitly disable automatic publishing of Gradle metadata when this plugin is used

[wilkinsona]

@alexei-osipov We seem to be going round in circles. As described above, #211 is already tracking some documentation updates that will describe how to achieve your second option. It will describe how to, as @jjohannes recommended above, configure Gradle to use the resolved versions. It may also be worth mentioning how to disable metadata publication but I don't think that's something this plugin should do by default.