Gradle reports incorrect jackson-bom dependency version

[five-iron]

See this issue for reference: FasterXML/jackson-databind#3428

I think the cause of this is due to the configuration of gradle-module-metadata-maven-plugin in jackson-base pom.xml:

<plugin>
  <groupId>de.jjohannes</groupId>
  <artifactId>gradle-module-metadata-maven-plugin</artifactId>
  <version>0.2.0</version>
  <executions>
    <execution>
      <goals>
        <goal>gmm</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <platformDependencies>
      <dependency>
        <groupId>com.fasterxml.jackson</groupId>
        <artifactId>jackson-bom</artifactId>
        <version>${project.version}</version>
      </dependency>
    </platformDependencies>
  </configuration>
</plugin>

${project.version} only works if the jackson-bom version matches the downstream project. E.g., if jackson-databind is at version 2.13.2.1 while jackson-bom is actually at 2.13.2.20220324, gradle will mistakenly report the jackson-bom dependency as 2.13.2.1, which there is no version for, and so it fails.

I'm not sure the best solution, but maybe this needs to be hardcoded to the same version as jackson-bom above, and changed with each release/snapshot cycle? Or maybe a variable or something in this file.

[cowtowncoder]

@jjohannes would be the person who knows, I think. Micro-patch handling is tricky, not sure what could be done here.

[jjohannes]

If the version number of micro-patch releases are not aligned -- which I was not aware of when adding the above configuration -- I would suggest to alway link to the latest major.minor version of the BOM. You can still upgrade individual components to their micro-patch release then. Or use one of the newer BOMs directly.

Would that make sense?

<version>${majorMinorVersion}</version>

We have to figure out how to get hold of majorMinorVersion I don't know how to do that in maven and if it is simple.

[cowtowncoder]

@jjohannes Just to make sure I understand: are you suggesting that jackson-databind version 2.13.2.1 should simply refer to jackson-bom of 2.13.2?
That is something I could consider and would likely work.

But there is one related aspect that I'd like to ensure works as well: from users perspective there is benefit from bom that would have the combination of jackson-databind:2.13.2.1 and everything else at 2.13.2.0.
So what I am thinking of would be to still release such a BOM but not refer to it from jackson-databind.
Would this work? So, literally just:

1. Publish jackson-databind:2.13.2.1 with parent pom of jackson-base:2.13.2 (which then refers to jackson-bom:2.13.2)
2. Publish jackson-bom:2.13.2.[timestamp] (and jackson-base) with managed dependencies of (1)

This would allow users to refer to BOM from (2) but without creating inconsistent loop for (1).

One possible kink, though, is this: jackson-bom:2.13.2 does specify managed dependency 2.13.2 for jackson-databind. I don't know if that matters -- it shouldn't, right? -- but thought worth pointing out.

[cowtowncoder]

I guess I may be overthinking this, fwtw, since I am not only considering module plugin but more generally version resolution by tooling. This is due to my non-OSS work where for a while I was working on figuring out how transitive dependencies are resolved, vulns reported against them, possible automated upgrades and such. This is a rather complicated area (as I am sure everyone participating here knows :) ) due to version conflicts that transitive closure brings in -- and then different strategies in selecting the "real" version.
Various tools have their own views, too; my (naive?) view is that Gradle takes the "latest version wins" approach whereas Maven attempts to find "the closest reference": this is challenging because it is common for there to be more Maven metadata that Gradle tooling needs to use -- so which logic should be applied.

Having said that it does seem to me that what I outline above would be workable. I will try to verify this locally and then think I'll proceed with 2.12.6.1 release.

And given the metadata issue I may (have to) consider releasing otherwise change-less jackson-databind version 2.13.2.2 (no, not ready to go for 2.13.3, not enough fixes as per https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.13.3) to get to working metadata.

[jjohannes]

@cowtowncoder yes what you describe works. And it it basically what I tried to say. I only missed the parent pom part, which ist a nice solution I think.

And yes. A user can still use the jackson-bom:2.13.2.[timestamp] version directly in Gradle and it will work (user will get jackson-databind:2.13.2.1) because Gradle picks the higher version.

If you want to go with that solution but still run into issues, let me know.

[jjohannes]

I think the "best" solution in genreal would be to not have timestamp BOMs, but instead have a BOM for each micro version.

So for the current situation there would be a BOM 2.13.2.1 which refers to jackson-databind:2.13.2.1 and to version 2.13.2 of all the other components for which there is no micro patch. Then the jackson-databind:2.13.2.1 metadata would work as it is now.
But I don't how easy (or if at all) that can be realized with the current setup.

[cowtowncoder]

@jjohannes I think date-based (realized "timestamp" is bit misleading) may actually better solution personally, for the main use case: extending set of boms for otherwise closed branches. Problem with attempts to use full 4-digit version is that this is not only more complicated to handle (2.13.2.3 of jackson-databind along with 2.13.2.1 of jackson-dataformat-xml, latter of which is released after former, for example, is a possibility) but gives wrong impression of there being a full set of components. Use of date only offers sorting to indicate "which BOM is later" (and has more up-to-date set).

I guess it would be possible to just increment bom micro-patch version, but honestly I also do not see any benefit to the user: you cannot really arbitrarily increase that number but have to use some specific version. And for that date-based version works fine, see f.ex:

https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom

as long as there's no mixing of sequence numbers and date-based.

Be that as it may, at this point I will not be changing this aspect. But I will proceed to change the way micro-patch - to -base/bom works to make GMM generation succeed.

[cowtowncoder]

Hmmmh. Actually. Looking at generated

target/publications/maven/module.json

I am not sure things work any better with "bom" of 2.13.2 (or similar patch-level only) approach. :-(

[cowtowncoder]

Ok. But would the fix not be as simple as using this:

  <configuration>
    <platformDependencies>
      <dependency>
        <groupId>com.fasterxml.jackson</groupId>
        <artifactId>jackson-bom</artifactId>
        <version>${project.parent.version}</version>
      </dependency>
    </platformDependencies>
  </configuration>

i.e. replace ${project.version} with ${project.parent.version}?
That seems to work if used directly from the project (I am modifying pom.xml of jackson-databind); not 100% sure yet if it'd work from jackson-bom/base/pom.xml.

Will propose a patch against 2.13 soon.

[cowtowncoder]

@jjohannes I published jackson-databind version 2.12.6.1 with the definitions I suggested. Is there an easy way to see whether resulting .module file works as expected and validates? It seems legit to me but I am slightly worried about possible discrepancy between versions 2.12.6 (that referenced jackson-bom defines for jackson-databind) and 2.12.6.1 (that referencing jackson-databind has as version).

[jjohannes]

@cowtowncoder thanks I tested it (see comment on the other issue) and it works!

And yes <version>${project.parent.version}</version> looks good. I missed that you'll need that (I do not speak Maven that well :) )

If you ever want to test something manually yourself. You can do:

1. Create a folder with two empty files settings.gradle.kts and build.gradle.kts
2. Put his into build.gradle.kts

plugins {
    id("java-library")
}

repositories {
    mavenCentral()
}

dependencies {
    // change dependencies and versions here to see effects
    implementation(platform("com.fasterxml.jackson:jackson-bom:2.12.6.20220326"))
    implementation("com.fasterxml.jackson.core:jackson-databind")
    implementation("com.fasterxml.jackson.dataformat:jackson-dataformat-protobuf")
}

3. Run gradle :dependencies --configuration compileClasspath --scan

You can look at the dependencies tree printed to console or open the Build Scan Link and explore the results in the browser.

[cowtowncoder]

@jjohannes Thank you for the quick response & all help here, as well as obviously adding GMM support in the first place.
I had to look for parent pom reference too, I am not much of a Maven expert.

I think I'll need to attach your test procedure somewhere as documentation, thanks!

[cowtowncoder]

One final twist: after thinking it through, I decide to actually publish jackson-bom version 2.13.2.1 so that jackson-databind:2.13.2.1 will "just work", wrt GMM. At least I hope that is the case, thinking it through :)

[jpcmonster]

@cowtowncoder just to confirm - on upgrading jackson-bom to 2.13.2.1, one should expect to see the jackson databind version go 'down' from 2.13.2.2 to 2.13.2.1, correct?
jackson-bom-2.13.2.20220328...jackson-bom-2.13.2.1#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8R62

[cowtowncoder]

@jpcmonster Not quite sure what you mean about downgrade, but yes, jackson-bom of 2.13.2.1 does specify jackson-databind version of 2.13.2.1.

[dewa-actico]

I think what they meant is, https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom/2.13.2.20220328 - which is from march, 29th, has a dependency to jackson-databind:2.13.2.2 whereas
https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom/2.13.2.1 - which is from march, 30th, has a dependency to jackson-databind:2.13.2.1 which could be considered a downgrade.

[jpcmonster]

thx @dewa-actico - yes, that's what I meant. Sorry if that was unclear; SpringBoot 2.6.7 makes this exact change to their jackson-bom dependency, which results in our artifacts now containing 2.13.2.1 where they had 2.13.2.2 when our dependency was on 2.6.6. This may be entirely correct, but was unusual enough that I wanted to ask.

[cowtowncoder]

Ok, right. Semantically, I would NOT consider 2.13.2.1 bom to be newer or later -- it was released only to work around the flaw in release of jackson-databind 2.13.2.1, after the fact. Normally all micro-patch BOMs have datestamp instead of fourth digit; this since there is no consistent set of micro-patches (different components may have different 4th digit, including none at all for most of them).
Not 100% sure why Spring Boot chose 2.13.2.1 but yes, it should be equivalent to the alternatives.

Once we release 2.13.3 this will all be untangled but that may take a while yet.

[five-iron]

@cowtowncoder This is broken again with the latest micro patch release

Edit: Build logs show

Could not find com.fasterxml.jackson:jackson-bom:2.13.4.1

[cowtowncoder]

@five-iron Yes, I did not backport fix in 2.13 branch, unfortunately. Adding it in now won't help with micro-patch although I should probably do it just in case.
But in addition I need to add same work-around as 2.13.4.2 of jackson-databind.

[cowtowncoder]

@ahattz11 That is unfortunately wrong thing to do: I never intend to release jackson-bom with such version because that would imply a full set of 2.13.4.1 components. I did do one-off earlier, but at this point I think everyone should just use proper bom -- 2.13.4.20221013.

So please upgrade to the new bom released today -- or, if you don't use bom, jackson-databind 2.13.4.2. Either one would resolve the issue.

[chadlwilson]

FWIW I had no apparent issues with Gradle 7.5 using 2.13.4.20221012 when declaring via Gradle platforms and either implementation platform(bomGav) or implementation enforcedPlatform(bomGav) (for a single module and multi-module project). So I guess particular Gradle usage via projects or dependencies might be required to end up with a problem here?

[cowtowncoder]

@chadlwilson That sounds plausible. There is a real problem, but it sounds like it will affect a subset of Gradle users: perhaps it requires direct "implementation" dependency to jackson-databind?

[kolotyluk]

I have been battling problems with failing Gradle builds all week related to these problems... every time I try to fix something, the problems seem to multiply.

Just for summary, can someone please tell me the latest stable version(s) of Jackson and friends.

I am using Gradle 7.5.1

[chadlwilson]

@kolotyluk See #52 (comment) - still the latest at time of writing. Or can explore via https://mvnrepository.com/artifact/com.fasterxml.jackson/jackson-bom

[cowtowncoder]

@kolotyluk There is only one problematic jackson-bom version as far as I know, 2.13.4.20221012. This is why I had publish 2.13.4.20221013.

All other boms should be fine. I am not sure what are issues you have since your description had very little information to go about. If you have issues with other boms please explain what specifically you have tried to do.

[kolotyluk]

So, it turns out the problem I had was caused by org.apache.spark:spark-core_2.13:3.3.1 which references com.fasterxml.jackson:jackson-bom:2.13.4.1 and causes all sorts of havoc... Switching to version 3.3.0 corrects the problem.

[cowtowncoder]

@kolotyluk Ah. Yes, micro-patches are... seemingly hard to explain.

But the idea is that only 3-part versions like 2.13.4 are Full Sets of all components.
And conversely anything with 4-parts is a so-called "micro-patches", one-offs for just individual components.
Since there is no full sets this version cannot be used for jackson-bom: instead an order-preserving date-based version is used as fourth number to indicate consistent set of latest versions on that date.

Not sure why spark-core would try referencing non-existing jackson-bom but here we are.
(aside from anything else one really needs to verify that a given version is in Maven Central -- but I can see why there is so much confusion).

[chadlwilson]

I think the problem there is that spark-core 3.3.1 references jackson-databind 2.13.4.1 which is the version with the incorrect gradle module metadata back to a BOM version that doesn't exist (see here). So the issue there is the databind metadata, rather than them intentionally using a non-existent BOM.

The problem there can be resolved without rolling back spark by

• spark-core update to databind 2.13.4.2 (done in [SPARK-40886][BUILD] Bump Jackson Databind 2.13.4.2 apache/spark#38355 but not released at time of writing, likely will be in 3.3.2)
• OR you override the databind version yourself to 2.13.4.2 yourself to control the transitive dependency
• OR you define a gradle platform to jackson-bom yourself to control jackson versions coming from spark (and any other dependencies you have which use jackson)

[cowtowncoder]

@chadlwilson Ok that makes sense, yes. It'd be good if they used jackson-bom instead of direct versions, but obviously both should work.