Renovate updates gradle dependency version that does not exist

[jamesdh]

What Renovate type are you using?
GitHub App

Describe the bug
Renovate creates a PR offering to upgrade the following dendency:

runtime "org.glassfish.web:el-impl:2.2"

To 2.2.1. None of the repositories in my build.gradle file contain that version, and in fact, I can't seem to find it in any repository. They do contain a 2.2.1.b01, 2.2.1.b02, etc. So it appears Renovate is not detecting the beta semantics on the versioning string and actually just chopping it off, which then breaks the version definition. Renovate attempts to use the Grails Core repo, which you can see here does not contain the version that it calculates is available.

Did you see anything helpful in debug logs?

DEBUG: Found 2 repositories for org.glassfish.web:el-impl
DEBUG: Looking up org.glassfish.web:el-impl in repository #0 - https://jcenter.bintray.com/
...
DEBUG: Found 15 new versions for org.glassfish.web:el-impl in repository https://jcenter.bintray.com/
DEBUG: Looking up org.glassfish.web:el-impl in repository #1 - https://repo.grails.org/grails/core/
...
DEBUG: Found 1 new versions for org.glassfish.web:el-impl in repository https://repo.grails.org/grails/core/
DEBUG: Found 16 versions for org.glassfish.web:el-impl
DEBUG: Url not found https://repo.grails.org/grails/core/org/glassfish/web/el-impl/2.2.1/el-impl-2.2.1.pom
DEBUG: org.glassfish.web:el-impl not found in repository https://repo.grails.org/grails/core/

...followed by this in the updates json:

{
    "name": "el-impl",
    "depGroup": "org.glassfish.web",
    "depName": "org.glassfish.web:el-impl",
    "currentValue": "2.2",
    "registryUrls": [
        "https://jcenter.bintray.com/",
        "https://repo.grails.org/grails/core"
    ],
    "updates": [
        {
        "fromVersion": "2.2",
        "toVersion": "2.2.1",
        "newValue": "2.2.1",
        "newMajor": 2,
        "newMinor": 2,
        "updateType": "minor",
        "isSingleVersion": true
        }
    ],
    "warnings": [],
    "sourceUrl": null
},

To Reproduce
Steps to reproduce the behavior:

1. Create repository with with a build.gradle file containing...

dependencies {
  runtime "org.glassfish.web:el-impl:2.2"
}

2. Let Renovate do it's thing, and you'll see a PR to upgrade it to 2.2.1

Expected behavior
"org.glassfish.web:el-impl:2.2" is the latest non-beta release available. Has been for years now. It should remain as is. If Renovate were to upgrade to the latest version regardless of beta/alpha versioning semantics (which it probably shouldn't do by default) then it should obviously respect those versioning semantics.

Screenshots
N/A

Additional context

[rarkins]

My attempt to reproduce this failed. @jamesdh are you able to reproduce it in a public repo I can test against?

[jamesdh]

@rarkins I had just disabled the gradle plugin because this was a show-stopper.

I created a new public sample project at https://github.com/moltenbits/renovate-gradle-bug and added a basic renovate config file that enables support for gradle.

Unfortunately now it fails to even run. It dies with this exception from some "renovate-plugin.gradle" file which I'm completely unfamiliar with:

FAILURE: Build failed with an exception.\n\n* Where:\nInitialization script '/mnt/renovate/gh/moltenbits/renovate-gradle-bug/renovate-plugin.gradle' line: 4\n\n* What went wrong:\nCould not compile initialization script '/mnt/renovate/gh/moltenbits/renovate-gradle-bug/renovate-plugin.gradle'.\n> startup failed:\n  initialization script '/mnt/renovate/gh/moltenbits/renovate-gradle-bug/renovate-plugin.gradle': 4: unable to resolve class org.gradle.api.artifacts.repositories.UrlArtifactRepository\n   @ line 4, column 1.\n     import org.gradle.api.artifacts.repositories.UrlArtifactRepository

That appears to be a Gradle 6.0+ class. Gradle 6+ is still pretty new and there's a lot of projects still using Gradle 5 out there.

[rarkins]

Is this sample failure on the hosted app, or running the CLI ?

[jamesdh]

@rarkins I assume the hosted app. That's the error I pulled from app.renovatebot.com

[jamesdh]

@rarkins guessing this has something to do with the fact that your plugin expects Gradle 6, but the project it's trying to run against uses a gradle 5 wrapper. I wonder if something is getting tripped up and it's actually running with that gradle 5 wrapper used by the project.

[rarkins]

@jGleitz is there any chance that the UrlArtifactRepository code you added recently is gradle 6 only?

[jGleitz]

It sure seems like it 😳:

I haven’t thought about that. I think I can provide a patch in a few hours. And then we should really set up those regression tests (#5408)!

[jamesdh]

Ok, thanks @jGleitz and @rarkins for the quick fix on that! This brings us back to the original issue...

If you take a look at https://github.com/moltenbits/renovate-gradle-bug/pull/1, you'll see the following:

The problem is that it tries to update to a dependency that doesn't exist in the one repo that I have configured. Take a look at https://repo.grails.org/grails/core/org/glassfish/web/el-impl/ and you'll see that there is no 2.2.1 version. There is 2.2.1-b01, 2.2.1-b02, etc...

[rarkins]

2.2.1 is listed as a release in https://repo.grails.org/grails/core/org/glassfish/web/el-impl/maven-metadata.xml

Cc @zharinov

[rarkins]

I think I recall discussing this with @zharinov in the past. A limited number of dependencies have invalid metadata, but we decided not to do the extra work required to check within subdirectories because it only affected very old releases. For example these appear to be from 2011: https://mvnrepository.com/artifact/org.glassfish.web/el-impl

[jamesdh]

Might be worthwhile taking a look at how https://github.com/ben-manes/gradle-versions-plugin handles this. I just tried it and it correctly fails (and therefore skips) this dependency.

Otherwise is there some way for us to say "leave this dependency alone"?

[rarkins]

@zharinov can you take another look at this and confirm? If we were to do a deep check as required, how many extra requests per dadasource lookup would that be? e.g. double, triple, etc?

Otherwise is there some way for us to say "leave this dependency alone"?

Add it to ignoreDeps is the easiest.

[jamesdh]

@rarkins @zharinov the first time I ran the gradle-versions-plugin on that project it did take a bit. But after the first run it was lightening fast, so there must be some sort of lookup caching strategy in place.

[renovate-release]

🎉 This issue has been resolved in version 19.225.1 🎉

The release is available on:

• GitHub release
• 19.225.1

Your semantic-release bot 📦🚀