Idea: version compatibility

[rarkins]

In SemVer, suffixes indicate pre-releases, e.g. apha, beta, rc, etc.

But in other contexts, they may mean "compatibility". It's especially common in Docker, e.g. 1.2.3-alpine, so we default to assuming the suffix is compatibility. But also in Maven versioning, we've seen cases like -java7 being added to indicate java 7 compatibility, or Google's Guava using the suffix to indicator flavor.

Renovate's allowedVersions feature was recently enhanced to support regex patterns, which could include for example /-android$/ to enforce a -android suffix or !/-/` to make sure no suffix is in place.

I'm wondering if a new feature called enforceSuffix (defaulting to false for all but Docker) could be useful for users to ensure e.g.:

• Never upgrade from no suffix to suffix (e.g. 1.14.1.1 to 1.14.2-java7)
• Stay on the same suffix (e.g. 1.2.3-android to 1.2.4-android.

I also just realized that we have a risk of confusion between the compatibility object that managers can return with extract results vs the compatibility field that the regex manager supports.

[viceice]

We should also consider a prefix

[rarkins]

We should also consider a prefix

Got any common cases?

[rarkins]

Instead of calling it "compatibility" perhaps we could adopt the term from Guava and call it "flavor". Then the new config option could be called suffixIsFlavor.

[viceice]

Some docker images have the version on then end

[rarkins]

The ideal way this would work is e.g. any versioning scheme can easily be used with Docker images, along with suffix/flavor.

So with Docker as the example, versioning could still be semver or pep440 while supporting compatibility/flavor as a complementary field.

This generic datasource logic would need to incorporate it before performing filtering:

renovate/lib/datasource/index.ts

Lines 109 to 113 in da0181e

	if (res.releases) {
	res.releases = res.releases
	.filter((release) => version.isVersion(release.version))
	.sort(sortReleases);
	}

[rarkins]

This is closely related to extractVersion which can be quickly extended to also extract compatibility in addition to version.

[rarkins]

@viceice I'm wondering if we should even reuse extractVersion itself as a post-extract step to separate version from compatibility/versionCompatibility. Question is whether there's a need to have separate extract patterns for the package file vs the datasource.

[viceice]

I'm not sure if this works. 🤔

[rarkins]

e.g. for Docker we'd default to ^(?<version>[^-]*)(-(?<compatibility>.*))?$ but users could override it with null if they want to use an image that supports semver strings without suffixes like 1.0.0-rc.0.

Also for handling this:

[rarkins]

Another example from Maven: https://mvnrepository.com/artifact/com.microsoft.sqlserver/mssql-jdbc

[rarkins]

I don't think we can reuse extractVersion because the following are quite difference concepts:

• How to split strings found within the repo into the version part and the compatibility part (e.g. 14.16.0-alpine into {currentValue:"14.16.0",compatibility:"alpine"})
• How to extract a the version we care about from a datasource, e.g. v14.16.0 -> {version:"14.16.0"} [currently extractVersion]

I doubt there'd be many cases where both are used together, but I don't think we can reuse the same field.

So to name the first concept - the topic of this issue - we could call it extractCompatibility or compatibleValue. I think in nearly every case it would be version+compatibility but we should use value+compatibility to be safe. It should be done as part of the repository worker process/lookup function and not in manager extract, because we want the ability for packageRules to be applied post-extract. Therefore currentValue would remain the full string (e.g. 14.16.0-alpine) but our versioning modules would all be parsed the post-transformed version, e.g. 14.16.0 would be passed to node versioning.

If using compatibleValue would have it either default to null and unused, or it could be more explicitly defaulted to ^(?<value>.*)$. In the above example the user would configure ^(?<value>.*?)(-(?<compatibility>.*))?$

[wrslatz]

Another use-case to consider: minor updates within the compatibility info (i.e. proposes updates for a -alpine3.14 suffixed image to -alpine3.15).

Practical example: Node.js images (see current tags here). It would be great to get proposed updates from any of 17-alpine3.14, 17.3-alpine3.14, 17.3.1-alpine3.14 to 17-alpine3.15, 17.3-alpine3.15, 17.3.1-alpine3.15 respectively.

You could use -alpine tags alone, but you lose precision into the underlying Alpine version.

[rarkins]

I'm not sure there's an elegant way to automatically achieve this with the current plans

[MarcelCoding]

#5990 (comment)
#5990 (comment)
@rarkins

We should also consider a prefix

Got any common cases?

I just found one that currently isn't supported by renovate - at least the debug logs say that:

git-renovate-1  |            {
git-renovate-1  |              "packageFile": "umami/docker-compose.yaml",
git-renovate-1  |              "deps": [
git-renovate-1  |                {
git-renovate-1  |                  "depName": "ghcr.io/umami-software/umami",
git-renovate-1  |                  "currentValue": "postgresql-v1.37.0",
git-renovate-1  |                  "replaceString": "ghcr.io/umami-software/umami:postgresql-v1.37.0",
git-renovate-1  |                  "autoReplaceStringTemplate": "{{depName}}{{#if newValue}}:{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
git-renovate-1  |                  "datasource": "docker",
git-renovate-1  |                  "depIndex": 0,
git-renovate-1  |                  "updates": [],
git-renovate-1  |                  "warnings": [],
git-renovate-1  |                  "versioning": "docker",
git-renovate-1  |                  "skipReason": "invalid-value"
git-renovate-1  |                }
git-renovate-1  |              ]
git-renovate-1  |            },

Have a look at the skipReason.

Edit: - and another one:

git-renovate-1  |            {
git-renovate-1  |              "packageFile": "jupyter/Dockerfile",
git-renovate-1  |              "deps": [
git-renovate-1  |                {
git-renovate-1  |                  "depName": "jupyter/scipy-notebook",
git-renovate-1  |                  "currentValue": "lab-3.4.3",
git-renovate-1  |                  "replaceString": "ARG VERSION=lab-3.4.3",
git-renovate-1  |                  "autoReplaceStringTemplate": "ARG VERSION={{#if newValue}}{{newValue}}{{/if}}{{#if newDigest}}@{{newDigest}}{{/if}}",
git-renovate-1  |                  "datasource": "docker",
git-renovate-1  |                  "depType": "final",
git-renovate-1  |                  "depIndex": 0,
git-renovate-1  |                  "updates": [],
git-renovate-1  |                  "warnings": [],
git-renovate-1  |                  "versioning": "docker",
git-renovate-1  |                  "skipReason": "invalid-value"
git-renovate-1  |                }
git-renovate-1  |              ]
git-renovate-1  |            }

[rarkins]

Yep, those would both be candidates for this feature. I'm not sure we could detect them automatically, but at least they could be fixed with config.

We could potentially have some intelligence where if a tag contains one - then we split it and see if left or right side is version-ish, and if so then set an automatic versionCompatibility rule.

[MarcelCoding]

Could you help me, I find anything in the docs?

[rarkins]

The feature doesn't exist yet, but you can try using regex versioning, which is in the docs

[MarcelCoding]

Do you mean this? https://docs.renovatebot.com/configuration-options/#versioningtemplate The isn't an example or the names of the regex groups, that should be used.

[rarkins]

No, regex versioning

[MarcelCoding]

Ohh, sorry - I didn't find it because it was named regular expression - my bad.

[wrslatz]

I got regex versioning working somewhat, but it relied on hacking the build capture group to cover the version in the compatibility (i.e. Alpine) part of the tag. That won't work if you also need the build capture group somewhere else in the versioning (we did, in our case) and may not propose correct version updates in all cases, since it does not understand that part of the tag matches an Alpine, Ubuntu, or other tool version.

[MarcelCoding]

I've done it like this: https://git.m4rc3l.de/marcel/stack/src/branch/main/renovate.json#L3-L22

[peterjuras]

Hi,

any plans to incorporate this feature soon? I'm using a Node.js docker image (e.g. node:18.17.1-alpine) and right now the updates are pointless since it always wants to go to latest instead of LTS.

Or is there a good workaround to only update to LTS versions right now?

[peterjuras]

It doesn't appear to work for me, it always tried to do updates to Node.js v20 (and not stay on 18 for LTS).

I added the following configuration:

{
  "matchDatasources": ["docker"],
  "matchPackageNames": ["node"],
  "allowedVersions": "<19"
}

But now it does not create any updates at all.

[rarkins]

This is not a good example because 20 is also LTS and a perfectly valid upgrade.

However if you have for example node:20.0.0-alpine and v21 is released, you'd get an update then (which would be non-LTS)

Never mind, I need my coffee first. Thought 20 was LTS already

[mike-podolskiy90]

Recently got an issue, Renovate automatically updated com.microsoft.sqlserver:mssql-jdbc from 11.2.1.jre8 to 11.2.3.jre18

[renovate-release]

🎉 This issue has been resolved in version 37.3.0 🎉

The release is available on:

• GitHub release
• 37.3.0

Your semantic-release bot 📦🚀

[peterjuras]

Hi @rarkins ,
thanks for the really quick fix! 🙏

I changed my config to the one suggested from your PR and will check what happens 😊

Thank you again!

[rarkins]

If you're using the hosted app, you'll need to wait a day or maybe two for it to be updated to the new version

[mike-podolskiy90]

Thank you for implementing this!
What the configuration would look like in my case with mssql-jdbc please?

[rarkins]

There are a couple of examples in the docs you can use as inspiration. If that doesn't work, please create a reproduction repo

[peterjuras]

@rarkins , do you know when this will roll out to the hosted versions? It has been a couple of days, but I'm getting an invalid config issue:

Config:

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": ["group:allNonMajor", "config:base"],
  "schedule": "on Friday",
  "lockFileMaintenance": {
    "enabled": true,
    "automerge": true,
    "automergeType": "branch"
  },
  "packageRules": [
    {
      "matchUpdateTypes": ["minor", "patch"],
      "matchCurrentVersion": "!/^0/",
      "automerge": true,
      "automergeType": "branch"
    },
    {
      "matchDatasources": ["docker"],
      "matchPackageNames": ["node"],
      "versionCompatibility": "^(?<version>.*)(?<compatibility>-.*)?$",
      "versioning": "node"
    }
  ],
  "rangeStrategy": "pin"
}

[esetnik]

I don't think the provided node example is working correctly. I tried adding:

    {
      "matchDatasources": [
        "docker"
      ],
      "matchPackageNames": [
        "node"
      ],
      "versionCompatibility": "^(?<version>.*)(?<compatibility>-.*)?$",
      "versioning": "node"
    }

And renovate generated a new invalid PR which dropped the compatibility suffix on my dockerfile

Then I noticed that the regex appears to be incorrect since ^(?<version>.*)(?<compatibility>-.*)?$ will be greedy on the
<version>.

I tried with regex ^(?<version>.*?)(?<compatibility>-.*)?$

But then renovate fails to find digests because it appends the suffix twice:

[esetnik]

Also note that the version compatibility specified in the docs does not match the one used in test case https://github.com/renovatebot/renovate/pull/24717/files#diff-92546f944a22f594ce76af3184652c505078794c8edb714cb14a05d09a76f177R259

[peterjuras]

I'm using the following now and it works. I also had the bug you described, and I can't remember where I got the fixed version from (docs?)

{
      "matchDatasources": ["docker"],
      "matchPackageNames": ["node"],
      "versionCompatibility": "^(?<version>[^-]+)(?<compatibility>-.*)?$",
      "versioning": "node"
    }

[esetnik]

I'm using the following now and it works. I also had the bug you described, and I can't remember where I got the fixed version from (docs?)

{
      "matchDatasources": ["docker"],
      "matchPackageNames": ["node"],
      "versionCompatibility": "^(?<version>[^-]+)(?<compatibility>-.*)?$",
      "versioning": "node"
    }

Hmm... that version is also not working for me but afaik it should behave the same way as ^(?<version>.*?)(?<compatibility>-.*)?$ Note the extra ? in (?<version>.*?) to make it a lazy.

[esetnik]

I'm using the following now and it works. I also had the bug you described, and I can't remember where I got the fixed version from (docs?)

{
      "matchDatasources": ["docker"],
      "matchPackageNames": ["node"],
      "versionCompatibility": "^(?<version>[^-]+)(?<compatibility>-.*)?$",
      "versioning": "node"
    }

Using the regex you provided I am getting

[rarkins]

Please create a reproduction repo and then new discussion

[KiwiKilian]

Working great with the example from docs, thanks!