Reduce maintenance via Renovate

[DanySK]

I'm not part of the maintenance team, but it looks like part of the maintenance burden is dealing with dependency updates. On other projects, I am using Renovate to have updates pull-requested to me. With proper CI in place, I found that it is helpful with:

1. automatically merging working updates, without having a human waste time
2. have a clear indication of which is the first version of a dependency that is breaking the build

For instance, I realized thank to @HannesWell that I can circumvent #2686 in several of my use cases by simply forcing an update version of ASM. With Renovate in place, such update would have been pulled in automatically.

If the board believes this can be a way to go, I can help by providing a tailored configuration for the bot.

[cdietrich]

Do you have experience with

• deal with target platforms
• Code that generates target platforms like our new project wizard
• Restrictions on Java versions

Or would this be about the bom only (which is also restricted by the Java version)

[DanySK]

I would recommend starting with a simple, lightweight configuration using mostly default and well-tested update targets; and, if possible and needed, dealing with non-pre-supported updates through xtext-specific "renovate regex managers."

This would initially target the pom.xml dependencies and the github workflows descriptors, opening a PR for each update (I'd cap the maximum number of PRs to something like 25 not to overwhelm the CI).
Closing a PR manually would instruct the bot to not propose the update anymore unless explicitly asked for (by interacting on a github issue).

[cdietrich]

the problem there is i almost never see a change change does not go accross all the things i mentioned.
so you would renovate do only 25-50% of the work and then require manual afterwork?

[cdietrich]

example change upcoming e.g.

https://github.com/eclipse/xtext/compare/cd/asm97

[DanySK]

It is a bit hard to tell, as most of the maven-related configuration is built-in. I applied it to an Xtext project and let it create a dependency dashboard here: Protelis/Protelis-Parser#72

I think it will, by default, propose an update to the XML files alone. I'm unsure about the target and manifest files, but I am pretty confident it won't touch any java file nor NOTICE.md.
As said, I'd consider this a starting point, we can then add further configuration manually.

Additionally, I believe that there's a lot of repetitions in that update. In these cases, much like application/library code, I modify the build to have a single point of truth for the version of some library (in Gradle I use the catalog, I don't know if there's anything akin for maven, but even a custom solution would be better than manually performing the update).

So let me amend my strategic proposal:

1. enable renovate, see how it behaves by default (or with a slightly improved config such as https://github.com/DanySK/renovate-config/blob/master/default.json (we can extend from there, fixing the assignees, or copy it and create a custom config for xtext)
2. if the updates are inconsistent (e.g., updating the XML but not the target files, then:
   1. make sure that the inconsistency breaks the CI pipeline
   2. decide whether to factorize the dependency or to write a custom renovate update rule
3. if the updates are consistent, 🍾 and look at the next update

[cdietrich]

do you have an example how many false positive this will do.
e.g. all or eclipse(jdt/platform) deps are currently outdated as these are the latest ones that run with java 11.
once we are working on the java 21 topic java 17 will become a minimal but soon after a newer jdt will require a newer java and the play starts from 1

=> would like see a real world update.

2nd question so you propose to take the bom update as a start and then do the orbit / tp update manually after it.
in most cases we see the orbit as leader and align the bom accordingly

@szarnekow @LorenzoBettini any thoughts

[DanySK]

The bot will propose updates as it cannot know that newer versions won't run on Java 11. In these cases, we do close the PRs, and/or we manually exclude them.

World update wise: I use it extensively. This is probably the largest project where I'm using it, https://github.com/AlchemistSimulator/Alchemist, if you look at the closed PRs you'll see that most of them are updates proposed by the bot that get automatically merged once the CI goes green.
The PRs are automatically rebased when master is updated, so in case of conflicting updates only one gets applied.
I'd recommend to start with automerge disabled, see how it goes, and when we have confidence that everything's smooth let the bot perform merges, too.
If you see other bots intervening in that project, it's because I also use a second bot (mergify) to speed up the merge/rebase process and a third one (semantic release) to automate releases, too -- but those are things that could make sense in a future in which updates become a well-oiled mechanism, which I think will require time.

For the latter question: in prospect, the way I see it is that for each dependency there is a single source of truth (you write your dependency once, in a single file, and then that value is propagated to the rest of the infrastructure), so updating that would update both bom and orbit at once.
However, to do so, we need to start somewhere, and if the final goal is to have updates without (or with minimal) human intervention, I'd start bottom-up and see what can be reasonable ways to configure an existing bot to adapt it to the project's necessities, also trying to co-adapt the project.
Again, from a point of view of maintenance, I would suggest to apply the DRY principle, as I see no reason for a dependency version to be specified multiple times in multiple files.
How to is something to be discussed, but to come up with viable proposals I'd like to investigate the bot behavior in this project.

[cdietrich]

yes but i have no clue how to the dry thing without very very extensive scripting

[DanySK]

Unfortunately, I agree. There are two main possibilities that I see:

1. I can hack into the bot config enough to make it produce updates the way we want them
2. We change the build adding scripts as needed, one dependency at a time.