Renovate pre-commit addition dependencies

[adam-moss]

What would you like Renovate to be able to do?

Renovate currently has the ability to update .pre-commit-config.yaml if the behaviour is opted into. At the moment however renovate is only updating the declared rev (repo tag) dependencies. It would be nice if renovate could also update any additional_dependencies if present, applying the normal renovate rules around pinning etc.

Minimal example .pre-commit-config.yaml :

%YAML 1.2
# yaml-language-server: $schema=https://json.schemastore.org/pre-commit-config.json
---
default_install_hook_types:
    - pre-commit

default_stages:
    - commit

fail_fast: true

repos:
    - repo: https://github.com/pre-commit/mirrors-prettier
      rev: v3.0.0-alpha.5
      hooks:
          - id: prettier
            additional_dependencies:
                - [email protected]
                - eslint-config-prettier
                - stylelint-config-prettier
                - prettier-plugin-java
                - prettier-plugin-sh
                - "@prettier/plugin-xml"

In the above example, if ran with renovate today, rev for the prettier repo would update to v3.0.0-alpha.6 however the additional_dependency of prettier would remain at v3.0.0-alpha.5.

The other dependencies would similarly be unchanged, but should apply pinning rules etc.

If you have any ideas on how this should be implemented, please tell us here.

Correctly identifying the upstream datasource of additional_dependencies will probably necessitate retrieving the .pre-commit-hooks.yaml file from the referenced repo to grab the language attribute:

https://github.com/pre-commit/mirrors-prettier/blob/6f3cb139ef36133b6f903b97facc57b07cef57c9/.pre-commit-hooks.yaml#LL5C12-L5C12

Is this a feature you are interested in implementing yourself?

No

[github-actions]

Hi there,

Get your issue fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the issue if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[adam-moss]

Minimal reproduction added: https://github.com/adam-moss/renovate-20780

[rarkins]

Reproduction forked to https://github.com/renovate-reproductions/20780

[aentwist]

Not a requirement, but would appreciate it if the matcher happens to be robust enough to handle YAML anchors on the map key!

additional_dependencies: &commitlint-additional-dependencies
  - [email protected]
  - "@commitlint/[email protected]"

[aentwist]

Hmmm this is an interesting problem.

1. This kind of matching

It isn't a problem regular expressions are well-suited to; I don't think they are strong enough. It feels more like a context-free grammar problem (a superset of regular expressions that is strong enough to describe any formal grammar, which regexps cannot do). Because of this it feels like the best path forward is to tokenize the YAML, then go from there.

2. Automatic detection of the datasource is not easy

There is no clear indication of the datasource without hardcoding a list of them for each package. However, the datasource for each group of additional_dependencies is the same and can be found in the .pre-commit-hooks.yaml at the URL specified by the repo, for example,

repos:
  - repo: https://github.com/pre-commit/mirrors-prettier

.pre-commit-hooks.yaml

    language: node

(=> npm)

---

I'm going to try rigging up an advanced regex matcher in the meantime, which I guess will unfortunately have to look like this since I'm playing by the regex rules and the datasource problem is not trivial:

repos:
  - repo: https://github.com/pre-commit/mirrors-eslint
    rev: v8.56.0
    hooks:
      - id: eslint
        files: \.(?:vue|[jt]sx?|[cm][jt]s)$
        types: [file]
        additional_dependencies:
          # renovate: datasource=npm
          - [email protected]
          # renovate: datasource=npm
          - [email protected]
          # renovate: datasource=npm
          - "@vue/[email protected]"
          # renovate: datasource=npm
          - "@vue/[email protected]"

{
  "customManagers": [
    {
      "customType": "regex",
      "fileMatch": ["^.pre-commit-config.yaml$"],
      "matchStrings": [
        "# renovate: datasource=(?<datasource>.*?)( versioning=(?<versioning>.*?))?\\s+-\\s+['\"]?(?<depName>@?[^@]+)(?:@(?<currentValue>[^'\"\\s]*))?"
      ],
      "versioningTemplate": "{{#if versioning}}{{{versioning}}}{{else}}semver{{/if}}"
    }
  ]
}

Regex fails especially hard at saying "if there is an opening quote, there must be an identical closing quote", and there cannot be identical capturing group names so I can't hardcode all 3 cases (no quotes, single, and double), leading to a very weak regex. Whatever, it hecking works!

https://regex101.com/r/LaPLMA/1

[aentwist]

I rolled it up into my renovate presets, if you just want the manager you can extend

github>aentwist/renovate//presets/pre-commit/internal/additional-dependencies-custom-manager

Then add the comments above each additional dependency. I documented this where the code lives https://github.com/aentwist/renovate?tab=readme-ov-file#pre-commit

Maybe a better idea if you are using some combination of eslint, prettier, and commitlint is to use the whole pre-commit javascript preset

github>aentwist/renovate//presets/pre-commit/javascript

Am open to expanding these presets, right now it is pretty much JS only because that is what I use right now

[mschoettle]

@aentwist Thanks a lot for this! I adapted it to Python dependencies for my own project as follows:

"customManagers": [
    {
      "customType": "regex",
      "fileMatch": ["^.pre-commit-config.yaml$"],
      "matchStrings": [
        "# renovate: datasource=(?<datasource>.*?)( versioning=(?<versioning>.*?))?\\s+-\\s+['\"]?(?<depName>[^=]+)(?:==(?<currentValue>[^'\"\\s]*))?"
      ],
      "versioningTemplate": "{{#if versioning}}{{{versioning}}}{{else}}pep440{{/if}}"
    }
]

Example use:

  - repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.9.0
    hooks:
      - id: mypy
        additional_dependencies:
          # renovate: datasource=pypi
          - pydantic==2.6.3

[aentwist]

Did using # renovate: datasource=pypi versioning=pep440 not work for you ?

pep440 is a very bad general default.

Or I guess this way you don't have to keep repeating pep440, but as I mentioned that is just the limitation of this workaround.

[mschoettle]

No, because the regex expects @ after the dependency name whereas I specified it as dependency==1.2.3 (I believe the syntax for additional_dependencies is language specific).

[aentwist]

Oh wow, right! Although in that case there are therefore infinite possibilities, I guess I should add as many common ones as I can to the regex. I guess I'll do that by skimming over the supported hooks list on the pre-commit website and seeing what I find..