Filter out ignored dependencies before fetching

[disposedtrolley]

Package manage/ecosystem
gomod
Manifest contents prior to update
N/A
Updated dependency
N/A
What you expected to see, versus what you actually saw
Attempting to ignore private Go modules using a wildcard name, e.g.

ignore:
      - dependency-name: "*github.com/myorg*"

still results in an error when Dependabot attempts to fetch the entire dependency list -- Dependabot can't resolve your Go dependency files.

The docs note that GitHub Dependabot checks for all allowed dependencies and then filters out any ignored dependencies or versions. which explains why the fetching fails when it hits a private dependency midway through.

Would it make sense to filter out ignored dependencies prior to performing the fetch?

Images of the diff or a link to the PR, issue or logs
N/A

[feelepxyz]

Would it make sense to filter out ignored dependencies prior to performing the fetch?

Unfortunately not with the way it works currently. Dependabot will exclude the ignored dependency when checking for new versions but as soon as it attempts to update any other dependency it will try to resolve all dependencies in your manifest/lockfile. This is usually done by the package manager and we don't have much control over it. We have some ideas to improve this by stubbing out packages but going to be a while before we can look at this.

We're working on adding private git repository support over the next few months. For now, I would recommend you keep using Dependabot Preview which does support private git repos if you add these to the app installation. You can install the Preview app from here: https://app.dependabot.com/ and configure it using a config file: https://dependabot.com/docs/config-file/

[disposedtrolley]

Thanks for the clarification @feelepxyz!

That makes sense; fetching new dependencies would be delegated to the package manager so there would be no easy method of ignoring them from the get-go.

With Go Modules, there's an option to set a GOPROXY environment variable to point go mod to a private modules source, such as Artifactory. Would something like this be included in the private git repository support?

[feelepxyz]

With Go Modules, there's an option to set a GOPROXY environment variable to point go mod to a private modules source, such as Artifactory. Would something like this be included in the private git repository support?

Yep, we're looking at adding support for environment variables.

[jeffwidman]

Support for private registries was added a bit ago: https://github.blog/2021-03-15-dependabot-private-dependencies/

So I'm closing this, as it looks like the root cause for why the OP needed to filter out ignored dependencies is resolved.

[Okeanos]

Please re-open this issue. Just because a workaround exists for some use cases doesn't mean this is actually solved. In particular please check my comment in #1884 . Having Dependabot fetch ignored dependencies before discarding them causes it to fail without being able to recover reproducibly and unnecessarily.

[owenhaynes]

@jeffwidman Do you have docs anywhere on how to set GOPROXY, as private registries say they are supported but no docs about limitations etc.

As things like Jfrog Artifactory, Artefact registry etc need GOPROXY setting to work.

[jeffwidman]

@owenhaynes please file a new issue, this is related to filtering ignored deps, not GOPROXY.

Speaking of filtering ignored deps before fetch, I looked at this a little more, and I think this isn't something we will do anytime for the forseeable future.

As @feelepxyz said above:

Dependabot will exclude the ignored dependency when checking for new versions but as soon as it attempts to update any other dependency it will try to resolve all dependencies in your manifest/lockfile. This is usually done by the package manager and we don't have much control over it. We have some ideas to improve this by stubbing out packages but going to be a while before we can look at this.

I don't think we'll very likely to go the route of stubbing out packages. We'd rather move Dependabot toward being the glue code between the native package manager and your repo, where Dependabot handles the cron, wiring up the native package manager, and then opening the PR. But move away from "re-implementation of native package manager behavior in Ruby, but poorly"... So I just don't see us doing pre-filtering in a way that doesn't force us to be pretty invasive in the native package manager... post-filtering is much easier.

I realize that means some deps will continue to break things, and I wish there were a better answer. Some package managers allow substitution or replacing deps within their manifest files, so it is something that can be supported at the package manager level. And if your preferred package manager doesn't support it, you can always file an upstream issue request / PR.