401 Authentication errors on a solution that has mixed dotnet and MSBuild dependencies

[bm-fez]

Describe the bug

The problem

I have been using this Azure DevOps task against a legacy solution that contains .NET 4.8 and .NET Core projects, hence it is doing a mixed dotnet and MSBuild process against a private Azure DevOps NuGet package feed.

Using the 1.30 Docker image, and the parameters for a private feed I can get most of it working.

My pipeline YAML

- task: dependabot@1
  displayName: dependabot-$(Build.Repository.Name)
  enabled: false
  inputs:
    # These two inputs are needed to workaround a bug in dependabot that breaks authentication to private nuget feeds in some cases, see: https://github.com/tinglesoftware/dependabot-azure-devops/issues/921#issuecomment-2162273558
    # Once the following issue is resolved, bump the image tag to the version with the fix and the inputs can be removed: https://github.com/dependabot/dependabot-core/pull/8927
    dockerImageTag: '1.30'
    extraEnvironmentVariables: 'WORKAROUND_CMD=sh -c "$(curl -fsSL https://aka.ms/install-artifacts-credprovider.sh)";NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED=true;VSS_NUGET_EXTERNAL_FEED_ENDPOINTS={"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v3/index.json","username":"unused","password":"$(System.AccessToken)"}]}'
    targetRepositoryName: $(Build.Repository.Name)
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

My config file

version: 2
registries:
  nuget-azure-devops:
    type: nuget-feed
    url: https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v3/index.json
    token: PAT:${{SYSTEM_ACCESSTOKEN}}
updates:
  - package-ecosystem: "nuget"
    directory: "/src"
    registries:
      - nuget-azure-devops

All the 403 errors I saw initially, prior to the extra parameters, are resolved. However, I am still seeing authentication issues for MSBUILD dependencies

Attempting to gather dependency information for multiple packages with respect to project '/home/dependabot/dependabot-updater/tmp/myorg/myproj/_git/myrepo/src/myproj.csproj', targeting '.NETFramework,Version=v4.8'
MSBuild auto-detection: using msbuild version 'Current' from '/usr/local/dotnet/current/sdk/8.0.300'.
Using credentials from config. UserName: user
An error occurred while retrieving package metadata for 'Microsoft.Graph.5.56.0' from source 'nuget_source_1'.
  Response status code does not indicate success: 401 (Unauthorized).

It looks as if a dynamic NuGet source nuget_source_1 is being used which is not picking up the configured authentication credentials.

A workaround (sort of)

After much experimentation, running the container locally I have a workaround.

If I set an environment variable on the Docker command line via a -e parameter

- e nuGetPackageSourceCredentials_<nuget source 1 as listed in error i.e. nuget_source_1> ="UserName=unused;Password=<my PAT>"

It works, however there is a problem applying this workaround to the Azure DevOps task

The problem with the workaround

The issue is that this extra environment variable needs to include a ; between the user and passwrod, but this character is being used to delimiter the parts of extraEnvironmentVariables that will become docker -e parameters. This means the new environment variable is split into two on the ; by the time it reaches the docker run command

I see

 -e NuGetPackageSourceCredentials_nuget_source_1="UserName=unused -e Password=***"

as opposed to the required

 -e NuGetPackageSourceCredentials_nuget_source_1="UserName=unused;Password=***"

I have tried all I can think of to escape the ; in the environment variable e.g. \x3b, so that the unescaped ; are used to split the parameter by Azure DevOps (in Typescript), and the escaped ; is handled by the Docker command line (BASH), but I can't get it to work, the escaping is not resolved.

Questions/Options

This work around raises a number of questions as to if it is really needed and how to apply it

1. Can anyone see if I missed something so that the credentials management so the NuGetPackageSourceCredentials_nuget_source_1 environment variable is not required?
2. Is there a generic form NuGetPackageSourceCredentials_<source> as I don't like that I have to set it up manually?
3. Can anyone think of a different way to escape the ; in task parameter?
4. Best option I thought of was to make the delimiter on the ; Azure DevOps task be made configurable, so a swap could be made to one that does not clash with the ;. Using \n might make the long parameter more readable (of course leaving the default as ; for backwards compatibility. Any thoughts on this?

Categorization

• This is not a permissions issue (We cannot solve permission issues)

To Reproduce

I am not sure of the exact solution structure to get a simple repo of this one

Expected behavior
Analysis of the solution should complete without authentication errors.

Server (please complete the following information):

• Region: westeurope
• Version: 1.30

[mburumaxwell]

Should be fixed in #1241?

[rhyskoedijk]

It looks like for projects that use NuGet.exe to perform the update (i.e. .NET Framework projects with packages.config), Dependabot proxies the nuget.config with its own version. See: https://github.com/dependabot/dependabot-core/blob/7c0e5e709eaafe7cfa543f0e85a9ea129a2791fa/nuget/lib/dependabot/nuget/nuget_config_credential_helpers.rb#L23:L56

I think ideally we don't want this to happen; it would be preferable to rely on the Azure Artifact Credential Manager to handle auth rather than having Dependabot fiddle with the config files. Additionally, the way Dependabot currently modifies the nuget.config file only allows for feeds supporting PAT's, not basic auth (e.g. Telerik NuGet feed).

I'll setup a test repo with a .NET Framework 4.8 project and see if I can reproduce this issue and see what options there are. My guess is if Dependabot didn't proxy nuget.config, it would probably "just work" using the auth already configured in Azure Artifact Credentials Manager.

Your comment about the extra environment variables is pretty valid. I wonder if we could modify the extension to accept a dictionary instead of a string, this way you can use ; and also make the variables more human readable.
e.g.

- task: dependabot@1
  inputs:
    dockerImageTag: '1.30'
    extraEnvironmentVariables: 
      WORKAROUND_CMD: "sh -c "$(curl -fsSL https://aka.ms/install-artifacts-credprovider.sh)"
      NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED: "true"
      VSS_NUGET_EXTERNAL_FEED_ENDPOINTS: '{"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v3/index.json","username":"unused","password":"$(System.AccessToken)"}]}'
      NuGetPackageSourceCredentials_nuget_source_1: "UserName=unused;Password=***"

@mburumaxwell thoughts?

[mburumaxwell]

I do not necessarily have joy dealing with packages.config. Have no legacy projects that use it either. My input around that is not valuable.
As regards the input, I don't think we can create an input of object type, neither can we have nested inputs.

What I think we can do is:

1. Change the input type to multiline.
2. Add a new input named extraEnvironmentVariablesSeparator which defaults to ; for backwards compatibility.

This way, we can do this

- task: dependabot@1
  inputs:
    dockerImageTag: '1.30'
    extraEnvironmentVariablesSeparator: '\n'
    extraEnvironmentVariables: |  # <-- the new line notation
      WORKAROUND_CMD: "sh -c "$(curl -fsSL https://aka.ms/install-artifacts-credprovider.sh)"
      NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED: "true"
      VSS_NUGET_EXTERNAL_FEED_ENDPOINTS: '{"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v3/index.json","username":"unused","password":"$(System.AccessToken)"}]}'
      NuGetPackageSourceCredentials_nuget_source_1: "UserName=unused;Password=***"

My only issue is the new line being either \r\n or \n in which case we can instead use a boolean (extraEnvironmentVariablesWithNewLine defaullting to false), then we can log a warning to encourage setting it to true when set to false and extraEnvironmentVariables is not null.

What do you think?

[bm-fez]

I have checked my solutions, and it was using old version of .NET & .NET Core frameworks, but I have updated all projects to either .NET 4.8 or .NET Core 8.0 and still get the same behavior.

Looking again at the looks, I think the problem area, trying to build a minimalist repo is as follows

• A .NET 4.8 web app project is being built via MSBuild
• This referencing another .NET 4.8 project class library project - it is the NuGet restore for the dependent build that has the issue

This would tally with your comment about proxying configs

My guess is if Dependabot didn't proxy nuget.config, it would probably "just work" using the auth already configured in Azure Artifact Credentials Manager.

[bm-fez]

NET48-Sample.zip

I have stripped my problematic solution down to the minimum that that exhibits the problem i.e. one class library (with most of it's logic removed), and an empty MVC website that references it. Thought it might help with the repo of this issues

This solution still reports the same 401 error when it tries to use a private Azure DevOps artifact feed (that has an upstream of Nuget.org) to get public packages

Attempting to gather dependency information for multiple packages with respect to project '/home/dependabot/dependabot-updater/tmp/blackmarble-source/ISS/_git/RFISS/Src/Expenses/BlackMarble.AD.Providers/BlackMarble.AD.Providers.csproj', targeting '.NETFramework,Version=v4.8'
MSBuild auto-detection: using msbuild version 'Current' from '/usr/local/dotnet/current/sdk/8.0.300'.
Using credentials from config. UserName: user
An error occurred while retrieving package metadata for 'Microsoft.Graph.Core.3.1.14' from source 'nuget_source_1'.
  Response status code does not indicate success: 401 (Unauthorized).

   at NuGetUpdater.Core.MSBuildHelper.ThrowOnUnauthenticatedFeed(String stdout) in /opt/nuget/lib/NuGetUpdater/NuGetUpdater.Core/Utilities/MSBuildHelper.cs:line 619
   at NuGetUpdater.Core.PackagesConfigUpdater.RunNugetUpdate(List`1 updateArgs, List`1 restoreArgs, String projectDirectory, Logger logger) in /opt/nuget/lib/NuGetUpdater/NuGetUpdater.Core/Updater/PackagesConfigUpdater.cs:line 148
  Writing update result to [/tmp/update-result.json].
I, [2024-07-25T15:16:50.872983 #18]  INFO -- : update result: {
  "ErrorType": "AuthenticationFailure",
  "ErrorDetails": "([https://pkgs.dev.azure.com/blackmarble-source/_packaging/BM-Libs/nuget/v3/index.json)"](https://pkgs.dev.azure.com/blackmarble-source/_packaging/BM-Libs/nuget/v3/index.json)%22)
}

[rhyskoedijk]

Thanks. I've managed to reproduce the issue now and have a draft fix; It's not pretty, but it is a fix.
I'll submit a PR for it soon and see what mburumaxwell thinks.

[rhyskoedijk]

I've put up #1248, which should fix this issue without needing to override NuGetPackageSourceCredentials_nuget_source_1. The auth should now pass-through to MSBuild from your credentials in dependabot.yml.

[rhyskoedijk]

@bm-fez can you please re-test this using the tag 1.30.3-ci0004 or latest and see if the issue is resolved in your repo. If not, could you share any new logs and I'll take another look.

Your pipeline.yml can now be simplified to:

- task: dependabot@1
  inputs:
    dockerImageTag: 'latest'
    targetRepositoryName: $(Build.Repository.Name)
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

No changes required to dependabot.yml (i.e. token: PAT:${{SYSTEM_ACCESSTOKEN}} should work)

[bm-fez]

@rhyskoedijk Thanks for providing the update, unfortunately I am still seeing the same problem (using the .NET 4.8 sample I uploaded previously in this issue).

I have tried various changes using the docker tag to 1.30.3-ci0004 (used for all test)

• the sample as I previously provided - did not fix the issue
• tried adding a key entry to the dependabot.yml for 'nuget_source_1' - did not change anything
• adding a nuget.config (in the same folder as the sln file) - previous it was running using the default 'nuget_source_1'
• tried adding a key entry to the dependabot.yml matching the source in the nuget.config

In all cases I see the same error (full log of task)

Attempting to gather dependency information for multiple packages with respect to project '/home/dependabot/dependabot-updater/tmp/blackmarble-source/ISS/_git/FezIIS/Src/BlackMarble.AD.Providers/BlackMarble.AD.Providers.csproj', targeting '.NETFramework,Version=v4.8'
MSBuild auto-detection: using msbuild version 'Current' from '/usr/local/dotnet/current/sdk/8.0.300'.
Using credentials from config. UserName: user
An error occurred while retrieving package metadata for 'Microsoft.Graph.5.56.0' from source 'nuget_source_1'.
Response status code does not indicate success: 401 (Unauthorized).

The settings I currently have are

• dependabot.yml

version: 2
registries:
  nuget-azure-devops:
    type: nuget-feed
    key: internalfeed
    url: https://pkgs.dev.azure.com/blackmarble-source/_packaging/BM-Libs/nuget/v3/index.json
    token: PAT:${{SYSTEM_ACCESSTOKEN}}
updates:
  - package-ecosystem: "nuget"
    directory: "/Src"
    open-pull-requests-limit: 6
    registries:
      - nuget-azure-devops

• Dependabot YAML pipeline step

- task: dependabot@1
  displayName: dependabot-$(Build.Repository.Name)
  enabled: true
  inputs:
    # These two inputs are needed to workaround a bug in dependabot that breaks authentication to private nuget feeds in some cases, see: https://github.com/tinglesoftware/dependabot-azure-devops/issues/921#issuecomment-2162273558
    # Once the following issue is resolved, bump the image tag to the version with the fix and the inputs can be removed: https://github.com/dependabot/dependabot-core/pull/8927
    dockerImageTag: '1.30.3-ci0004'
    extraEnvironmentVariables: 'WORKAROUND_CMD=sh -c "$(curl -fsSL https://aka.ms/install-artifacts-credprovider.sh)";NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED=true;VSS_NUGET_EXTERNAL_FEED_ENDPOINTS={"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/blackmarble-source/_packaging/BM-Libs/nuget/v3/index.json","username":"unused","password":"$(System.AccessToken)"}]}'
    targetRepositoryName: $(Build.Repository.Name)
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

and the nuget.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
<config>
    <add key="internalfeed" value="https://pkgs.dev.azure.com/blackmarble-source/_packaging/BM-Libs/nuget/v3/index.json" />
</config>
</configuration>

Have I missed something obvious, or misunderstood ?

[rhyskoedijk]

Sorry, one other thing I forgot to mention is that you'll need to use the "vNext" update script as the auth changes were not backported to the [default] update script that you are using. I will put up a PR to backport this change, but for now you should be able to test this using the vNext script with:

- task: dependabot@1
  inputs:
    dockerImageTag: '1.30.3-ci0004'
    useUpdateScriptvNext: true # <-- use the new script, which has the auth fixes
    targetRepositoryName: $(Build.Repository.Name)
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

I can see from your logs that the username being detected is still "user", which tell me the fix was not run.

Using credentials from config. UserName: user

If the fix is running, it should say "UserName: PAT".

The nuget.config file shouldn't be required in your scenario, this was just something I noticed and fixed at the same time as your issue.

[bm-fez]

@rhyskoedijk adding useUpdateScriptvNext: true fixed the issue thanks