Problems with dotnet restore on Unbuntu 18.04.1: SPNEGO cannot find mechanisms to negotiate

[jeroenlandheer]

I've installed the .Net Core SDK, added a authenticated feed from a local TFS instance and I get this error when running dotnet restore:

/usr/share/dotnet/sdk/2.2.101/NuGet.targets(114,5): error : Unable to load the service index for source https://xxxxxxxxx/tfs/xxxxxxx/_packaging/xxxxx/nuget/v3/index.json. [/home/beheerder/net/hello/hello.csproj]
/usr/share/dotnet/sdk/2.2.101/NuGet.targets(114,5): error :   GSSAPI operation failed with error - An invalid status code was supplied (SPNEGO cannot find mechanisms to negotiate). [/home/beheerder/net/hello/hello.csproj]

The server is using NTLM and Kerberos Authentication, we have no issues with using these feeds on Windows.

.Net SDK v2.2
Tried on Ubuntu 18.04 and 16.04, both have the same issues.

On IIS Extended Protection is off, Kernel-mode authentication is on, Negotiate and NTLM provider are both enabled. Besides that Anonymous authentication is enabled on the /tfs directory. (Standard configuration AFAIK.) SPN of the server is registered for Kerberos Auth. (HTTP/my-server-address points to the TFSService account)

[davidsh]

We see this problem frequently on Linux. The root cause is that the Linux client machine is failing to downgrade from kerberos to NLTM during the Negotiate scheme.

The server might offer both HTTP auth schemes such as Negotiate and NTLM. Given that, the client will take Negotiate as the scheme. Negotiate will try Kerberos first. But frequently this is not possible because the Linux machine is not configured to work against the Windows Active Directory domain (i.e. it is not "joined" to the domain). So, Negotiate scheme will then try to downgrade to NTLM. However, on Linux, this doesn't work also due to the GSSAPI provider on Linux not being able to downgrade from kerberos to NTLM using the Negotiate scheme.

In cases like this where mixed operating systems (Windows and Linux) are used, a workaround is to only use NTLM on the server. Or to re-configure the Linux client machine so that it can probably downgrade from kerberos to NTLM during the Negotiate scheme.

[davidsh]

@karelz

[jeroenlandheer]

@davidsh Only NTLM on the server isn't an option because we rely on Kerberos for too many things here. So I guess my question is... how do I configure my Linux machines to properly downgrade to NTLM, or join it to AD so I can use Kerberos on those machines? (I also think that disabling Kerberos isn't a best practice.)

Most of the install here is fairly standard, so I guess when others install TFS on prem and try to use dotnet core on Linux the way we do here, they are going to run into the same kind of issues.

We have at the moment for our build agents a workaround in place, they have this same issue.
We use Mono+NuGet to restore packages from our feeds and use the --no-restore option on dotnet to build our solution. NuGet uses a credential provider and is able to authenticate.

I've tried using a credential provider for dotnet, but I can't seem to get this to work.

[davidsh]

how do I configure my Linux machines to properly downgrade to NTLM, or join it to AD so I can use Kerberos on those machines?

Take a look at this issue #26418. If you read thru the issue toward the end, I did some research about how to properly configure a Linux machine for Kerberos so that it would be part of a Windows Active Directory domain. There are also some links there with more information.

[davidsh]

GSSAPI operation failed with error - An invalid status code was supplied (SPNEGO cannot find mechanisms to negotiate)

This error with the specific text "SPNEGO cannot find mechanisms to negotiate" usually refers to a problem trying to use Kerberos but no Kerberos ticket is active. On Windows, there is the concept of "default credentials" which are the credentials of the logged on user. But on Linux, there isn't such a concept. If you want the equivalent of it, you need to create a Kerberos ticket with the user identity of someone that has a valid Windows Active Directory account. Then, the GSSAPI with Kerberos will be able to use that as the "default credentials".

See these instructions: https://github.com/dotnet/corefx/issues/30203#issuecomment-395990520

Specifically you will need to use 'kinit' in order to establish a default Windows credential on the Linux machine.

[davidsh]

how do I configure my Linux machines to properly downgrade to NTLM,

It turns out that Negotiate being able to downgrade from Kerberos to NTLM is not working on Linux. This is a bug in our HTTP stack on Linux.

See: dotnet/corefx#34878.

[davidsh]

Fixed with PR dotnet/corefx#35383

[ryanthedev]

When can we expect this to be released?

[davidsh]

When can we expect this to be released?

It is fixed in the master branch which is .NET Core 3.0. .NET Core 3.0 is now available in preview releases. See:

https://devblogs.microsoft.com/dotnet/announcing-net-core-3-preview-3/

[kirkone]

Hi.

I ran into this issue again. I am using preview 8 and 9 of dotnet 3.0.
The issue occurs in an ubuntu based docker container while running dotnet restore.

Any ideas?

[davidsh]

@kirkone - Please open a new issue to describe your problem. Be specific in terms of your network topology and what kinds of authentication is being used by servers and/or proxy. Enterprise authentication issues such as this are difficult to diagnose generally without a lot of information to reproduce the problem. Thanks.