Unix: machine certificate store throws on load when it contains certificates the user has no read permission for #26294
Comments
|
This was added in dotnet/corefx#29351 Instead of skipping the cert file we are now throwing. The handle is not valid for files that the user can't read. |
tmds
changed the title
|
cc: @pjanotti |
|
The files that trigger the issue are not part of the default system installation. So it does not affect every system. You need to have installed a package that brings such files. In my case, they come with the package that configures my system for access to the corporate vpn. |
This is a regression in 2.1.0 (dotnet#29351). Fixes https://github.com/dotnet/corefx/issues/29942.
|
@tmds for shiproom do you have data suggesting how commonly systems are configured this way? Is there a feasible workaround? Do you know how long this was regressed? |
For me it is due to the company vpn package adding certificates that are only readable by root.
The workaround is to change permissions on the system certificates. You need to have root permissions and be willing to change permissions on those files. This may have security side-effects.
It regressed after 2.1-rc1. If you're not including it for servicing we may patch our .NET Core builds for Fedora/RHEL/CentOS. Our preference is to minimize such differences, so that's why I'm asking here first. CC @omajid |
|
That's a very recent regression - you're sure the certificates were present at the time you tested with RC1? @stephentoub any idea where the regression might have been? |
https://github.com/dotnet/corefx/pull/29351/files#diff-28edeba8fcd86130f28b8c85afd19f37R257 |
|
I missed this earlier... yes, the recent change to keep the SSL error queue clean introduced this. |
|
We've also encountered this issue on CentOS 7.5.1804. Our install of Apache (2.4.6) and mod_ssl created a default localhost.crt which the user we are running the site under did not have access to. To avoid using a permissions workaround (running the site as root or giving read access to the certificate file) we simply moved the localhost.crt to a different folder and updated the Apache config. |
* Skip certificates we can't read when populating machine store. This is a regression in 2.1.0 (#29351). Fixes https://github.com/dotnet/corefx/issues/29942. * Add test * Assert chmod returns 0 * Skip test on OSX * Add valid content to the unreadable cert file
…t#29973) * Skip certificates we can't read when populating machine store. This is a regression in 2.1.0 (dotnet#29351). Fixes https://github.com/dotnet/corefx/issues/29942. * Add test * Assert chmod returns 0 * Skip test on OSX * Add valid content to the unreadable cert file
|
Fixed in master |
|
@danmosemsft in the meanwhile, it is not only folks with our vpn package that are hitting this. |
|
For me, this bug was triggered by a broken symlink in |
|
@dlech thanks. Hopefully until this can be serviced, folks getting this stacktrace will find this issue and figure out the workarounds. |
|
I get this same error when a chromecast device tries to hit my nginx server running emby which depends on mono. Should I be putting in a issue with the mono group, or is this upstream? Sorry, just came here by googling my errors. |
|
Interesting about Mono. @marek-safar do you guys need the fix dotnet/corefx#29973? |
|
@danmosemsft not yet, we didn't switch X509Certificates yet |
|
I'm getting a similar error -- except it is Interop+Crypto+OpenSslCryptographicException: error:2006D080:BIO routines:BIO_new_file:no such file without any indication of the problematic file. I suspect the pull request that addresses this issue may also fix this... Any way to determine what the problematic file would be? |
|
@hpinsley Try an |
|
@bobberb it looks like you should open an issue directly against Mono as they don't use our code for this yet. |
|
@omajid Thanks. I neglected to mention that this was running in a Docker container. Note that I tried to install strace but when I ran it, it didn't show the failed file open for some reason. It turned out to be a broken symbolic link in /etc/ssl/certs. We are running the container on a Linux host that has our corporate certs installed. To get SSL to work in our .NET Core code running within a hosted container, I was mapping the host's /etc/ssl/certs folder into the container. As I had already seen symbolic links from /etc/ssl/certs to /usr/share/ca-certificates, I mapped that folder as well but missed a single file that was linked to /usr/local/share/ca-certificates. I solved my problem by also mapping /usr/local/share/ca-certficates. |
|
We are having this issue after upgrade to 2.1 (from 1.1) with Ubuntu 16.04.1. There is really nothing special about this machine. Probably some of the problematic certificates are installed by AWS LB. I'm not very familiar with how the releases are happening so what are our options in this case to get this fix? |
|
@ydekova until the patch is out, you could use a nightly build: https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/dogfooding.md#advanced-scenario---using-a-nightly-build-of-microsoftnetcoreapp Or you could follow this suggestion https://github.com/dotnet/corefx/issues/29942#issuecomment-396420408 to try to find out the problematic cert and either change access to it or move it. |
|
@danmosemsft thanks for your quick reply. I tried to run the process with root user, check the /etc/ssl/certs/ for broken links and what their permissions are, but none of those worked for us. I managed to work around it by overriding the server certificate validation via the ServerCertificateCustomValidationCallback property of the HttpClientHandler. Here is what I found: The certificate (cert parameter in the method above) seems valid (It is issued by Microsoft graph/outlook API, subject name is "CN=Outlook.office.com, O=Microsoft Corporation, L=Redmond, S=Washington, C=US", I'm no expert in certificates though..). I doubt it the problem is with the certificate because:
The SslPolicyError (errors param in the above code) is RemoteCertificateNameMismatch Any idea what might this means? I don't feel comfortable with disabling ssl certificate validation altogether. If I ignore this kind of error in the method above is this a potential security bridge? |
|
@ydekova did you get a stacktrace that contains these lines: Try running this on your system: using System;
using System.Diagnostics;
using System.IO;
using System.Runtime.InteropServices;
namespace console
{
class Program
{
internal const string CryptoNative = "System.Security.Cryptography.Native.OpenSsl";
static void TryOpenFile(string filename)
{
try
{
using (File.OpenRead(filename)) {}
}
catch (Exception ex)
{
System.Console.WriteLine($"--> Cannot open file '{filename}': {ex.Message}");
}
}
static void ListFiles(string path, bool isDir)
{
System.Console.WriteLine($"ls {path}");
Process.Start("ls", $"-lah \"{path}\"").WaitForExit();
System.Console.WriteLine();
if (isDir)
{
foreach (var file in Directory.GetFiles(path))
{
TryOpenFile(file);
}
}
else
{
TryOpenFile(path);
}
System.Console.WriteLine();
}
static void Main(string[] args)
{
ListFiles(GetX509RootStorePath(), isDir: true);
ListFiles(GetX509RootStoreFile(), isDir: false);
}
internal static string GetX509RootStorePath()
{
return Marshal.PtrToStringAnsi(GetX509RootStorePath_private());
}
internal static string GetX509RootStoreFile()
{
return Marshal.PtrToStringAnsi(GetX509RootStoreFile_private());
}
[DllImport(CryptoNative, EntryPoint = "CryptoNative_GetX509RootStorePath")]
private static extern IntPtr GetX509RootStorePath_private();
[DllImport(CryptoNative, EntryPoint = "CryptoNative_GetX509RootStoreFile")]
private static extern IntPtr GetX509RootStoreFile_private();
}
} |
|
@tmds this program gives me ls /usr/lib/ssl/certs ls /usr/lib/ssl/cert.pem --> Cannot open file '/usr/lib/ssl/cert.pem': Could not find file '/usr/lib/ssl/cert.pem'. And this is the stack trace of my apps exception as far as i can track |
|
@ydekova it is a different issue, I've created https://github.com/dotnet/corefx/issues/30388 based on the info you gave. It should work on 2.1 when using the curlhandler (set DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER=0). |
|
@tmds thanks for your assistance. DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER - if this is an environment variable setting it to 0 didn't really fixed it for us. I'll add my temp workaround here https://github.com/dotnet/corefx/issues/30388 |
…t#29973) * Skip certificates we can't read when populating machine store. This is a regression in 2.1.0 (dotnet#29351). Fixes https://github.com/dotnet/corefx/issues/29942. * Add test * Assert chmod returns 0 * Skip test on OSX * Add valid content to the unreadable cert file
…t#29973) * Skip certificates we can't read when populating machine store. This is a regression in 2.1.0 (dotnet#29351). Fixes https://github.com/dotnet/corefx/issues/29942. * Add test * Assert chmod returns 0 * Skip test on OSX * Add valid content to the unreadable cert file
… (#30155) * Skip certificates we can't read when populating machine store. This is a regression in 2.1.0 (#29351). Fixes https://github.com/dotnet/corefx/issues/29942. * Add test * Assert chmod returns 0 * Skip test on OSX * Add valid content to the unreadable cert file
|
Fixed in release/2.1 branch in PR dotnet/corefx#30155. Will be part of 2.1.3 release. |
|
You can also try the following on Ubuntu, if you have invalid certs for whatever reason: This will remove invalid symlinked certs for you. |
|
@joshuataylor Thanks, that fixed the problem for me! |
|
The 2.1.3 release fixed this for me on Cent OS 7.5. Thanks for getting this resolved. |
|
The 2.1.3 release is not yet available on the Ubuntu 17.10 apt repositories. |
|
@leecow is that expected? @Beanow can you please raise it on dotnet/core repo? |
|
Ubuntu 17.10 is EOL: |
|
Thanks for pointint it out @mkurz |
|
17.10 went EOL 7/19/2018 and 2.1.3 released 8/21. See https://github.com/dotnet/core/blob/master/os-lifecycle-policy.md for .NET Core version / distro version support details. |
ghost
locked as resolved and limited conversation to collaborators
On Fedora 27:
With
2.1.300-rc1-008673givesBut with the
2.1.300(https://github.com/aspnet/Home/wiki/2.1.0-Early-Access-Downloads) throws:The text was updated successfully, but these errors were encountered: