Installing and Running on OS X 10.11 SSL Fails to Link Using Brew

[idavis]

Steps to reproduce

Following steps here: https://www.microsoft.com/net/core#macos

brew update
brew install openssl
brew link --force openssl

Expected behavior

System is configured

Actual behavior

$ brew link --force openssl output:

Warning: Refusing to link: openssl
Linking keg-only OpenSSL means you may end up linking against the insecure,
deprecated system version while using the headers from the Homebrew version.
Instead, pass the full include/library paths to your compiler e.g.:
  -I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib

If I try to use the CLI, I get crypto errors:
$ dotnet restore output:

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Crypto' threw an exception. ---> System.TypeInitializationException: The type initializer for 'CryptoInitializer' threw an exception. ---> System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Interop.CryptoInitializer.EnsureOpenSslInitialized()
   at Interop.CryptoInitializer..cctor()
   --- End of inner exception stack trace ---
   at Interop.Crypto..cctor()
   --- End of inner exception stack trace ---
   at Interop.Crypto.GetRandomBytes(Byte* buf, Int32 num)
   at System.IO.Path.GetCryptoRandomBytes(Byte* bytes, Int32 byteCount)
   at System.IO.Path.GetRandomFileName()
   at Microsoft.DotNet.InternalAbstractions.TemporaryDirectory..ctor()
   at Microsoft.Extensions.EnvironmentAbstractions.DirectoryWrapper.CreateTemporaryDirectory()
   at Microsoft.DotNet.Configurer.NuGetPackagesArchiver..ctor()
   at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(INuGetCacheSentinel nugetCacheSentinel)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)
Abort trap: 6

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview2-003121)

Product Information:
 Version:            1.0.0-preview2-003121
 Commit SHA-1 hash:  1e9d529bc5

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.11
 OS Platform: Darwin
 RID:         osx.10.11-x64

$ brew -v output:

Homebrew 0.9.9 (git revision b999e; last commit 2016-07-29)
Homebrew/homebrew-core (git revision a69e; last commit 2016-07-29)

$ brew info openssl output:

openssl: stable 1.0.2h (bottled) [keg-only]
SSL/TLS cryptography library
https://openssl.org/
/usr/local/Cellar/openssl/1.0.2h_1 (1,691 files, 12M)
  Poured from bottle on 2016-07-29 at 18:47:22
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/openssl.rb
==> Dependencies
Build: makedepend ✘
==> Options
--universal
    Build a universal binary
--without-test
    Skip build-time tests (not recommended)
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

This formula is keg-only, which means it was not symlinked into /usr/local.

Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries

Generally there are no consequences of this for you. If you build your
own software and it requires this formula, you'll need to add to your
build variables:

    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include

[lokinfey]

I have the same problem about that.

[fhibf]

I have the same problem about that (2).

[idavis]

@lokinfey @FerHenrique Did you very recently install an OS update? When I opened my laptop yesterday, There was a pending OS update. I ran the latest update, rebooted, then tried to get dotnet core running.

[tomliversidge]

Same problem here

.NET Command Line Tools (1.0.0-preview2-003121)

Product Information:
 Version:            1.0.0-preview2-003121
 Commit SHA-1 hash:  1e9d529bc5

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.10
 OS Platform: Darwin
 RID:         osx.10.10-x64

brew -v

Homebrew 0.9.9 (git revision 1244; last commit 2016-07-30)
Homebrew/homebrew-core (git revision 37d2; last commit 2016-07-30)

[fhibf]

@idavis @lokinfey Yesterday I installed the update OS X El Capitan 10.11.6.

brew -v

Homebrew 0.9.9 (git revision 1244; last commit 2016-07-30)
Homebrew/homebrew-core (git revision 37d2; last commit 2016-07-30)

dot net --version

1.0.0-preview2-003121

[MineLucky]

I have the same problem about that (3),and installed OS update too.

[chanans]

I follow the solution from the link below and issue is resolved.
http://stackoverflow.com/questions/38670295/brew-refusing-to-link-openssl

[idavis]

@chanans I followed those steps and everything is running. Thanks! I think the install instructions will have to be updated with this breaking change.

[nbellocam]

This is related with Homebrew/brew#597

[carlsoncoder]

I followed the same instructions on the link @chanas mentioned, and I'm still getting the "Refusing to link" message. Anyone have any other ideas?

[nbellocam]

@carlsoncoder did you set yourself as owner of the /usr/local folder?

sudo chown -R whoami /usr/local

To recap, for workaround this issue you need to:

1. Remove the openssl version you installed (1.0.2): brew uninstall openssl
2. Set yourself as owner of the /usr/local folder (the -R is for recursively): sudo chown -R whoami /usr/local
3. Install version 1.0.1 of openssl: brew install homebrew/versions/openssl101
4. Perform the linking: brew link --force homebrew/versions/openssl101

[carlsoncoder]

@nbellocam - thanks for the quick reply. What I ended up doing (may help some others), is I found out how to just update the OpenSSL version on my Mac to latest (1.0.2h) from source.

[http://bytefish.de/blog/net_core_mac_os/]

Once I did this, "openssl version" reported 1.0.2h, and the dotnet new command ran without errors.

I understand this might not be the BEST approach (if you had something else on your system depending on a specific version of OpenSSL), but it worked for me!

[fhibf]

Hi all!
Now it's running.

I uninstalled OpenSSL, installed OpenSSL 1.0.1, linked brew with OpenSSL 1.0.1, executed 'dotnet new', 'dotnet restore' and 'dotnet run'.

Everything worked well.
Tks!

[chanas]

I think I may have been added to this conversation by mistake.

Thank you,

C. Hanas

Teacher of Principles of Engineering
Teacher of Computer Science and Software Engineering
HTHS Robotics & Coding Club Adviser
High Technology High School
765 Newman Springs Rd
Lincroft, NJ 07738

On Sun, Jul 31, 2016 at 9:20 PM, Justin Carlson [email protected]
wrote:

I followed the same instructions on the link @chanas
https://github.com/chanas mentioned, and I'm still getting the
"Refusing to link" message. Anyone have any other ideas?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dotnet/cli/issues/3964#issuecomment-236470039, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AHW1DGmWcsdAnm51jISXzuZAAATSTPx3ks5qbUnkgaJpZM4JYtzq
.

[joshka]

The stackoverflow / earlier version of openssl approach will cease working once you update brew (see Homebrew/brew#612).
The following workaround worked for me on a hello world project:

export DYLD_LIBRARY_PATH=/usr/local/opt/openssl/lib
dotnet new

See https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/DynamicLibraryUsageGuidelines.html#//apple_ref/doc/uid/TP40001928-SW21 for info on how the library search process works on OS X.

[bartonjs]

When the documentation gets updated it will be suggesting of manually bringing in the dylib symlinks, but not doing the rest of the work that brew link did. Therefore the recommendation is

ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

after doing brew install openssl (the 1.0.2 version)

[joshka]

Thanks Jeremy,
After having another read of the homebrew issue, it seems the HomeBrew crew are against that option due to the potential to break unknown things globally. I.e. any software that expects the headers it uses to align to the libraries installed in /usr/local/lib will now be in an inconsistent state. That said, my DYLD_LIBRARY_PATH suggestion causes the same problem.

I looked into the other option that was suggested for setting the rpath on the library. I think the following is a better solution that will only effect this specific library.

sudo install_name_tool -add_rpath /usr/local/opt/openssl/lib /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/System.Security.Cryptography.Native.dylib

In effect, rather than telling the operating system to always use the homebrew version of SSL and potentially causing something to break, we're telling dotnet how to find the correct library.

Additionally this solution can be easily incorporated into the homebrew cask version of the installation process as it should know where openssl is installed (I believe that this was your contention to this approach).

As an aside, I found it a little odd that the sudo was necessary to fix this up as most things I've installed in /usr/local via homebrew are owned by "myuser:admin". Is there are reason why this is necessary / suggested for the dotnet installation?

[MikeMcQuaid]

I looked into the other option that was suggested for setting the rpath on the library. I think the following is a better solution that will only effect this specific library.

From the Homebrew perspective: I also agree this is a better approach. Relying on symlinks as @bartonjs has suggested may not work consistently.

[bartonjs]

@joshka I had no idea that someone had made a dotnet installer for brew (or, at least, brew cask). This makes me feel warm and fuzzy for some reason.

Asking brew where it put openssl and doing add_rpath sounds great to me when a builtin script is doing it. The problem I have with recommending that as the general approach is that the install_name_tool command feels to me like it's more black magic than ls. So macports users, and perhaps build-from-source users, might not know that /usr/local/opt/openssl/lib might not match their environment. Or maybe I have completely the wrong opinion here, since my Mac experience is mostly limited to using bash to test my code before checkin.

Is the uh.. cask... something that we should be maintaining and/or encouraging people to use? I'm intrigued.

[joshka]

@bartonjs had it not been there already, it was next on my list of things to do. I'm thankful that someone else did it already also :)

For a build-from-source user, the right thing to do is tell the build process where the libs are, whether that's installed via homebrew and living in Cellar / /usr/local/opt or installed via macports and living somewhere else. If you're building on my machine, use my libs. A quick thought experiment for you: lets say I'm a dev working on a security bug outside fix for openssl outside of the system level directories, how else would I check that dotnet is / isn't vulnerable? Alternatively I'm a .Net web developer and I want to test my site against a recently released version of openssl to ensure nothing breaks so that I can roll the fix out to production before I get pwned.

See https://github.com/Homebrew/brew/blob/028d155e97014076adf4e4dbc7c888fd871de321/share/doc/homebrew/How-to-build-software-outside-Homebrew-with-Homebrew-keg-only-dependencies.md for how this fits in a build that lives outside the homebrew world,
or https://github.com/Homebrew/homebrew-core/blob/ae2206f3e5bb2a7c0065ae1b164d2d011b85858b/Formula/clamav.rb#L38 for how it lives if you're calling the build from within homebrew.

Regarding changing the library's rpath after it's installed, I think a good analogy that fits is the bindingRedirect element of your app.config in .Net. This isn't 'magic' so much as the right way to redirect versions of libraries that don't match the environment.

As for the last question, IMHO yes, install should be as easy as brew install dotnet with nothing else needed (no brew linking or symlinking libs). MacPorts users may suggest that it's also useful to maintain a second macports approach, but I can't comment on that.

In summary, the following make the most sense to me:

• If building dotnet from source, allow the user to set the correct lib path and bake that path into the dylib
• If installing from the pkg file, use the install_name_tool to set the correct rpath on the dylib
• If installing from homebrew-cask do the install_name_tool fix automatically (I'm happy to put a PR for this).

As repetition from above, my suggested fix is:

sudo install_name_tool -add_rpath /usr/local/opt/openssl/lib /usr/local/share/dotnet/shared/Microsoft.NETCore.App/1.0.0/System.Security.Cryptography.Native.dylib

[zh-wowtv]

The only method works for me is to install from source code. Please refer to: http://bytefish.de/blog/net_core_mac_os/

[joshka]

@zh-wowtv can you please provide some insight about what in the solution above doesn't work for you? This would be helpful in providing a more generic solution to others.

[MikeMcQuaid]

Homebrew maintainer here!

If building dotnet from source, allow the user to set the correct lib path and bake that path into the dylib

Agreed 👍

If installing from the pkg file, use the install_name_tool to set the correct rpath on the dylib
If installing from homebrew-cask do the install_name_tool fix automatically (I'm happy to put a PR for this).

Agreed as a short-term hack but the "right" OS X solution is to bundle OpenSSL with your software. As a longish-term OS X user/dev anything that asks the user to manually to install things before installing/running it points to a badly configured installer or application bundle.

[zh-wowtv]

@joshka Installing from the source code is working for me. But the other way they mentioned not working.

[joshka]

@MikeMcQuaid I agree with your points there.

@zh-wowtv without more information on what didn't work and how it didn't work, it's difficult to provide more assistance. Can you go into a bit more detail about what you tried and what you saw?

It may help to have a read of http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

[MikeMcQuaid]

It may help to have a read of http://www.chiark.greenend.org.uk/~sgtatham/bugs.html

Or, in short:

To help us debug this issue can you explain:

• What you were trying to do (and why)
• What happened
• What you expected to happen
• Step-by-step reproduction instructions (with as minimal input data as possible)

[derakoola]

1. install port: https://guide.macports.org/
2. install or upgrade openssl package: sudo port install openssl or sudo port upgrade openssl
3. that's it, run openssl version to see the result.

[TheRealPiotrP]

Installation instructions have been updated and verified. Closing the remaining issue.

[MikeMcQuaid]

@piotrpMSFT Homebrew maintainer here: your solution is a bad one for a few reasons so I don't think you should close this issue. These instructions will break when Homebrew upgrades to a newer version of OpenSSL so you're just delaying the inevitable, here. The right solution (as has been stated multiple times in this and other threads): if your macOS application needs a library: you ship it with the .pkg file and embed it in the relevant .app files. If you are not able to do this then a worse solution (but better than this change) is @joshka's solution above is to set the RPATH correctly.

[MikeMcQuaid]

@piotrpMSFT I can probably find and recommend some paid macOS consultants if that would be more helpful.

[bartonjs]

@MikeMcQuaid If Homebrew had a fixed location for where it put OpenSSL AND you used the same location as a fixed location used by MacPorts, we'd set RPATH to that. But that isn't the case.

Since our current position is that we aren't bundling OpenSSL, our current instructions are the best middle-ground that we have come up with.

It's all temporary, anyways, since I'm working as fast as I can to drop the OpenSSL requirement on macOS (https://github.com/dotnet/corefx/issues/9394). If you know anyone who knows C# and SecurityTransforms, send'em my way; it'll possibly speed up the effort to make this discussion moot.

[MikeMcQuaid]

It's all temporary, anyways, since I'm working as fast as I can to drop the OpenSSL requirement on macOS

Good to know, thanks.

[TheRealPiotrP]

@bartonjs is there an issue we can reference to track the work you're doing?

[bartonjs]

https://github.com/dotnet/corefx/issues/9394

[TheRealPiotrP]

Ok. Closing this one. Cli is just a down level dependant

[katopz]

It's already close? I still getting this issue (all up to date) and I didn't see any correct step mention?

[bartonjs]

@katopz If the instructions at https://www.microsoft.com/net/core#macos (Install pre-requisites) don't work for you please open a new issue describing what you've done and what problem you're having.

[HanSeoulOh]

I am also still getting this issue following the steps outlined:

brew update brew install openssl brew link --force openssl

more specifically I'm also trying to update python's SSL but the following is not working for me as well.

brew install python --with-brewed-openssl

I resorted to building openssl myself following these steps:

http://stackoverflow.com/a/38710248

and using openssl version I can see that it has been installed correctly, however I am still unable to update python's openssl through brew

[atanasa]

Check the instructions again, those do not use the link option any more. This would allow your python to use the standard OS implementation of the crypto while managed code would use the openssl in /usr/local/lib.

[patrickbower]

I was also returning the error Warning: Refusing to link: openssl

Each package was returning the same message with both a Ignoring and a Try: prefix ie:

Ignoring eventmachine-1.0.4 because its extensions are not built. Try: gem pristine eventmachine --version 1.0.4

You can simply run each separate Try: command to restore it which will resolve (I presume reinstall), each one separately ie:

gem pristine eventmachine --version 1.0.4

If you have a large number of these messages as I did you could also put all the Try: packages into one line in an editor with each one separated with a && and run them together to save on doing each one individually.

This fixed my issue and is now working as expected.

[banshee]

Speaking as a long-time osx and brew user - your interim solution isn't a solution that your target audience is going to accept. I hit it when I was trying to get C# support working in Visual Studio Code. The window does tell me what's going on:

Finished
[ERROR] The debugger cannot be installed. A required component, OpenSSL, is not correctly configured.
In order to use the debugger, open a terminal window and execute the following instructions.
See https://www.microsoft.com/net/core#macos for more details.

  brew update
  brew install openssl
  mkdir -p /usr/local/lib
  ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
  ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

But my immediate reaction was that Visual Studio Code was doing something insane with OpenSSL.

[erensogut]

I have a problem about installing Visual studio Code on mac enviroment.I can not work the debugger because it gives the error like
The debugger cannot be installed. A required component, OpenSSL, is not correctly configured. In order to use the debugger, open a terminal window and execute the following instructions. See https://www.microsoft.com/net/core#macos for more details.
Also there are a solution to solve this problem in the fault screen ,despite I did this instruction I can not handle the problem.

[bartonjs]

@erensogut Did you follow the instructions at https://www.microsoft.com/net/core#macos?

[erensogut]

@bartonjs Yes I did follow them but that did not work.:(When I wrote the which openssl command it gives /usr/local/bin/openssl .I thought that was a origin a problem.Is it possible

[15949893621]

@nbellocam I did follow you but that did not work.

1. tuorungongsideMacBook-Air:~ tuorungongsi$ brew uninstall openssl
   Uninstalling /usr/local/Cellar/openssl/1.0.2k... (1,696 files, 12MB)
2. tuorungongsideMacBook-Air:~ tuorungongsi$ sudo chown -R whoami /usr/local
   chown: whoami: illegal user name

The problem is ：
tuorungongsideMacBook-Air:~ tuorungongsi$ brew install homebrew/versions/openssl101
Updating Homebrew...
Error: No available formula with the name "homebrew/versions/openssl101"
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
==> Searching taps...
Error: No formulae found in taps.
why? Please help me. Thanks!

[gerchicov-bp]

@Error: No available formula with the name "homebrew/versions/openssl101"
Still doesn't work

[MrVibe]

same issue. opensssl version returns : OpenSSL 0.9.8zh 14 Jan 2016
tried installing and uninstalling multiple times. Cant seem to update openSSL and cant seem to link it.

[gerchicov-bp]

@syberkitten
before linking with that issue you should metion that it appears with osx 10.12 too

[chauhannneha]

Hi,

Can anyone please help me with this issue which I am getting while running dotnet on my mac?
I came across a good number of solutions, which I tried implementing but I think I am just getting confused with so many options to resolve this issue.

Details are as followed:

.NET Command Line Tools (2.1.4)

Product Information:
Version: 2.1.4
Commit SHA-1 hash: 5e8add2

Runtime Environment:
OS Name: Mac OS X
OS Version: 10.13
OS Platform: Darwin
RID: osx.10.12-x64
Base Path: /usr/local/share/dotnet/sdk/2.1.4/

Microsoft .NET Core Shared Framework Host

Version : 2.0.5
Build : 17373eb129b3b05aa18ece963f8795d65ef8ea54

[elahmo]

@joshka regarding your comment on #6690 (comment)

Do you know is there a possible alternative for this on BigSur (and M1)? It seems there is no way to handle libssl/libcrypto and make it available to be used by AesGM ecncryption. I have tried many approaches, but DYLD_LIBRARY_PATH is ignored (even with SIP disabled), and symlinks are not being read.