link: don't link openssl. #597
Conversation
| Refusing to link: openssl | ||
| Linking keg-only OpenSSL means you may end up linking against the insecure, | ||
| deprecated system version while using the headers from the Homebrew version. | ||
| Instead, specify the full include/library paths to your compiler. |
There was a problem hiding this comment.
The “[…] to your compiler.” reads kinda awkward to me. Maybe “[…] during the build.” works better?
|
👍 and one minor nit. |
|
Should we add something like "Consider running |
|
I think it should be allowed when prefix isn't exactly |
|
Blog post explaining this phenomenon: https://langui.sh/2015/07/24/osx-clang-include-lib-search-paths/ |
See https://langui.sh/2015/07/24/osx-clang-include-lib-search-paths/ for an explanation and https://gist.github.com/tdsmith/4b502c5cc6e7d358acdf for reproduction instructions.
|
Have tweaked message, added compiled path guidelines based on projects that do this properly and made this only apply to |
|
Oh, and added that blog post to the commit message. |
|
@MikeMcQuaid https://github.com/dotnet/core We'll happily accept different instructions to provide. We change the dylib reference to use rpath for the specific version; so what we care about the most is that libcrypto.1.0.0.dylib and libssl.1.0.0.dylib end up on the default probing path. We don't care about the unversioned library forwarder, or particularly care about the header file path (since the unlinked location is good enough to be found by cmake). So something like |
Can you explain in a bit more detail what this means? I don't understand.
Why is this necessary? Why can't you look into an arbitrary user-provided path? Not everyone installs Homebrew to
This looks like a very narrow use case for a single consumer, so I doubt we're going to provide something like this. (But as stated in the previous paragraph, I think you're limiting yourself unnecessarily by relying on the default lookup paths and it would be good to understand where this limitation comes from before we can discuss possible alternatives for you and your users.) |
|
@UniqMartin We want to support a) homebrew users, b) macports users, and c) people who want to distribute a standalone copy of the library with their application. rpath seemed to be the best way to do this. In particular, since we distribute precompiled assets (vs building on the end user machines) it would be the path it had on our build machines that was encoded for the library load, so we need to be flexible on the install destinations. |
|
@bartonjs Have you considered using On macOS you're expected to link to libraries with their full path (either by directly using an absolute path or by using paths relative to the executable or loading dylib). What you currently seem to rely on for production use is what is described as A more macOS-like experience would be of course to bundle all necessary libraries with your package. And if your crypto component relies on OpenSSL as a backend and cannot use the libraries natively provided by the OS, the most user-friendly solution would be to bundle those external dependencies to make your package fully self-contained.
The paths you've used during compilation are not set in stone. On macOS, it is fairly common for build systems to adjust those (usually absolute) paths after binaries have been built by using |
As @UniqMartin says: unfortunately we're not going to provide this.
Agreed. |
I'm not sure I follow. We certainly know where the libraries are on our build machines, but we have no idea where the user's machine would have put them. We already use
If I recall correctly, the rpath/probing support is 'relatively' recent on MacOS, meaning that Apple saw a need for more flexibility (and to acting like all the other OSes). And CMake certainly seems to think that rpath-flexibility is better: https://cmake.org/cmake/help/v3.0/prop_tgt/MACOSX_RPATH.html.
Yeah, but then we're in the boat of needing to release an updated Mac release every time there's an OpenSSL security fix, which (I'm sure you're aware) is a fairly regular occurrence. So, since we've already shipped, and this change now makes our getting started instructions not function, do you have any particular recommendations? Is there an ABI-version-only path under a default Our goals:
Heck, now that I've written this I feel like I have to ask: Why can't |
I don't think delegating the responsibility of that to Homebrew is particularly responsible to your users. I'm afraid regardless of need: your use-case is not something we're willing to support. Sorry. |
You're basically talking about We also don't promise API/ABI compatibility IRT to external projects; it won't happen immediately but OpenSSL 1.1.0 is due to land at some point & eventually will take the If you want stability, bundling the libraries into your project is a reasonable step & the "done" thing on OS X/macOS. I understand it can be a frustration around having to issue updates purely for OpenSSL security issues, but that is part of the OS X/macOS ecosystem really & is what Apple themselves try to prod people towards. |
Still they expect you to link to a library using it's absolute path, and relying on the default search path of the dynamic linker is mostly a developer feature (in my understanding). That you can use
It's just stating that the system is more flexible (which is true). It's not saying that it's better.
Because that's not in the design of Homebrew. We either link everything outside of Ironically, if Apple had just removed the To be a bit frank, you've been relying on discouraged behavior from the start. That you had to use the It's obviously up to your project how you operate and where your priorities are, but I think on that matter you should really consider bundling OpenSSL as a short-term solution and then port your SSL/TLS layer to use Secure Transport on macOS as long-term solution (because that's what Apple expects you to use and where you are covered by the vendor's security updates). |
I'm hoping they'll eventually move the remaining OpenSSL libraries to a private prefix inside |
|
Alright, we seem to have come up with our immediate workaround. Sorry for the derailment.
Yeah, we're watching for that. I don't suppose that any of the package management-style systems are looking too fondly at the non-backwards-compatible release.
Wouldn't that've been grand?
It had a clear point of confusion/contention in 10.10, but on 10.11 it seemed to not (however, that seems to have been an incorrect conclusion). Since 10.11 was our minimum MacOS we followed the prevailing "how do I make this work?" advice from the internet. Though, clearly, "prevailing" isn't guaranteed "right".
It's (sort of) there, but I can see how it would get missed/not-understood. There's a Community link in the header which then lists all of the Github projects which feed into the effort; but it doesn't have an obvious "if you're trying to give us advanced warning" focus, which I'll pass on.
FWIW, that is the long-term plan. Situations like this one certainly remind us of the importance of that effort. BTW: If you hadn't noticed yet, some of your users are pretty clever at finding workarounds, and they found that |
|
Thanks for the detailed reply, for taking some action (e.g. more obvious contact information), and for understanding our point of view. This is much appreciated! ❤️
We noticed: #612. I'm looking forward to the creative workarounds they will find next. Sadly, quite a few users (and I think they might be mostly your users and have installed Homebrew due to your instructions) will ignore all warnings and do the unsafe thing nonetheless, because they just want their toy to work and then move on. Spreading this bad advice, as has happened in a few of your GitHub issues and on StackOverflow, is unfortunate, but I guess there's nothing we can do about this. |
This extends the approach in #597 to further prevent linkage of formulae that conflict with the system OpenSSL and can cause the issues described in that issue.
I think this is mostly on us; Homebrew should (and, in this case, now does) actively stop people from doing things we know to be harmful rather than allowing them. Code is much easier to change than documentation here even if the impact isn't quite as pleasant, unfortunately.
FWIW I was the one who looked at https://www.microsoft.com/net/core#macos, looked for a GitHub link, looked at https://github.com/dotnet/core/issues/new and didn't think any of those repos looked suitable to report to.
That's just been fixed as-of #612 being merged 😆 |
|
@MikeMcQuaid No worries. Either the prepopulated message there is new, or many people just erase it and post a new issue (possibly when they hit "uh.. which one?"). Honestly I can't even tell you which one I'd have thought the right answer is. dotnet/corefx is where the dependency comes from, but that's not the same as saying "the docs are bad". But, in the end, we got it sorted out. |
|
Glad you got it sorted. |
As someone who came to this discussion while trying to determine what to do when the instructions at https://www.microsoft.com/net/core#macos failed, I am wondering how and when I will learn of your immediate workaround. I'd rather wait for that than do something insecure with openssl. |
|
So will the issue fixed by microsoft dotnet project? |
|
@chenbojian @talkingscott you might want to follow https://github.com/dotnet/cli/issues/3964 rather than messing more with this on the Homebrew PR. As an aside, a great way to keep developers happy and productive is to hit the subscribe button instead of asking when something will be fixed or commenting with a +1. |
Due to Homebrew/brew#597, .NET Core's installation instructions of `brew link --force openssl` are now invalid (for good reasons, due to security holes). Until .NET Core updates their libraries to find the OpenSSL libraries correctly, we can update them as part of `Start-PSBootstrap`. See https://github.com/dotnet/cli/issues/3964
|
.NET Core for macOS users: In light of the Homebrew changes (and resultant/associated commentary) here we've increased the priority of the move off of OpenSSL on macOS. https://github.com/dotnet/corefx/issues/9394 is the main issue that will be tracking the incremental progress (which already has one linked PR), and any discussion should be there (rather than over here on the Homebrew side of things). (Sorry for the somewhat-of-a-cross-post, but wanted to leave breadcrumbs for any latecomers) |
Homebrew no longer allows this per Homebrew/brew#597
|
I ran the installer as Microsoft suggested. brew update I had to add 2 symlinks. 1.) ln -s /usr/local/Cellar/openssl/1.0.2j/bin/openssl openssl The first symlink made it so openssl version showed the right version. The installer still complains about the openssl version, but appears to have worked. |
brew testswith your changes locally?See https://gist.github.com/tdsmith/4b502c5cc6e7d358acdf for reproduction instructions of the security hole this creates. Unfortunately this is recommended in locations like https://www.microsoft.com/net/core#macos (which provide no specific contact information) that we will unfortunately break but there's not really any other way around. This will scale better than the current system of "Homebrew maintainers notice sites recommend this and ask them to not do so".