Linux improvements (fixing linker args & Documentation updates)

[sebastianulm]

Added -nopie to default linker args (following a gitter conversation) to help compile on Linux distributions that use PIE by default.
Should solve #4988 (and a bunch of others)

Added some hints in the documentation to make CoreRT compile (and run) with more recent clang versions on Arch Linux
#6835

Let me know if there are any issues.

[dnfclas]

All CLA requirements met.

[jkotas]

Suggested change

	This is the default error message when a [DLLImport] could not be loaded. CoreRT nuget packages distribute this file, but it might be failing to load depencies.
	This is the default error message when a `[DllImport]` could not be loaded. CoreRT nuget packages distribute this file, but it might be failing to load dependencies.

[jkotas]

Suggested change

	In this Arch Linux example, libtinfo.so.5 is missing. Its part of ncurses5 but aur has a compatibility package here:
	In this Arch Linux example, libtinfo.so.5 is missing. Its part of ncurses5 but our has a compatibility package here:

[sebastianulm]

:) Not a Typo. Its an Arch Linux thing:
AUR is short for https://wiki.archlinux.org/index.php/Arch_User_Repository

But I'll change it to upper case, so its more clear.

[jkotas]

What is the error you are working around with -nopie?

If I understand it correctly, -nopie disables ASLR and thus makes the binaries less secure. We do not really want the CoreRT binaries to be less secure...

[sebastianulm]

Oh, I totally agree, and I hope you guys will be able to fix it "propper".
Here is what I pieced together so far:
The current binary distributed by the nuget package segfaults for me. To investigate, I checked out the repo and tried to build it. However, while compilation succeeded, all JIT-Tests fail with:
R_X86_64_PC32 against symbol `memcpy@@GLIBC_2.14' can not be used
at the Linking stage.

The reason behind this seems to be this, but I am unable to tell you if that is true.
https://stackoverflow.com/questions/43367427/32-bit-absolute-addresses-no-longer-allowed-in-x86-64-linux

Arch Linux (the distribution I use) ships with compilers that assume -use-pie by default, for security reasons. I assume that the build machine does not, (hence tests working)
Mainly because the distributed ILC itself has PIE switched off:

> checksec --file /home/yasuo/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/1.0.0-alpha-27317-01/tools/ilc
RELRO           STACK CANARY      NX            PIE             RPATH      RUNPATH	Symbols		FORTIFY	Fortified	Fortifiable  FILE
Partial RELRO   Canary found      NX enabled    No PIE          RPATH     No RUNPATH   No Symbols      Yes	0		5	/home/yasuo/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/1.0.0-alpha-27317-01/tools/ilc

So I decided to make a pull request, with this work around. AFAIK it does not make things works.
But I'd like a different fix much more, PIE adds a lot of benefits for a tiny performance hit.

I use this script: https://github.com/slimm609/checksec.sh to check if files are made with PIE.

I hope this information helps.
Edit: Adding a gist to the linker output here https://gist.github.com/sebastianulm/f1bba12ec15bd5cb707f04388b171edd

[jkotas]

R_X86_64_PC32

We had several reports of this https://github.com/dotnet/corert/search?q=R_X86_64_PC32&type=Issues . It is a bug in the objectwriter component. #538 has best hints about what the fix may be.

AFAIK it does not make things works.

How this does not work? Is the binary crashing when you try to run it?

[sebastianulm]

Yes, object writer emits only 32 bit Symbols, which can not be relocated in 64 bit memory, hence it cannot link with -fPIE. I don't consider this a bug. They are not broken, there are valid considerations to disable this for some builds. For example if you build render software or a bitcoin miner or anything very CPU intensive, you want to use 32 bit symbols.
They are a bit faster, even on 64 bit platforms.

Aside from these special cases virtually everybody wants to build full 64 bits, because its required to randomize the address space.

So, in a perfect world, libObjWriter would have a flag that switches between the two. I consider it more a missing feature the a Bug. There are genuine reasons for making this a parameter (which is why all linux compilers offer this flag)

AFAIK it does not make things works.

Sorry, this is a typo. I meant to write, "AFAIK it does not make things worse".
Currently, if I check out clang-3.9, build it and use it as a linker, it will not produce PIC-Binaries by default.
The Microsoft.NETCore.Native.Unix.props file does not state -fPIC nor -Fno-PIC. It simply uses what ever has been set as the default when building clang.
Most modern distribution ship binaries of clang with –enable-default-pie, which changes the default from -Fno-PIC to -fPIC. Ubuntu started doing this in with 16.10 https://wiki.ubuntu.com/SecurityTeam/PIE

I think the build machine either has an older version, or does not use the binary from the distribution. Hence all the tests are passing on the build machine, but fail out in the wild.

I may be wrong here, so this is something you should check. Grabbing an artifact from the build machine and checking if it was build with PIC enabled should be enough to find out.

The people from the issues you linked all have the same issue as me, and I know not nearly enough to add that missing feature to objWriter.

[jkotas]

Could you please exclude this change from this PR?

Hopefully, #6847 will land soon and we will have proper fix for this issue.

[sebastianulm]

Done!

I'll try to help with testing on those new changes asap.
Unfortunately I know very little about clang and assembler. I know a bit about the elf file format, and linux tools, so testing things is probably something I can do.

[jkotas]

Thanks!