Handle new DT_RUNPATH in object::load_needed()

[nyh]

Issue discovered and analyzed by @wkozaczuk when he was trying to run an unmodified Java taken as a PIE from a Linux distribution.

/scripts/run.py -e '/usr/lib/jvm/jdk-zulu11.31.11-ca-jdk11.0.3-linux_x64-java-base/bin/java --version'

works fine, but

/scripts/run.py -e '/java --version' 

fails, because it cannot find libjli.so (to see that, add -V to the command line). Why does this happen?

If we look at readelf -a java, we see

 0x0000000000000001 (NEEDED)             Shared library: [libz.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libjli.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [lib.so]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib/amd64/jli:$ORIGIN/../lib/amd64]

The thing is, the requirement libjli.so is not found in the system's main library directories. Rather the RPATH says where it is, and $ORIGIN is replaced by the executable's path. And we actually handle correctly in core/elf.cc. Or almost correctly - when the executable is a symbolic link, we need to resolve what it links to, and use that as $ORIGIN, not the link's path.

This means checking with readlink(_pathname) if this is a link and if so using the result. But it's not enough to do it once: we may have a link to a link, and so on, so we need to do this repeatedly as long as we still have a link - but use a limit, to avoid infinite iteration in case of link loops.

Interestingly, as others noted before (e.g., https://enchildfone.wordpress.com/2010/03/23/a-description-of-rpath-origin-ld_library_path-and-portable-linux-binaries/), Linux's "ldd" has this same bug. For example, on my host,

$ ldd /usr/bin/java
	linux-vdso.so.1 (0x00007ffc5fdca000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f7351f2a000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f7351f10000)
	libjli.so => not found
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f7351f0a000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f7351d44000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f7351f84000)

Note that /usr/bin/java is a symbolic link, and libjli.so is not found by ldd!
Here too, /usr/bin/java is actually a link to a link (!) to /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc29.x86_64/jre/bin/java and if I do ldd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc29.x86_64/jre/bin/java, it does work fine:

$ ldd /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc29.x86_64/jre/bin/java
	linux-vdso.so.1 (0x00007fffe0dfb000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0444c08000)
	libz.so.1 => /lib64/libz.so.1 (0x00007f0444bee000)
	libjli.so => /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.fc29.x86_64/jre/bin/../lib/amd64/jli/libjli.so (0x00007f0444bdd000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f0444bd7000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f0444a11000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f0444c62000)

[wkozaczuk]

Actually the reason for the original failure is different but related. It turns out we are properly handling symlinks by using the canonicalize() function which is uses realpath() and is called by program::load_object().

@nyh I bet if you ran unmodified java from your Fedora it would have worked just fine. Here is what I saw after adding printf() to _pathname in load_needed():

--> _pathname: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-5.fc28.x86_64/jre/bin/java
--> _pathname: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-5.fc28.x86_64/jre/lib/amd64/jli/libjli.so
--> _pathname: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-5.fc28.x86_64/jre/lib/amd64/libverify.so
--> _pathname: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-5.fc28.x86_64/jre/lib/amd64/libjava.so
--> _pathname: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181.b15-5.fc28.x86_64/jre/lib/amd64/libzip.so

when calling with command line referencing /usr/bin/java that was a symlink.

Now the real issue shows up on Ubuntu when we see DT_RUNPATH in java:

readelf -a /usr/bin/java | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../lib/amd64/jli:$ORIGIN/../lib/amd64]

And we actually do not handle it at all. Per this - "Finding shared libraries at run time" DT_RUNPATH is a newer version of DT_RPATH. The only difference is that it should be resolved after LD_LIBRARY_PATH which I am not sure it matters to us.

[nyh]

@wkozaczuk interesting. I think you're right. I missed that program::load_object() indeed "canonicalizes" the pathname before continuing to do the rest of the work including processing DT_NEEDED at all! It maybe does more canonicalization than we actually need here, but it should have indeed worked.