Can't run Python SDK on Linux

[maximevince]

When trying to build the PluginExample on Arch Linux, with Python 3.8 installed:

$ tap python build PluginExample

Unable to load '/home/vinz/.tap/pythonplugin/libpython3.8.so' assembly might be in an invalid format.
Unable to load Python.

Where I configured the TAP python path as follows:

tap python set-path /home/vinz/.tap/pythonplugin

And symlink the python library as follows:

file /home/vinz/.tap/pythonplugin/libpython3.8.so
/home/vinz/.tap/pythonplugin/libpython3.8.so: symbolic link to /usr/lib/libpython3.8.so

Any idea what assembly might be in an invalid format is referring to? x86_64 vs i686 ? Something else?
How to proceed?

[maximevince]

After some tracing using LD_DEBUG=all, it seems it was looking for liblibdl.so instead of libdl.so.
So by symlinking libdl.so to liblibdl.so, I could compile the PluginExample that way.

I suppose this is a library naming issue in the Linux build somewhere. Can you guys look into it?

Thanks

[rmadsen-ks]

Hi @maximevince, thanks for submitting an issue.

I think it actually belongs at http://gitlab.com/opentap/python, eventually it will be moved to github.com/opentap/python, probably, but anyway, maybe we can just discuss it here - OpenTAP also needs to load libdl.

it sounds a bit strange that we were not able to load libdl. Maybe libdl.so is located a bit differently on Arch than on Ubuntu?

I guess the problem is that we try to do this to load dlopen and dlsym.

// ...
            [DllImport("libdl.so")]
            static extern IntPtr dlopen(string fileName, int flags);

            [DllImport("libdl.so")]
            static extern IntPtr dlsym(IntPtr handle, string symbol);
// ...

In the future we are also going to test "libdl.so.2", maybe that helps.

Can you locate the file libdl.so and libdl.so.2, if they exist on your harddrive, and then tell me the location of those?

[maximevince]

Hi @rmadsen-ks, I noticed the Python repo is on GitLab indeed, apologies.

libdl.so on Arch is indeed not present in /usr/lib, but libdl.so.2 is:

ls -al /usr/lib/libdl\.*  
-rw-r--r-- 1 root root     8  6 jun 11:45 /usr/lib/libdl.a
-rwxr-xr-x 1 root root 14360  6 jun 11:45 /usr/lib/libdl.so.2

So that's probably the issue indeed, which explains why when I created a symlink to libdl.so.2 and used LD_PRELOAD_PATH it started working.

The liblibdl.so I mentioned earlier is probably just an artifact of the ld loader on Linux looking for variations of the same library filename. (trying to append lib to the library name already containing lib)

I guess moving to libdl.so.2 will solve the issue on Arch Linux.

[maximevince]

Just FYI:

I could find a .NET/Mono project with a very similar issue:
ied206/Joveler.DynLoader#1

They have solved it like by splitting the loader code for Linux and using libdl.so.2, like you mentioned: ied206/Joveler.DynLoader@b26e4ca#diff-31039d3b57ff4757d68b549f42fa62223b1477b6b61faec9b2f70cd575b8154fR78

[rmadsen-ks]

It seems pythonnet does the same thing now, so for us this will be fixed in the next version of the Python plugin

https://github.com/pythonnet/pythonnet/blob/master/src/runtime/Native/LibDL.cs

[maximevince]

Awesome, thanks for the update!

[rmadsen-ks]

No problem. Let's call this resolved then.