-
-
Notifications
You must be signed in to change notification settings - Fork 563
Desktop Linux Platform Issues
To make "Desktop Linux" a viable platform (as in: Windows, macOS) to develop against, we either need to find the "least common denominator" by experimentation (as we have done for over a decade now), or get the main Linux distributions (Ubuntu, Fedora, openSUSE, Debian, etc.) to agree on a certain set of infrastructure that can be expected to be "there", in a consistant way, without unnecessary differences.
Your thoughts are highly welcome. Please join the discussion here. However, please do not make substantial edits to this page before reaching consensus in the discussion on https://github.com/AppImage/AppImageKit/issues/445.
Credit: MuseScore (CC BY 2.0)
openSUSE Chairman Richard Brown pointed out the need for this in a talk at openSUSE Conference 2017:
This is not to say that the "least common denominator" approach does not work. In fact, the arguably two most successful "Desktop Linux" applications, LibreOffice and Firefox, are built on old build systems and bundle many libraries in order to be able to deliver one set of binaries to (almost) all "Desktop Linux" systems, independent of distributions, and have been doing this for quite some time now. But given some sane platform decisions, life could be made much easier for application developers who want to target the "Desktop Linux" platform.
This page is here to collect ideas on what would be worthwhile to standardize on. It is meant as a basis to get the discussion started rather than finalized recommendations. You are welcome to edit this page to add factual information, but if you fundamentally disagree with the direction of this page please talk to us first before editing this page.
In the over a decade experience with application bundles the kernel has proven never to be an issue, since the interface between the kernel and userland is stable, and Linus Torvalds is heavily defending a stable userland interface. So no action required as far as AppImage is concerned. Applications should ensure they can run on a vanilla kernel without extra patches.
The names of libraries should reflect the version as given by upstream. Currently it is not, resulting in the necessity for each application using it to bundle a private version, since the application cannot rely on the system-provided one.
Example:
- Ubuntu 12.04 has
libarchive.so.12
desipte it being version 3.x - Ubuntu 14.04 has
libarchive.so.13
desipte it being version 3.x
This should be changed so that every distribution that carries any version of libarchive version 3.x would contain libarchive.so.3.xxx
and a symlink to libarchive.so.3
, so that applications could rely on being able to link against libarchive.so.3
.
The libarchive upstream developers say they can do nothing about this since the library filename is decided by distributions.
We might be able to make the number more consistent across platforms, but it would require someone to do the research into how cmake and autoconf generate build numbers on the many different platforms.
References: https://github.com/libarchive/libarchive/issues/817
Despite there being the FHS, filesystem paths are different on different reason for no apparent reason. They should be standardized between all distributions.
Example:
- Debian, Ubuntu:
/lib/i386-linux-gnu/
,/lib/x86_64-linux-gnu/
- openSUSE:
/lib/
,/lib64/
References:
Should be installed to a standard location that is the same in all distributions. Currently this is not the case and where certificates go is apparently entirely random by distribution, as of mid-2017:
"/etc/ssl/certs/ca-certificates.crt", // Debian/Ubuntu/Gentoo etc.
"/etc/pki/tls/certs/ca-bundle.crt", // Fedora/RHEL
"/etc/ssl/ca-bundle.pem", // OpenSUSE
"/etc/pki/tls/cacert.pem", // OpenELEC
"/etc/ssl/certs", // SLES10/SLES11, https://golang.org/issue/12139
"/system/etc/security/cacerts", // Android
Source: https://serverfault.com/questions/62496/ssl-certificate-location-on-unix-linux
The result is that if we bundle something like libgnutls
, then it fails to find the certificates if the target system happens to store them in a different place.
Workaround:
Patch whatever consumes certificates (e.g., libgnutls
) to look in all known places. Example: https://github.com/darealshinji/vlc-AppImage/issues/1#issuecomment-321041496
Applications should be able to rely on certain basic infrastructure libraries to be provided by all distributions in the default installation. They should be standardized between all distributions.
Ideally this library would be frozen indefinitely. Why is it necessary to still keep adding symbols to it, making software compiled against later versions fail on earlier versions? If freezing this is not realistic, then at least build systems should use older versions than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
- ld-linux.so.2
- ld-linux-x86-64.so.2
- libanl.so.1
- libBrokenLocale.so.1
- libcidn.so.1
- libcrypt.so.1
- libc.so.6
- libdl.so.2
- libm.so.6
- libmvec.so.1
- libnsl.so.1
- libnss_compat.so.2
- libnss_db.so.2
- libnss_dns.so.2
- libnss_files.so.2
- libnss_hesiod.so.2
- libnss_nisplus.so.2
- libnss_nis.so.2
- libpthread.so.0
- libresolv.so.2
- librt.so.1
- libthread_db.so.1
- libutil.so.1
These files are all part of the GNU C Library which should never be bundled. List was generated from a fresh build of glibc 2.25.
References: https://www.gnu.org/software/libc/
Ideally this library would be frozen indefinitely. Why is it necessary to still keep adding symbols to it, making software compiled against later versions fail on earlier versions? If freezing this is not realistic, then at least build systems should use older versions than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of it should be guaranteed to still run with later versions of this.
- libstdc++.so.6
References: https://gcc.gnu.org/wiki/Libstdc++
This should be abstracted in a way so that software compiled against OpenSSL would work against whatever version of OpenSSL comes with the system. Currently software compiled against a certain "letter" version (e.g., "f", "l") of OpenSSL will fail when another version of OpenSSL is available on the system. Applications written against earlier versions of it should be guaranteed to still run with later versions of this.
Currently applications cannot rely on these:
- libssl.so.1
- libssl.so.1.0.0
- libcrypto.so.1
- libcrypto.so.1.0.0
They should be standardized across all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.
References:
- https://github.com/probonopd/AppImages/commit/6c7473d8cdaaa2572248dcc53d7f617a577ade6b
- http://stackoverflow.com/questions/32644157/forcing-a-binary-to-use-a-specific-newer-version-of-a-shared-library-so
An application compiled against libgnutls26
should continue to work on systems that are using subsequent versions, such as libgnutls28
and libgnutls30
. Possibly the versioning scheme of libgnutls would have to be changed for this, with new major versions (with ABI-breaking changes) coming out as rarely as possible (every 5-7 years or less frequently), and distributions should carry not only the recent version but also the version before to give sufficient time to developers to update.
Currently it looks like libgnutls26
, libgnutls28
and libgnutls30
are all in use and not compatible.
The following libraries currently cannot be safely assumed to be part of every distribution:
- libkrb5.so.26
- libkrb5.so.3
- libkrb5support.so.0
Distributions should agree on libkrb.so.5
being the name for Kerberos 5. Then applications could rely on these from the system rather than having to privately bundle their own copies.
Workaround:
Patch libcurl
so that it does not need Kerberos. https://github.com/AppImage/AppImages/blob/c9e6b6b365fcd97e2980c0844c478904c030a25a/recipes/meta/Recipe#L194
X11 libraries, Xlib, and the X protocol C-language Binding (XCB) should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
- libX11.so.6 (uncertain if really all distributions bundle it)
- libxcb.so.1 (uncertain if really all distributions bundle it)
- libxcb-dri3.so.0 (uncertain if really all distributions bundle it)
- libSM.so.6
- libXp.so.6 (currently not available on Fedora default installations)
- Many other libX...
OpenGL and the and Direct Rendering Manager (DRM) should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
- libGL.so.1
- libdrm.so.2
References:
Sound should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
- libasound.so.2
- PulseAudio libraries (tbd)
Gtk+ and Gdk should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
Ideally, there would be a defined library to develop against, e.g., libgtk.so.2
for Gtk+2 and libgtk.so.3
and applications would be compiled against those instead of a multitude of individual libraries that make up "Gtk+".
Example:
- libgdk_pixbuf-2.0.so.0
- Many more (are they defined somewhere)
GTK+ should follow semver.
According to https://blog.gtk.org/2016/09/01/versioning-and-long-term-stability-promise-in-gtk/, "Updates within long-term stable series will be ABI stable". This is good, but in order to minimize developer confusion about which GTK+ version can be relied on, only such long-term stable series should get major version numbers.
nkiesel nails it in https://lwn.net/Articles/691131/
However, there is the psychological problem of such applications always using the "previous" release. So innocent bystanders will see "GTK 5.0 released with all those new cool features", followed by "app FOO upgraded from GTK 3 to GTK 4". Therefore, GTK should make it very clear in it's press release that GTK 5.0 is just the first release candidate of GTK 5 and not intended to be used by external applications.
In this example, I would propose "GTK 5" to be called that only after it has a stable API and should be used by external applications. This would be more in line with the idea of a sane, dependable platform to build on. Everything that is still a moving target and has no API guarantees should not be worthy of a new major version number.
GLib should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older version than what is available in the oldest still-supported available distributions, and the library should be only very infrequently and only if there are severe reasons that really require an update. Applications written against earlier versions of this should be guaranteed to still run with later versions of this.
- libglib-2.0.so.0
Fonts and font rendering libraries should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
- libfontconfig.so.1
These would ideally never have to be linked against directly:
- libpangoft2-1.0.so.0
- libpangocairo-1.0.so.0
- libpango-1.0.so.0
- libpangomm-1.4.so.1
With increasing frequency, we are running into symbol lookup error: /usr/lib64/libpangoft2-1.0.so.0: undefined symbol: hb_buffer_set_cluster_level
. Why?
Bundling HarfBuzz seems to cause symbol lookup error: /usr/lib64/libpangoft2-1.0.so.0: undefined symbol: hb_buffer_set_cluster_level
.
HarfBuzz should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of these should be guaranteed to still run with later versions of these.
References: https://www.freedesktop.org/wiki/Software/HarfBuzz/
Currently every distribution names these differently:
- libtiff (Fedora and openSUSE lack
libtiff.so.4
) - libpng
- libjpeg
They should be named the same in all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.
Currently distributions ship no or incompatible versions of:
- libnss3.so
- libnssutil3.so
- libsmime3.so
- libssl3.so
- libfreebl3.so
- libfreeblpriv3.so
- libnssckbi.so
- libnssdbm3.so
- libnsssysinit.so
- libsoftokn3.so
They should be standardized across all distributions. Then applications could rely on these from the system rather than having to privately bundle their own copies.
References:
- https://github.com/probonopd/linuxdeployqt/issues/35#issuecomment-256213517
- https://github.com/probonopd/AppImages/pull/114
libpcre should be handled by the distribution and they should provide a clearly defined interface for applications to develop against. Build systems should use older versions than what is available in the oldest still-supported available distributions. Applications written against earlier versions of this should be guaranteed to still run with later versions of this
- openSUSE (as of 2017) lacks libpcre.so.3 but provides libpcre.so.1
- Ubuntu (as of 2017) lacks libpcre.so.1 but provides libpcre.so.3
Certain distribution functionality/libraries should be purely optional. Software compiled on distributions should never require these as hard dependencies, but should rather check whether these are available, and fall back gracefully if they are not there.
-
libselinux.so.1
. Currently some software compiled on CentOS requireslibselinux.so.1
even if SELinux is not used or wanted
Workaround: dummy libraries that resolves all of their symbols but just do nothinge.g., https://github.com/AppImage/AppImages/issues/83#issuecomment-321966519
Desktop environments should handle ELF files with missing executable bits more gracefully. https://github.com/AppImage/AppImageKit/issues/374
In the realm of security, distribution vendors are going different paths (Red Hat using NSA-originated SELinux and others using AppArmor or other technologies. Which is fine, as long as these systems don't spill over into each and every application, as is the case with libselinux.so.1
, which gets linked into each application compiled on Red Hat based systems like CentOS, while it is not available on some other systems.
Security enhancements in the operating system should not require applications to link to libraries like libselinux.so.1
.
Workaround: Bundle libselinux.so.1
with every application that needs it (but since it depends on libpcre.so.3
we need also bundle that because distributions can't be relied on to ship the correct version under the same name). Or, better, replace it by a dummy library that resolves all of its symbols but just does nothing, e.g., https://github.com/AppImage/AppImages/issues/83#issuecomment-321966519
Where should we discuss this? Please add ideas here.