[META] Use the Conda package management system or AppImage for rootless portable installations

[ehfd]

https://docs.conda.io/en/latest/

conda-forge feedstocks maintained for the purpose of deploying Selkies:

Please help maintain by adding yourself as a maintainer or create non-existent feedstocks.

Future feedstocks required for conda install -c conda-forge selkies-gstreamer:
conda-forge/gstreamer-feedstock#131 (GStreamer WebRTC plugins)
conda-forge/staged-recipes#26241 (X.Org Libraries and Utilities including xrandr and Xvfb)
conda-forge/xorg-makedepend-feedstock#9 (comment) (Revamp existing X.Org Libraries)
conda-forge/conda-forge.github.io#1891 (MSYS2 stack for X.Org Libraries)
conda-forge/staged-recipes#25800 (PipeWire)
conda-forge/mesalib-feedstock#78 (comment) (mesalib OpenGL provider) - Alternative is libglvnd-feedstock, now available below.

Dependencies managed by Selkies team:

X.Org X11 library/utility feedstocks including:
https://github.com/conda-forge/xorg-libxmu-feedstock
https://github.com/conda-forge/xorg-libxtst-feedstock
https://github.com/conda-forge/xorg-xauth-feedstock
https://github.com/conda-forge/xorg-libxcvt-feedstock
https://github.com/conda-forge/xorg-libxv-feedstock
https://github.com/conda-forge/xorg-libxshmfence-feedstock
https://github.com/conda-forge/xorg-libxxf86dga-feedstock
https://github.com/conda-forge/xorg-libxxf86vm-feedstock

Non-X.Org feedstocks:
https://github.com/conda-forge/pynput-feedstock
https://github.com/conda-forge/evdev-feedstock
https://github.com/conda-forge/xsel-feedstock
https://github.com/conda-forge/xclip-feedstock
https://github.com/conda-forge/wmctrl-cli-feedstock
https://github.com/conda-forge/xdotool-feedstock
https://github.com/conda-forge/libva-feedstock
https://github.com/conda-forge/libva-utils-feedstock

OpenGL feedstocks:
https://github.com/conda-forge/libglvnd-feedstock

Adjacent, but not depending feedstocks also managed by Selkies team:

https://github.com/conda-forge/wmctrl-feedstock
https://github.com/conda-forge/pyperclip-feedstock

[ehfd]

This is possible. Conda enables installing packages without regard to ABI or OS compatibility, and GStreamer/GST-Python are usually updated to most new stable and development versions. The plugins are also in the repository. Even if the plugins are old, we can request Anaconda to update them to the recent versions.

The CUDA Toolkit is also always available from NVIDIA. Most dependencies including the X11-related ones will also be ready in an OS-agnostic way.

If we create a conda package for Selkies-GStreamer, this will lead to effortless installations of both gst-web and the Python components absolutely without root whatsoever, because it creates its own dependency environment. This is also the most academia way to install a package (lots of support from ML and other related projects), and it will help using Selkies in HPC clusters that use batch schedulers such as SLURM.

We have a choice; adding this package to the conda-forge channel and letting them compile the project, or building the package with GitHub Actions and pushing it to our own channel. In the latter case, we may provide custom GStreamer builds as well.

This can also enable distro-agnostic ready-to-go distribution of Selkies-GStreamer with GStreamer, gst-web, Python files, and CUDA ready to run anywhere, if we just build and package everything with Conda's own Python interpreter and build environment.

Contribution from interested parties is welcome; this is also a good first issue if you are accustomed to conda.

[ehfd]

Edit: This is solved for now because GStreamer 1.24 pulls and compiles these missing dependencies automatically. However, for Selkies to become a feedstock for conda install -c conda-forge selkies-gstreamer, these all need to become new feedstocks. More information above.

On Conda:

From @elgalu:

Cool! so I installed a few gstreamer gst-plugins-base gst-plugins-good gst-plugins-bad gst-plugins-ugly gst-python gst-libav webrtcvad and now the list of missing plugins is reduced to 3: 'Missing gstreamer plugins:', ['nice', 'webrtc', 'srtp']

Me:

sudo apt-get update && sudo apt-get install --no-install-recommends -y adwaita-icon-theme-full build-essential python3-pip python3-dev python3-gi python3-setuptools python3-wheel tzdata sudo udev xclip x11-utils xdotool wmctrl jq gdebi-core x11-xserver-utils xserver-xorg-core libopus0 libgdk-pixbuf2.0-0 libsrtp2-1 libxdamage1 libxml2-dev libwebrtc-audio-processing1 libcairo-gobject2 pulseaudio libpulse0 libpangocairo-1.0-0 libgirepository1.0-dev libjpeg-dev libvpx-dev zlib1g-dev x264

This was the long list of dependencies, where I believe you might want to find or build 'nice', 'webrtc', 'srtp' manually on Conda. An alternative for apt's libwebrtc-audio-processing1, libsrtp2-1, and gir1.2-nice-0.1 or libnice10 is needed. I see that there is a separate package gstreamer1.0-nice in apt.

I think: pylibsrtp provides srtp.

https://webrtc.googlesource.com/src
https://gitlab.freedesktop.org/libnice/libnice

https://developer.ridgerun.com/wiki/index.php/GStreamer_WebRTC_Wrapper/Getting_the_Code/Building_and_Installation_Guide
https://gitlab.freedesktop.org/libnice/libnice/-/issues/112
https://webrtcmagazine.wordpress.com/tag/libnice/

This is pretty much uncharted territory, and I would really appreciate if you could test things. You could ask on #gstreamer at oftc.org IRC or #_oftc_#gstreamer:matrix.org on Matrix for help from GStreamer devs.

[ehfd]

Would be really nice if we knew all the set of dependencies which could be installed with just conda and pip.
In theory, anything can be built using Conda's own toolchain.

https://anaconda.org/anaconda/gcc_linux-64
https://anaconda.org/anaconda/gxx_linux-64

[ehfd]

This seems easier because the CUDA dependency was reduced to NVRTC only.

[ehfd]

If anyone does this, make sure to start from Miniconda and enable only conda-forge. anaconda channel is proprietary.

[ehfd]

https://github.com/linuxdeploy/linuxdeploy-plugin-conda
https://github.com/linuxdeploy/linuxdeploy-plugin-gstreamer

Conda packages can be deployed as AppImages. Best of both worlds.

https://github.com/LizardByte/Sunshine/tree/nightly/packaging/linux/AppImage

https://conda.github.io/conda-pack/
https://github.com/conda/constructor
https://chtc.cs.wisc.edu/uw-research-computing/conda-installation

[ehfd]

The initial implementation of the Conda build toolchain to generate a portable distro-agnostic distribution is ready. It will support any distro with glibc >= 2.17 (anything as old as CentOS/RHEL 7, Debian 8, Ubuntu 14.04).

conda install -c conda-forge selkies-gstreamer still has some way to go.

[ehfd]

libglvnd is implemented, making the Conda build workflow much cleaner.