Provide a Flatpak bundle

[yosbelms]

I know an AppImage is available but regarding the increasing Flatpak popularity to the point that LinuxMint now supports it out of the box, I think it should be taken in consideration

[zonkmachine]

Can you please add a description to this issue?

[tresf]

Have you thought of to provide a Flatpak bundle?

Nope, but we do offer an AppImage.

[tresf]

For those wondering, they're really competing technologies. More here:

https://www.reddit.com/r/linux/comments/4kxbqp/appimage_snaps_flatpak_pros_and_cons_comparison/

[yosbelms]

@tresf What about to have both? Is it hard to maintain both?
I know an AppImage is available but regarding the increasing Flatpak popularity to the point that LinuxMint now supports it out of the box, I think it should be taken in consideration

[tresf]

@yosbelms we'd welcome a PR with this functionality but if that doesn't happen we won't be dedicating any time to this in the near future. In regards to the effort someone would be taking on, please see #3688.

In regards to popularity, that will change with time and we'd be happy to reopen this when most distros have chosen a stance on the underlying technology. I've been able to extract and run the AppImage on distros as obscure as FatDog, so we have a very wide range of compatibility.

On a side note, since Toby left, I've been the primary person packaging this product, so until someone volunteers to take that role, I get to make this decision. :)

[tresf]

Tagging @probonopd incase he has some thoughts on the topic.

[alexgleason]

Just wanted to chime in - I'm not attached to any specific tech to accomplish this. In fact, I'd prefer to use my distro's repos if only it was kept updated. But flatpak at least offers a flatpak update command which updates all flatpak applications on the machine. AppImage requires downloading a new AppImage for each release, and puts the burden on the user (or the software itself) to check for updates.

[tresf]

AppImage has some items in the queue for this as well...

https://github.com/AppImage/AppImageUpdate/wiki/Self-updating-AppImages#using-the-existing-appimageupdate-user-interfaces-to-perform-updates

Whatever update mechanism we use should be cross-platform and pull from our own hosting, so it's more likely that we'll just offer this as an in-app feature without depending on 3rd party repositories to keep track of our stuff.

[probonopd]

AppImage requires downloading a new AppImage for each release, and puts the burden on the user (or the software itself) to check for updates.

No. Check out AppImageUpdate.

Application developers can even use libappimageupdate to embed this functionality right into the application, making it self-updateable.

https://docs.appimage.org/packaging-guide/updates.html

[hfiguiere]

Flatpak for flathub in review at flathub/flathub#829

[musikBear]

what is considered safest?
Also observe that many diddent even know the term 'flatpack', (me neither) and that is in a tech-forum..

[tresf]

what is considered safest?
Also observe that many diddent even know the term 'flatpack', (me neither) and that is in a tech-forum..

To the end-user, it's just another way to install. The term is flatpak (pak, not pack). Please don't speculate on safety out of fear alone. Windows installs everything as admin by default. If you want a safer format, DMG, AppImage and flatpak are all "safer" than our current installer for Windows. 😈

Something I'm not yet sure about is where the flatpak gets created. I assume @hfiguiere is familiar with this and that organizations like flathub can offer assistance. The XML appears to have instructions for building dependencies from source, which is a bit more holistic than our current process of using precompiled binaries as it gives us the option to enable/disable certain features at build/packaging time (avoiding problems like #3446).

[hfiguiere]

In that case it would be on Flathub, that act as builder and repository. From a manifest file, the dependencies are pull and everything is build from source.

Flathub is kinda the de-facto Flatpak repository, and this is where I aim to make this available.
See https://flathub.org/

[tresf]

Thanks to @hfiguiere and per flathub/flathub#829 (comment) we have a flatpak to test.

The command is:

flatpak install --user http://repo-test.flathub.org:8080/build-repo/1719/io.lmms.LMMS.flatpakref

Testers welcome.

[hfiguiere]

Let me know if you have any issue. I have another change to push that add carla and pin the release to 1.2.0-rc7.

[Anyeos]

Flatpak version does not include the ZynAddSub instrument.
Version 1.2.0-rc8

[PhysSong]

@Anyeos I'm working on the issue, see flathub/io.lmms.LMMS#1. Could you test the fix?

flatpak install --user https://dl.flathub.org/build-repo/2949/io.lmms.LMMS.flatpakref

[PhysSong]

I also found that:

• JACK and SDL are disabled
• Mallets and GIG player are disabled
• VSTs are disabled

I'm not sure how many issues we can address, but I think we should try to improve the usability of the bundle.

[hfiguiere]

JACK is a problematic with Flatpak right now since it can't connect to the system jackd. So it is disabled.
Not sure about the rest. I think VST needs to add Gtkmm. I'll check that next week at the earliest.

[hfiguiere]

How is SDL useful?

[PhysSong]

@hfiguiere #1600 is the relevant conversation, but I wasn't here at the moment.

[hfiguiere]

I see. I'll update the flatpak soon with SDL.

[hfiguiere]

I have pushed changes that do the following:

• enable SDL
• enable Carla UI (it was missing PyQt5)
• STK Mallets
• GIG player

[tresf]

Hmm... If this is true, how does LMMS load other plugins? We aren't bundling Wine into the flatpak too, are we?

[hfiguiere]

No yet. ^-^. You can still install plugins in the home directory though.

[tresf]

Well, Carla is just a plugin, so I'm still a bit confused as to why LMMS is depending on PyQt5 when it's not required for the code to compile or run. We don't have to do this for any other systems. We expect people to obtain Carla on their own and LMMS tries to open it. If we need to adjust the open logic for a flatpak, that would seem more reasonable than adding PyQt5 as a flatpak dependency. At some point, a flatpak has to talk to 3rd party programs. If they're 100% sandboxed, it seems like there will be other scalability issues.

On Mac, you can install LMMS.app in $HOME and then install Carla.app in $HOME too. We just assume when running from the .app file, Carla is side-by-side. The dependencies for Carla are the responsibility of the Carla plugin, not LMMS. Not sure how flatpak handles this scenario but I'd expect Wine (Vestige instrument) to have the same issue.

[hfiguiere]

Mac works differently, that's for sure.

I'll see if there is a better solution. PyQt5 is only in there for Carla UI to work.

[hfiguiere]

And yes for VST we'd have to build Wine as there is no flatpak runtime for it.

[tresf]

And yes for VST we'd have to build Wine as there is no flatpak runtime for it.

This is that part I do not understand. Is the goal of the Flatpak universe to never talk to any system program ever? Is it 100% sandboxed? If so, shouldn't you consider killing Carla support and killing Wine support? At which point do you stop the sandbox from being a mess?

PyQt isn't needed for LMMS because it calls Carla from a launch script. It's not LMMS' responsibility to package a dependency that another Flatpak uses. Same goes for Wine, don't you think? Why can't a Flatpack just open a remote process? What is the technical blocker for this? Is it just against philosophy?

[hfiguiere]

AFAIK there is no way to start an external program, ie one not included in the sandbox (either directly of via a runtime). This is unfortunate.

As I said I added PyQt for Carla to work - I know LMMS doesn't need it directly. Might not be the best solution though.

Apparently someone filed flathub/io.lmms.LMMS#2 to get VST support.

[Anyeos]

Then I think are this kind of things like flatpak here for make the things worst? We have dependencies in a DEB file because we need to save space and time not to reinstall all the libraries again. If we have an improvement in some library but the main program, it can benefit from that only installing the library and not all the entire program.

What is the sense of making such things like flatpak if there are no way of doing something like a DEB can do?

As a minimum it must be possible to launch other flatpak or external programs like Android can do in some way. Next it must be a need to load things like as plugins.

Android is an example of something like sandboxing apps. But in Android you can eventually launch an app from another one. Embed apps and libraries can be done too.

There are no sense of isolate everything specially if you are in the same computer. That destroy the sense of networking too. Then we must live in an island isolated of all the rest of the applications so nothing can have relation with nothing. What is that?

I know it is not an issue with LMMS but the question raised here.
I think we need to file a bug on flatpak about this topic.

[tresf]

@Anyeos I don't know enough about the flatpak specification to counter your points, but I wholeheartedly agree with the philosophy -- that a flatpak's purpose is to make installation easier. At some point, there are libraries that don't make sense to bundle.

I'm still not convinced that the statements are true though. If they are, how would you point LMMS at -- say -- a custom LADSPA directory?

[hfiguiere]

If the LADSPA directory is in $HOME it should be okay. We could also add standard locations. I'll look into it.

Meanwhile 1.2.0 should be building right now. With ZynAddSubFX support.

[tresf]

@hfiguiere the way we do Carla on Mac, it supports $HOME by looking in the adjacent directory (which also works if LMMS and Cara are both in the standard /Applications directory.

Can flatpaks not look at non-$HOME directories/plugins?

[hfiguiere]

I am currently working on a solution to install (and package) plugins in Flatpak. It's not specific to LMMS but will definitely work with it. I managed to load some LV2 plugins through Carla. I still have to build Carla in LMMS to make this work because it looks for Carla as a dependency.

[BrunoVernay]

PipeWire may not be so far to be usable? https://pipewire.org/

[hfiguiere]

PipeWire may not be so far to be usable? https://pipewire.org/

In time, but right now this is not a blocker. LMMS do work as intended, with just a little glitch about the Carla icons I haven't been able to figure out. See flathub/io.lmms.LMMS#4

[hfiguiere]

@hfiguiere the way we do Carla on Mac, it supports $HOME by looking in the adjacent directory (which also works if LMMS and Cara are both in the standard /Applications directory.

macOS is macOS. Flatpak is different and is meant to sandbox applications so simply put no. Carla has to be built in the flatpak. Even a stand alone Carla wouldn't work. (do you support macOS VST? AudioUnits? just to have a reference)

Can flatpaks not look at non-$HOME directories/plugins?

No. And loading binaries from $HOME is doomed to fail. (if it works you are lucky)

However, now there is general support for loading audio plugin installed as flatpak: this is not specific to a single app, and for LMMS this only work from within Carla rack, be it LADSPA, DSSI, LV2 or VST/VST3 (Linux native). There won't be support for freeloading Linux audio plugins otherwise.

In theory it would be possible to support Windows VST with VeSTige by treating the Windows plugins as data, but this is not currently done as this needs to build Wine as well part of the flatpak

LMMS doesn't seem to honour LADSPA_PATH, but then it ship so many plugins that it's almost moot (swh, tap and CMT are the only one I have as flatpak locally).

[tresf]

LMMS doesn't seem to honour LADSPA_PATH, but then it ship so many plugins that it's almost moot (swh, tap and CMT are the only one I have as flatpak locally).

Right, and we had to fork and refactor Calf since LV2 reused the calf.so name and segfaults. This change is only on master currently.

Flatpak is different and is meant to sandbox applications so simply put no. Carla has to be built in the flatpak.

This is flatpak's philosophy and it's their prerogative, but it's a serious design mistake to assume all plugins in which are supported are bundled into our app. It even sends the wrong impressions to our users as we don't (nor are we interested in) bundling more plugins. This issue would go away for many plugins once we get LV2 support finished, but the sandboxed behavior ends up making our app a fraction of the size of the plugins in which it can leverage (e.g. Wine, Carla).

macOS is macOS. Flatpak is different and is meant to sandbox applications so simply put no.

Well, the same strategy works fine with AppImage. MacOS actually is fantastic at sandboxing but I'd rather not go off-topic. The point of that statement was that the process of installing Carla separately is something that we currently support, it's tested and it works. If you find a way to get Carla and LMMS to share the same Qt instance it should help reduce the overall size, but when Carla can use wine, LMMS can use wine and LMMS can use Carla, it seems like you're shipping quite a bit in the name of sandboxing, something that seems flawed by design -- albeit not your own design. 🙁

[tresf]

LMMS doesn't seem to honour LADSPA_PATH

I double-checked this statement, and it appears LMMS does honor this here:

lmms/src/core/LadspaManager.cpp

Line 45 in ce5c744

QStringList ladspaDirectories = QString( getenv( "LADSPA_PATH" ) ).

[hfiguiere]

LMMS doesn't seem to honour LADSPA_PATH, but then it ship so many plugins that it's almost moot (swh, tap and CMT are the only one I have as flatpak locally).

Right, and we had to fork and refactor Calf since LV2 reused the calf.so name and segfaults. This change is only on master currently.

Flatpak is different and is meant to sandbox applications so simply put no. Carla has to be built in the flatpak.

This is flatpak's philosophy and it's their prerogative, but it's a serious design mistake to assume all plugins in which are supported are bundled into our app.

No you misunderstand. I am saying that right now there is support to load plugins, but they also have to be build as flatpak. I do have locally (the submission process to flathub is in progress) a bunch of plugins in various format, and Carla see them all, from within LMMS. And from within other applications.

And it seems with LMMS I can only do that with Carla:

Here is Dexed as LV2 (DISTRHO port) in Carla rack in LMMS.

This issue would go away for many plugins once we get LV2 support finished, but the sandboxed behavior ends up making our app a fraction of the size of the plugins in which it can leverage (e.g. Wine, Carla).

If you support LV2 then it's good. It will work. Same with VST and and VST3 (Linux).

The point of that statement was that the process of installing Carla separately is something that we currently support, it's tested and it works.

How does it work on Linux?

[tresf]

How does it work on Linux?

Well, historically the package manager would grab it once added: https://kx.studio/Repositories and then we'd build support, but this has had a bit of a rocky history. I guess some components that falktx added required PyQt which meant we had a custom version called carla-git but then we upgraded our CI and the PPAs weren't available so we switched to a weak-linking technique here: 9f3d3d2

It still uses binary linking though, so I'm not sure how we'd load it without using a proper plugin system. Despite LMMS having its own proprietary plugin system it's never really been leveraged as a plugin framework, so it doesn't really search for the proprietary stuff in any special locations. This has been largely due to the fact that the proprietary plugin system wasn't ever made portable (some attempts have been made and I'd be happy to dig those up).

So currently, our build system builds stub plugins but just silently fails and removes them from the list if dependencies that they use fail to load. This means LMMS on Linux and Mac ship expecting Carla libs in predictable locations with predictable names, but silently fails if they're missing.

[tresf]

Carla see them all, from within LMMS. And from within other applications.

And it seems with LMMS I can only do that with Carla:

LMMS has very limited plugin support (at time of writing this, just LADSPA and Windows VST2 through wine), so there's a good chance LMMS won't be able to load plugins with UIs (e.g. LinuxVST won't work, LV2 won't work)

• LinuxVST support is being tracked here: LinuxVST support #416 (probably better documented in the MacVST issue here: Add native VST support for Mac OS X (WIP) #3975)
• LV2 is mostly being tracked here: Lv2 core implementation #4899

For many users, Carla is a stop-gap to provide plugin support until it's offered natively.

Some plugins (e.g. Calf) have migrated to LV2, so we still bundle a custom version of them until LV2 support is completed. Others (e.g. swh) should be nearly identical between what we ship and what's in the package managers.

[hfiguiere]

So currently, our build system builds stub plugins but just silently fails and removes them from the list if dependencies that they use fail to load. This means LMMS on Linux and Mac ship expecting Carla libs in predictable locations with predictable names, but silently fails if they're missing.

Basically what the flatpak is doing: build it and ship it. Just the standard fare.

LMMS doesn't seem to honour LADSPA_PATH

I double-checked this statement, and it appears LMMS does honor this here:

lmms/src/core/LadspaManager.cpp

Line 45 in ce5c744

QStringList ladspaDirectories = QString( getenv( "LADSPA_PATH" ) ).

I'll need to check what's happening, maybe it is just that the plugins that ship with LMMS are the same I have as plugins.

For many users, Carla is a stop-gap to provide plugin support until it's offered natively.

Which is how I got things to work. The problem now is have plugins to appear on flathub for users.

So we are good. It's mostly how everything is working at the moment.

(and there are more issues with flatpak and linux audio, but that's a different story"

[hfiguiere]

So I checked again and yes I only had LADSPA plugins that were in lmms.

I installed "vlevel" (I just did flatpak for the occasion) and it appears in the LADSPA plugin list now.

[tresf]

So we are good. It's mostly how everything is working at the moment.

I'm still curious what you're doing for Carla though. Is it working? If so, how? Do we need to change any code so that it can be found?

[hfiguiere]

It is built as a dependency like any other dependency and shipped in the flatpak. (includes building sip and PyQt5) It's like if it was installed by your distro package manager, but in a different prefix.

My only only issue is that its icon doesn't appear in the UI. See flathub/io.lmms.LMMS#4

And I haven't figured out why.

[tresf]

It's like if it was installed by your distro package manager, but in a different prefix

Ok, and then we invoke carla to find where it is? That makes sense.

My only only issue is that its icon doesn't appear in the UI. See flathub/io.lmms.LMMS#4

I had the same issues here: #4813 (comment) and when it fixed itself I didn't have an explanation as to why.

In regards to the mallets bug in the linked bug report, it probably need a code change. Currently, we look in a relative path for AppImages as they're running from /tmp. If you're not running from /tmp, I expect this logic to fail.

lmms/src/core/ConfigManager.cpp

Line 551 in 84d1091

if ( qApp->applicationDirPath().startsWith("/tmp/") )

[tresf]

Correction, the images were fixed with this commit, but I just couldn't explain why... f8add69

[hfiguiere]

In regards to the mallets bug in the linked bug report,

There was a bug with the mallet-stk installation that I fixed. The OP never confirmed it was fixed, but to me it does work.

[hfiguiere]

Correction, the images were fixed with this commit, but I just couldn't explain why... f8add69

lmms/plugins/carlabase/CMakeLists.txt uses a glob to get all png added as resource, and the plugin likely expects the resource. But it end up being the same for Rack and Patchbay.

[tresf]

it end up being the same for Rack and Patchbay.

Yeah, I just was a bit puzzled why it actually worked, since the code was largely unchanged from previous versions. Did it help our your situation?

[hfiguiere]

Definitely.