VST3 Support

[dodgepong]

VST3 Support

Feature Summary

Allow users to use VST3 plugins as audio filters.

Motivation

OBS Studio currently supports VST2 plugins. At the time when VST2 support was added, the VST3 SDK was not GPL-compatible, so we did not invest any time into including VST3 support when the feature was developed. Since then, Steinberg has released a GPL-compatible SDK for VST3, which now allows us to add support for VST3 plugins in public builds. Some VST plugins are VST3-exclusive, and often users of plugins that have both VST2 and VST3 versions released prefer to use the VST3 version.

Request For Proposal

Submit a proposal explaining how you will implement this feature such that it meets the following requirements:

• Create a new plugin called obs-vst3 that adds support for VST3 plugins

  • The plugin should be implemented on Windows, macOS, and Linux*

  • The plugin should use the Steinberg VST3 SDK directly instead of an intermediate library such as JUCE

  • The plugin should run VST3 plugins in an external process to avoid OBS crashing due to a VST plugin misbehaving

    • Ideally we can use a shared memory region to pass the audio data, and then use pipes or sockets to handle the commands that go to and from a VST.
      Yabridge and lmms have some good code showing off how this would work:
      https://github.com/robbert-vdh/yabridge
      https://github.com/LMMS/lmms/blob/master/plugins/vst_base/VstPlugin.cpp

  • The plugin should search for VST3 plugins according to standard plugin location paths found in Steinberg documentation

    • Ideally the plugin has support for shell plugins, if would add too much extra work, expand on how you would leave room for this support in the future.

  • The plugin only needs to support effects plugins, things like compressors, limiters, etc. instruments are outside the scope of this bounty

  • If it's possible to integrate with the same sidechain code path the exisitng OBS compressor plugin has then the plugin should support it. If it would require more codepaths, API calls, UX then sidechaining can be another project.

  • The plugin is not required to support midi, but it would be nice to have in the future.

  • The plugin should match the number of channels that OBS provides.

    • The plugin doesn't need channel reassignment support, VST3 should handle that.

  • The plugin doesn't need additional UX for up/down mixing, VST3 should already have dynamic channel support.

* Multiple contributors may assist with implementation on different platforms and may split the bounty according to the level of contribution they made to the project.

Please read Tips for Writing a Good Proposal for guidance on how to author your proposal.

The OBS development team will consider all submitted proposals and select one submission to be implemented by the developer who submitted it. That developer will be eligible to collect the bounty upon completion of the project according to the criteria listed below.

Bounty

The bounty for implementing this feature is $3000.

You can learn more about the OBS Project Bounty Program here.

If you wish to contribute funds toward increasing the amount on this bounty, contribute to the OBS Project Bounty Fund and select the "VST3 Support" option (or just click here).

Completion Criteria

You may collect the bounty when the following are true:

• Your proposal addresses all items listed in the "Request For Proposal" section.
• You have submitted one more more pull requests that fully implement the feature as you proposed.
• Your pull request(s) are reviewed, and you have responded to review comments within a reasonable time frame.
• Your pull request(s) are merged into the master branch of the appropriate obsproject repositories.

Additional Information

Related idea from the OBS Ideas Page:

• https://ideas.obsproject.com/posts/75/add-vst-3-x-support

[falkTX]

Would this be ok if using a host library to do the heavy lifting?
I am the author of Carla which is a plugin-host that also works as a plugin itself and exposes its own libraries for other applications to use.
So far LMMS, Zrythm and Ildaeil are using it (the last one being a project of my own to ensure Carla as library works well)
Carla already supports plugin bridging.

PS: Even though core parts of VST3 SDK are GPL-licensed, there are still legal issues with using it with other GPL software. I have been told Fedora legal team is investigating the matter.

[falkTX]

Relevant link https://lists.fedoraproject.org/archives/list/[email protected]/thread/EOI3CKZPJEQIWEN2HPVNRYK5DI5QJG6Z/

[DDRBoxman]

Honestly Carla seems like it might be a good fit for this. When I wrote up the original requirements I was mostly trying to avoid having people write proposals where they pulled in the entire JUCE library in and just ship that.

[falkTX]

Well it is kinda the same in a way, juce or carla. Though Carla can use JUCE too internally for a few extra formats, so perhaps it counts as JUCE++ :)

On a serious tone, the VST3 UI does not need to be embed on the OBS window, correct? I need to ask this because, while possible to reparent windows of different processes on Windows and Linux, such thing is not possible on macOS.

Also curious about the dynamic IO stuff. This can get very complex, so would be great to clarify this point.
Are plugins expected to keep up with changes to IO from the OBS side? (Or say, once you add a plugin and change IO, the configuration is invalidated)

Extra questions:

1. Is it expected to run more than 1 plugin at once? In parallel? In series too (like a chain of FX)?
2. How are plugin paths going to be configured? Or nothing at all and just require the paths from the VST3 spec and nothing else..?

[ericfont]

I'm a long time user of OBS and Linux Audio (e.g. Jack & Carla), and I just want to say that I think falkTX is thinking the right way on this to use Carla's library to do the heavy lifting. The big plus to me is that Carla also supports LV2 plugin format as well as VST3 (and a few other LADSPA, DSSI, VST2, and AU), and there are tons of open-source LV2 plugins which have been made. So it would would work great if OBS used a framework like Carla's library which could support these different plugin formats, rather than only getting support for VST3.

[falkTX]

I would like to try and do this.
As mentioned in other comments, there is usefulness in reusing Carla components for this, because a lot of the hard work (plugin hosting and bridging) is already done.

Since the request mandates the use of plugin bridges (so plugin crash does not crash OBS) I would skip the integration with Carla and just code directly the bridge support instead.
I have these definitions at https://github.com/falkTX/Carla/blob/main/source/utils/CarlaBridgeDefines.hpp
It basically works with shared memory over 3 data points, one has a semaphore for syncing audio thread and the other 2 use a ringbuffer.
We can call OBS the server, and plugin processing the client. These data points are needed for:

1. Server => Client RT calls, access controlled via semaphore, where server gives the audio/midi buffers, time position information and other things needed for processing.
2. Server => Client Non-RT calls, going over a ringbuffer, for everything that is not related to audio processing such as saving/loading state or showing/hiding GUI
3. Client => Server Non-RT calls, also going over a ringbuffer, for the plugin to report stuff about itself and notify of for example latency changes.

I can do a custom VST3 implementation in Carla so it does not require JUCE for that (which is in my plans anyway).
This is already the case for every plugin format supported (LADSPA, DSSI, LV2 and VST2) except AU which still uses JUCE.

I wouldn't go into the path of requiring Carla to be installed in order to use plugins, but rather copy over any code necessary for this. Or if allowed add carla as a submodule and add a special "make obs-plugin" kinda target for it.
Carla license is GPLv2+, so don't think there is a problem there.

How does this sound?

[DDRBoxman]

Honestly this sounds great! We'll get this marked as accepted and you can get started when you are ready.

I think having Carla as a submodule inside the plugin folder makes sense if that's what you meant.

[falkTX]

Great, and yes I meant for having carla as submodule yes. tbh I havent even looked at the OBS code yet, but I kinda expect how it should work, this is basically "just another" OBS plugin.

I do have other things planned already, so I would start the real work in the beginning of 2022.
Until then I can begin compiling OBS myself and use these builds to get familiar with the build system, I intend to use OBS during December anyway.

Is there an official developer oriented channel for OBS real-time chat? Hopefully something on IRC or Matrix.

[falkTX]

I got confused by github interface on these "discussions" page, meant to reply to your post above, sorry.

[Fenrirthviti]

We have an IRC channel on Libera, but it's pretty much just a placeholder and is not monitored. Almost all of our development discussion takes place on our Discord. https://obsproject.com/discord

[falkTX]

I see, thanks for the offer but I have my disagreements with discord's ToS so won't join there.
It is still possible to keep discussion on this page, will just not be realtime.

[dodgepong]

I have marked this bounty as "In Progress". Thanks for volunteering! Feel free to get to work on your proposal at your earliest convenience. You can post your questions here and we will respond as soon as possible.

[dighawaii]

Any news?

[beddari]

As a reference, VST3 support in Ardour:

• https://ardour.org/features.html#plugins
• https://steinbergmedia.github.io/vst3_doc/

[jpcima]

Steinberg is yet to clarify the license situation. 14days without answer

There is no success in getting Steinberg to address the issue so far, and my oldest request is from 6 month ago.
If someone wishes, feel free to add your voice in that discussion of their forum, if only to make evident that it's a community-wide problem and not me only.

[robbert-vdh]

The issue is more whether or not the VST3 SDK is actually licensed under the GPLv3 license, right? If it is, then this would be covered by Section 7 of the license. Steinberg's VST3 SDK licensing page and the LICENSE.txt file that's part of the VST3 SDK mention that the SDK is available under the GPLv3 license. The actual license text is missing from the repository/archive and the source files don't contain the license headers, but if it is actually licensed under the GPL then the following passage from Section 7 of the license would apply:

All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.

But I am not a lawyer.

[jpcima]

The Fedora team has been queried about the legal problem on their relevant mailing list.
From what I collect, they don't reach a clear-cut conclusion; and they are more willing to strip all VST3 source out of every program, than to get to the end of this dilemma.
It seems to be about whether the license supplement meets the prerequisite for that GPL3 section to be applicable.
There is wait for more news on that front, but it seems in hiatus, and there hasn't been a public statement.

[falkTX]

Hi. Just to give some little report on this.

I have started a native/custom VST3 host implementation, though still on Carla side and not as OBS plugin just yet.
As mentioned above my intention is to have this implemented in Carla, and then integrate the carla host lib directly as an OBS plugin.
This will give not just VST3 support, but all other formats that Carla provides - LADSPA, DSSI, LV2, VST2 and as recently contributed by @jpcima JSFX too (those special FX files from Reaper). AU is possible, but that goes through JUCE inside Carla, can be enabled or disabled as wanted.

Code is split into several parts.

1. Common utils at https://github.com/falkTX/Carla/blob/main/source/utils/CarlaVst3Utils.hpp (WIP, I will keep updating this one)
2. Discovery from an external process, this part is now done https://github.com/falkTX/Carla/blob/91cee1f997508aa23b46911b2cb6050ea6c95bad/source/discovery/carla-discovery.cpp#L1450 (note I might split up this file into multiple ones for easier management)
3. VST3 hosting is in progress, implementation at https://github.com/falkTX/Carla/blob/main/source/backend/plugin/CarlaPluginVST3.cpp

The plugin bridging is something Carla already does, will need to integrate it into such OBS plugin later on.

In order to learn the API properly, I have been implementing the plugin-side of it in DPF develop branch. This part is also still WIP, but already produces working plugins.

VST3 is a LOT more complex than VST2. It is not strictly hard, just more complex due to how many little things it has. And sadly just like how it was with VST2, on VST3 not all hosts and plugins behave the same, some completely break out of spec..
Some things are quite tedious to get right, so it takes longer than I typically hope for.

Anyhow at the moment TODO list is:

1. Create a repo with stub/empty implementation of OBS plugin
2. Finish Carla's VST3 native implementation
3. Map OBS calls into Carla ones
4. Deal with plugin discovery from OBS side (store in its DB for caching known plugin data with timestamps, note not needed for LV2 and JSFX)
5. Deal with OBS + VST3 dynamic IO (Carla assumes static IO, but seems easy enough to change once all other bits are in place)

[yoshinorisano]

Hi!
As there seems to no progress, I’ve just started the VST3 plugin development as my personal experiment, few days ago. This is very early stage implementation. Here is the working demo. Do you interested in making it as an official plugin?
https://twitter.com/sanodigitallab/status/1565577963642376193?s=21

[falkTX]

using the vst3 sdk is a problem for having OBS in a few linux distributions, as previously mentioned before.

and there is progress, I mentioned it just above.

also, I have been reorganizing Carla's frontend code to be able to take its plugin list and other GUI tools (since it is also Qt based) in order to have it working standalone and then be integrated in OBS, WIP branch at https://github.com/falkTX/Carla/tree/reorganize

the plugin discovery is a tough one, we want to have a cache/db of previously scanned files so we dont need to scan anew everytime we start OBS. but it also needs to happen on a separate process as to not crash OBS while the background scanning takes place (it is not unusual for plugin scanning to crash hosts, we want to avoid it at all costs).
Carla already provides this, the mapping of a cache/db file with the external scanning tool is what I need to handle next.

the kinda lack of news is due to me having to deal with employer bankrupcy leaving me in a tough position having to quickly find work hours to do elsewhere.
would be great to work on this obs+plugin for funding, but it is not a simple project that takes only a few weeks of time, as such I had to re-prioritize and work on a few other tasks/jobs first.
once I am a bit more financially comfortable again I can continue my approach.

[falkTX]

CLAP is kinda included in my proposal, by virtue of Carla eventually having support for it too.
My current work is actually for CLAP related things, so that helps the Carla side too because in 1-2 months I will be quite familiar with the spec.

so this is "why not both?" scenario

[BillyDM]

Yeah, I also propose supporting CLAP. We don't even need to use Carla. It's really simple to create a basic CLAP host from scratch in either C or C++. CLAP is going places, especially in embedded environments like game engines and streaming/recording applications.

[yoshinorisano]

Except VST3 SDK is a big mess full of bugs and ignored compiler warnings.
See https://github.com/juce-framework/JUCE/blob/master/modules/juce_audio_processors/format_types/juce_VST3Headers.h for a good showcase of this.

Oh, it's really terrible thing...
I didn't know that.
I understand your point.
Using VST3 SDK is the wrong way to go...

I read some of the CLAP example code.
https://github.com/free-audio/clap/blob/main/src/plugin-template.c
https://github.com/free-audio/clap-host/blob/main/host/plugin-host.cc

It is easy to understand.
Very clean, future proof APIs, I think.

It doesn't solve the original issue (VST3 Support),
but what about adopting CLAP instead of VST3...?

[jeremydk]

I know that I'm on the outside of this, and I understand the challenges around the licensing and the overall code quality of the provided SDK.

CLAP would, unfortunately, not get me what I would need out of this work item. For me, I'm looking specifically for VST3 (Or AU, etc) since some of the plugins that I am interested in running have moved away from offering VST2 versions, and I suspect people that aren't active developers in this space likely have similar requirements.

[BillyDM]

CLAP would, unfortunately, not get me what I would need out of this work item. For me, I'm looking specifically for VST3 (Or AU, etc) since some of the plugins that I am interested in running have moved away from offering VST2 versions, and I suspect people that aren't active developers in this space likely have similar requirements.

Yeah, that is a very valid point. It could be a year or more before a lot of the big plugins get a CLAP release. So I would say if we you want plugins in OBS now, then VST3 is still the best bet.

That being said, CLAP is definitely more future-proof (both legally and in terms of maintainability IMO), and there's no reason we can't support both formats for the time being (other than development time of course). I've actually had quite a bit of experience writing CLAP hosting code, so I might consider adding the CLAP support to OBS myself.

[falkTX]

A small and quick update on this.
I am still finalizing some OBS-unrelated work, but been slowly pushing torwards making carla GUI stuff usable outside of it.
Carla GUI stuff is written in Qt, same as OBS, but uses PyQt and not C++.

As of yesterday I have a working setup where the existing Carla GUI is calling Qt + C++ compiled code, starting with a few dialogs for testing.
The purpose of this is to be able to convert the GUI elements that will be reused on the OBS plugin to C++, while still have that same code working for the current Carla releases, so nothing is "wasted".
First will be the plugin list dialog (for listing and selecting which plugin(s) to load), then the various elements that make GUI-less plugins usable.

For the curious ones, this is how the plugin list looks in Carla now:

On a slightly related topic, CLAP hosting via Carla works now.
So we will be covered for that too, in due time.

[hevisko]

Any progress to perhaps alpha/beta test on OBS 29?

[falkTX]

Progress has been very slow. I have moved to a different contry for work related reasons and for the time being do not have my usual setup for coding (which I have left in the office, for now).
I haven't given up on the task, just life situation being on some weird state at the moment.

The main thing I was able to do before my last post is to generate a single static lib which will provide the audio-hosting side. Basically so that building the project is not a big mess.

I was hoping to do more...
Either work situation gets better and then proper not so stressful working hours, or it gets worse for real and I take a break for finishing this OBS side. I should have a clear timeline by end of march.

[falkTX]

good news...? kinda. as work situation is still not the best at the moment, after speaking with my employer for April I am only doing half the work hours.
this leaves enough free time to handle OBS and plugin support, hopefully I can handle it in a single month, but I am confident it is doable due to already having done the heavy lifting on the carla plugin host side. most of the time will be spent learning exactly how the plugins in OBS work, setting up a project, importing the Qt dialog for plugin list and integrating the audio processing. still need to find a nice way to auto-scan plugins in the background while keeping old scan data, maybe some DB thing

[falkTX]

Finally with something that I can start showing.

There are quite a few ways to go about this, but for a start I am hardcoding loading a specific local plugin and getting some integration going.
Stuff like audio, parameters and UI.

The libobs API seems quite limited, but I am trying to target it anyhow without too many Qt hacks.
Everything that can't be done directly with libobs I am placing on a separate file, maybe those calls eventually are added in the official API in some manner.

I am placing my code at https://github.com/falkTX/Carla-OBS for the moment.
Still way too hacky and experimental, not something you should build.
I read a bit of OBS code and other plugins earlier this month, but only now was able to do actual coding. So it is still missing alot.

Will share more as progress continues.
And here is a screenshot because everyone likes those:

[Fenrirthviti]

I think we are certainly interested to hear what parts of the API were lacking, feel free to share those with us when you can!

[falkTX]

the big one is being able to add idle callbacks on the main thread.
all major plugin APIs (CLAP, LV2, VST2 and VST3) require this in order to get some plugins to work.

looking around libobs I see things related to tasks, but these are on 2nd threads.

for some plugin calls it is mandatory they happen on the main thread.
this is one of the reasons obs-vst doesnt work well on Linux, because it lacks the idle callbacks that give idle time to plugin UIs (on Linux due to lack of common event loop requires special integration with the host for this).

if there is something for this in libobs, please do tell. for now I plugged into Qt directly for enabling main-thread idle callbacks

[falkTX]

I should have looked more deeply, just saw obs_add_tick_callback which does exactly what we need.
So one less custom Qt thing to do, cool cool

[falkTX]

One thing I am not seeing how to handle...
When we have a plugin GUI interaction, it can change plugin parameters as expected. But I am not able to make it update the parameters exposed on OBS side (the source properties).

I have something similar to this for when such an event from the GUI side happens:

    obs_source_t *source = carla->source;
    obs_data_t *settings = obs_source_get_settings(source);
    obs_data_set_double(settings, pname, value);
    obs_source_update(source, settings);
    obs_data_release(settings); 

my OBS module update is called, but the OBS built-in GUI widgets are not updated.

what am I supposed to use for this then?

[falkTX]

Plugin list dialog is slowly being integrated. I have not dealt with the binary plugin discovery yet, but for the formats where it is not required (LV2 and JSFX) we already see it in action.

Need to find a way for OBS to invalidate current properties and request new ones after changing the active plugin, for now closing the "filters" dialog and opening it back up does the trick.

[falkTX]

Now changing active plugin 😎

obs-carla-2023-04-14_01.51.29.mp4

Updating plugin sliders on OBS side updates the GUI, but so far not the other way around yet.

[pkviet]

wonderful work !

[duncte123]

The sliders not updating would explain why the vst 2 plugin does not show the vst settings outside of the custom GUI

[falkTX]

Lots of progress this weekend..

after getting to know the OBS APIs a bit more, I think I have a reliable way to reload the plugin properties (aka the generic sliders).
also found a way to force OBS to update the sliders after a GUI event, a bit hacky as I am basically telling it to reload the entire properties, but it is something that works for now until we have something better.

the state is being saved and restored correctly now, plus audio processing without latency.
(the fixed-block-size that introduces latency is broken atm, will handle it soon.)
my current idea is to present a combo-box with a few buffer size values, so users can decide latency vs cpu usage trade off, with a "dynamic" option that simply takes the audio from OBS and feeds directly into the plugin without any extra latency BUT a few plugins do not like this as the amount of audio frames to process keeps changing..

I added a "input" plugin mode too, useful for having audio plugins working as source of audio input.
for example generators, or those plugins that get audio from an internet stream - those will be usable as "audio sources" within OBS.

plugin bridging is already working because Carla already supports it. I need to tweak the build so it generates the bridge binaries (for out of process we require to run external tools after all).
for the moment I simply build these binaries separately, cmake is not really my thing, something else to deal with a bit later too.
oh and we got CLAP too :)

I split the code in several parts, because I want to rework the way the plugin processing integration is done a bit later on.
for now the current approach works, just a bit limited.

here is a short video demonstrating general things working:

obs-carla-3-2023-04-17_03.15.04.mp4

so in short, TODO remains:

• binary plugin scanning integration (manually loading a DLL already works by the way, useful for testing)
• fixed-buffer-size option
• finishing native vst3 and clap host implementation
• nicely packaging everything
• submit PR

[falkTX]

Time for an update, after weekend and 1st may "holiday".
I am starting to get quite a few details working, but sadly was unable to complete it in the one month that I took for it.
There are just a lot of details involved, so I will need more time.

That said, to give an updated status report:

1. I made the codebase have 2 different "OBS plugins" from the same repository: carla-patchbay that is easy to test and already works 100% but requires external Carla to be installed, and carla-bridge that has everything built-in, from plugin list dialog to bridged audio (the executable for the bridge is still separate, but that is expected)

Both builds implement the API defined in
https://github.com/falkTX/Carla-OBS/blob/main/carla-wrapper.h

Then the entry point that integrates that into an OBS plugin is done in
https://github.com/falkTX/Carla-OBS/blob/main/carla.c

I consider these 2 pieces to be done.

The 1st carla-patchbay OBS plugin is basically taking this API and adapting to one of Carla.
It allows to test everything in regards to OBS side of things, before the more integrated plugin bridge stuff is finalized.
While it is not the purpose of this discussion directly, it can still be very handy, giving a plugin patchbay to setup complex chains of plugins.
Note that we still have all plugins loading in bridge mode.
I dont consider this part of the project to be what the discussion and bounty is about, so I am not counting it.
The code for this is in https://github.com/falkTX/Carla-OBS/blob/main/carla-patchbay-wrapper.c and already works well, would need testing later on if we decide to keep it.

The carla-bridge OBS plugin, what this discussion/bounty is all about, is still in progress.
I switched from using carla helper and engine code (which is mostly not needed and just bloats the plugin) to directly spawning the process that loads the selected audio plugin. As in, there is no need to include the full carla engine code and its dependencies, we can deal with the details directly. The bridge communication and sync mechanism is still through carla, but a very minimal subset.
I got far enough to have audio, parameters, and GUI working, partial state save/restore done and still the same items as in the last post regarding binary plugin scanning and native vst3 hosting.
Not yet in a state where testing would be useful, will continue on it as I have time.

2. The fixed/buffering vs dynamic/direct buffer size is implemented, not yet with a GUI option but already verified to work with the full carla-patchbay. idea is that we can either have OBS drive the audio in slices and give those to the plugins directly (thus no extra latency) or buffer for 128, 256 or 512 samples to ensure plugins receive a fixed-size audio block every run (needed for some plugins)

3. I started to try to understand the CI workflow used by OBS, so we can have automated builds going.
   Have tests happening in https://github.com/falkTX/obs-studio/tree/carla but still a bit early and the repo has some local hacks of mine to allow running OBS without installing.
   Last time I checked the Windows builds were failing in a random point, macOS complains about missing cmake call (is it using the OBS compat/legacy cmake setup?), but at least the Linux builds are succeeding.
   Once I am a bit more ahead on the project, I can clean up my hacks on another branch to get some early testing going.

[falkTX]

Things are progressing, finally have a working CI setup so the generated binaries can be tested.
Was particularly painful for Windows though as I do not have a setup with MSVC compiler, and not used to it anyhow, but after slowly tweaking things to get it to accept my code it is happy :)
(the carla codebase was not made with MSVC in mind, so lots of tweaking needed)

This brings me to 2 questions for OBS developers/team:

1. Is it more useful to have an early pull request for early review, or only open the PR after everything is pretty much finalized?

2. For the discovery/bridge binaries I am hoping we can skip the MSVC compat, and build them with mingw "out of tree", or perhaps I create a redistributable project/download alike CEF with automated builds and releases.
   It can be part of OBS too, I have the Carla builds automated via github actions after all, but I would very much appreciate if it could be its own separate step and not need to involve it within the cmake build step. (Carla source code uses hand-written Makefiles, not cmake).
   So we would build these tools separately (say coming from a new job rule in the BUILD.yml) and then package it together in a plugin subfolder at the end.
   Would this be acceptable?

[falkTX]

I will take a look at how obs-deps does things, thanks!
Can we assume Windows to have some form of mingw installed, or not is that not even a guarantee?

[pkviet]

No you shouldn't assume for mingw on any windows install. The msvc redists can be assumed though.

[falkTX]

I can setup something that works via cross-compilation on Linux first (since I already have that working for my own builds), and then later we figure out how to deal with this better.
What do you think?

PS: Note that the OBS plugin is built directly within OBS with cmake. I am only talking about a few separate, standalone tools that deal with plugin discovery and bridging. These binaries are fully static and self-contained

[PatTheMav]

We only cross-compile FFmpeg on Linux right now, introducing an unrelated package might mess this system up, but apart from Carla being bundled as an ‘FFmpeg’ library in the package it could be made to work, but that’s a last resort-kind of thing.

As for the plugin that depends on whether the idea is to make it a core plugin - that’s the only scenario in which we support an in-tree build, but it’s then required not to build any compile-time dependencies itself (those need to be packaged in obs-deps).

[falkTX]

ok I am going to try with regular all-within-cmake build.
when disabling all the extras (sf2, sfz, jsfx, internal plugins) the needed cmake setup is not that complex in the end: https://github.com/falkTX/Carla-OBS/blob/main/cmake/carla-bridge-native.cmake

so far works on linux, fixing up macOS next, then will see how far I get with MSVC.

[PatTheMav]

What's the status of native Apple Silicon (arm64/arm64e) and AU support? Last I checked Carla only supported 32-bit AUs which seemed quite out of date. 😅

Also when I tested your code on macOS 13, none of my VSTs (VST1 or VST3) or AUs were detected.

[falkTX]

What's the status of native Apple Silicon (arm64/arm64e) and AU support? Last I checked Carla only supported 32-bit AUs which seemed quite out of date.

Carla and ARM builds have been a thing before Apple came with support for it.
There is even custom code in Carla to handle bridging x64 binaries in arm64 systems, and I have added this part in the OBS plugin too (for now requiring libmagic/libfile for figuring out the binary arch of a plugin)

Also when I tested your code on macOS 13, none of my VSTs (VST1 or VST3) or AUs were detected.

There is no auto-scanning, it needs to be done manually.
The AU and VST3 on current Carla release is done via JUCE, so that wont be available as-is for this OBS plugin. (I am purposefully disabling the JUCE-Carla integration for the OBS plugin)
VST3 is coming, already in progress with audio and parameters, with a native host implementation. (not using JUCE)

[falkTX]

Another update: I think I got most things working now, still a bit in a hacky way so I need to do some cleanup.
The deal with MSVC was a bit of a pain, the cmake version provided by VS2019 is too old to target VS2019 itself or something alike that I couldnt really grasp too well. I was able to get things building just enough to see what the main points for compatibility would be, and it was mostly VLAs and a few functions that mingw has but MSVC does not.

I have an implementation for auto-plugin-scanning now, though for now it rescans on every run (need to see how OBS plugins can save their own global settings and hook into that).
It is enough to get plugins listed automatically, which blocked other things from happening.

With that in place I continued on the VST3 hosting, finally in a state where it starts to be usable.
This means audio, parameters, state and custom GUI.

During the week I will try to open a PR for early review and comments, as to not waste time in case I am doing things incorrectly or wrong. Also because at this point I need confirmation regarding the bounty, but let's discuss things once I do a little bit of cleanup and hopefully also have it in some "early tryout but dont use in production" kinda build going.
Soon...

[PatTheMav]

Yeah adding both arch switches works, clang will create a universal binary then. Standalone plugins are actually expected to be shipped as universal (I'll also switch our default dependencies to universal arch soon).

OBS itself will be a separate app for the time being though, because shipping CEF as a universal binary makes OBS a massive download.

Also check the dal-plugin in our current tree, which we build for x86_64, arm64, and arm64e - you can in theory overwrite the target architecture for just that target in a specific way - that might interfere with xcodebuild, but only testing will tell.

[falkTX]

the -arch... flags didnt work, but setting some cmake properties did.
the OBS CI builders dont have libmagic that I was relying on, but I found a manual way to see if a specific binary is x64.
so this now works too.
can be seen on the screenshot below, host system is arm64 (an M1 mac mini), with OBS built for arm64 but loading a x64-only plugin.

[PatTheMav]

You can just use xxd to check the first 4 bytes - we use that on obs-deps to check if a binary is universal or single-arch. There’s no need for extra libraries.

Or did you require this in Carla itself?

[falkTX]

Needed in carla too, and for arm64 vs intel64 it is 5 bytes we need, the first 4 are the same between arm64 and intel binaries.

Things are more tricky when it involves universal/fat binaries, as we can have builds for x64+i686+ppc, the older rosetta stuff.
For now I just added a simple check to be able to load x64-only plugins, there are very few universal non-arm64 macOS binaries out there so it is not worthwhile to check all the options for OBS. This is where libmagic in Carla comes into play, it can list all the weird combinations inside an universal binary for us.
But here for the case we care about (old macOS x64 plugins not yet ported to arm64) we should be good now.

[PatTheMav]

Just saw, we just check for cafebabe and cffaedfe because those are the only "valid" ones for modern macOS. I don't think Logic supports PowerPC-based and 32-bit AUs or VSTs anymore, but I haven't checked that explicitly.

[falkTX]

Alright, the initial pull request is now at #8919
I cleaned up the code a bit and got rid of the extra stuff not relevant to the PR.

As I mentioned before, it is not meant to be merged as-is right now, but to be used for feedback regarding my approach to the plugin hosting details.
The implementation should be considered work-in-progress.

Now I ask the OBS team for feedback. either on the PR side or on this discussion topic.
Is the code structure and approach okay? Specially for using Carla as submodule, what is your take on it?

I will fix any show-stopper bugs that might appear, but be on hold for now until some positive reply.
Thanks!

[falkTX]

Bringing some news, but not all good.

For those that have not been following the pull request, the entire thing pretty much works now besides a few minor points.
There is a crash when starting OBS on Windows, I suspect some memory corruption, but from my testing it works fine on macOS and Linux. Need help for those more familiar with debugging on Windows to sort out the issue.
I have setup an Ubuntu PPA for easy install (for Ubuntu users only of course) at https://launchpad.net/~falktx/+archive/ubuntu/carla+obs/
For macOS sadly there are no automated builds of any kind, because the OBS CI setup only generates macOS packages when they are created from the OBS repo and it needs the other pull-request at the obs-deps side to be merged first before the feature can be activated here.

Now I was hoping for a little more feedback and a faster response on OBS side. Initially my idea was to take some days to do this bounty, for which I could directly invoice OBS once that part was complete. But this requires the OBS folks to actually merge things to get it rolling, which did not happen (speaking about the obs-deps PR here, which is required to enable the plugin host feature)
Due to how freelance stuff works in Portugal, I need to pay a monthly fee/tax for social security while I keep my freelance activity open. At this point I was losing money expecting things to move... and sure I could have finished a couple more details on this PR/bounty, but the major points are not on my side anymore.

So I am now employed with a regular work contract, no longer able to send invoices as freelancer.
With no confirmation on this PR going forward, I need to stop pursuing this bounty in at least any sort of "want to get it done asap" fashion.
The feature works well under Linux, which is where I tested it on most of the time. But I can't dedicate much time to it now due to personal circunstances again. I will continue at it, but at much slower pace and still focused on the Linux side of things.

[falkTX]

The Ubuntu PPA at https://launchpad.net/~falktx/+archive/ubuntu/carla+obs is updated to match the recently released OBS 29.1.3

This PPA is basically a repackage of the official OBS one from https://launchpad.net/~obsproject/+archive/ubuntu/obs-studio
See https://github.com/KXStudio/Repository/tree/master/sources/apps/obs-studio for the script and patch file used to generate the custom OBS packages.

[ThorinWolf]

Note! This no longer compiles, at least on Arch, as it requires Qt3 - which no longer compiles on Arch under normal operation. This could be still considered an open issue/in need of forking.

[Fenrirthviti]

As a note, this bounty is open again and we are accepting new proposals for VST3 support, if there is any interest in working on this.

Further context can be found on the previous PR

[hevisko]

What is/was wrong with the previous work?

…

On Mon, 11 Nov 2024, 04:13 Joel Bethke, ***@***.***> wrote: As a note, this bounty is open again and we are accepting new proposals for VST3 support, if there is any interest in working on this. Further context can be found on the previous PR <#8919> — Reply to this email directly, view it on GitHub <#5074 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABFCG7KTNOGHIQP7CG4BS32AAHGPAVCNFSM5BUG7WF2U5DIOJSWCZC7NNSXTOSENFZWG5LTONUW63SDN5WW2ZLOOQ5TCMJSGA4TKOBQ> . You are receiving this because you commented.Message ID: ***@***.***>

[ThorinWolf]

Two things:

1. the previous proposed solution relies on Carla as an external program, rather than being a full solution that can exist within the OBS codebase.
2. Nobody can get it to compile because it relies on Qt3, and nobody can compile Qt3 anymore.

[falkTX]

1. the previous proposed solution relies on Carla as an external program, rather than being a full solution that can exist within the OBS codebase.

My initial proposal included Carla as submodule, but got rejected and I was asked to make carla an external dependency.

Nobody can get it to compile because it relies on Qt3, and nobody can compile Qt3 anymore.

It works with both Qt5 and Qt6, I did the Qt6 early port of the plugin dialog specifically for OBS. I dont know where the Qt3 notion comes from. There is some old Qt4 specific code in place but that is legacy things that I didn't bother to remove just yet.

[ThorinWolf]

1. the previous proposed solution relies on Carla as an external program, rather than being a full solution that can exist within the OBS codebase.

My initial proposal included Carla as submodule, but got rejected and I was asked to make carla an external dependency.

...well if that wasn't the issue, then what was? I'm curious, as that seemed like the issue from reading the mentioned discussion.

Nobody can get it to compile because it relies on Qt3, and nobody can compile Qt3 anymore.

It works with both Qt5 and Qt6, I did the Qt6 early port of the plugin dialog specifically for OBS. I dont know where the Qt3 notion comes from. There is some old Qt4 specific code in place but that is legacy things that I didn't bother to remove just yet.

This is interesting, because someone built a MAKEPKG script for the git version on the AUR that demands Qt3.

[PatTheMav]

To dispel any possible myths around this:

The first suggested solution was ultimately rejected because it was architecturally too complex for the project and thus had a high risk of becoming long-term unmaintainable associated with it.

What OBS needs is an implementation of a VST3 plugin host. The suggested solution shipped a trimmed down version of a 3rd-party plugin host that is interacted with by the plugin. This creates an unnecessary additional layer of redirection in the application (OBS to Carla to VST) but also would have required OBS Studio to effectively ship a hidden, but also customised, version of Carla with it.

Architecturally it would not have been different to - say - ship a plugin that can open photoshop files by including a hidden copy of Photoshop itself.

It added complexity not only to our dependency build systems, but also lots of additional complexities to the plugin itself and OBS packaging systems, because the way Carla had to be shipped, deployed, discovered at runtime, etc. were different for each OS.

[PatTheMav]

I'd also be amiss not to point out that we appreciate the effort that was put in by @falkTX even if we ultimately decided against using their proposed solution. That decision was (and should not be read as) a judgement on Carla itself in any way.