Extendable OSC controller

[Zalastax]

As requested by ghales in https://club.tidalcycles.org/t/rfc-improving-the-interface-for-interacting-with-tidal-streams/3528/4, Tidal could allow for extending the OSC controller. The requested idea is to allow callbacks to incoming OSC messages.

My suggested way to implement this would be to make the whole OSC controller replaceable and provided through configuration. The default configuration would be the current OSC controller. We should also create a function that returns an OSC controller that has the same base behaviour as currently + accepts a list of callbacks that are invoked for each incoming message.

[matthewkaney]

Sooo, as it happens, I've actually built a mostly-complete, if still speculative, version of that in this branch. There's still some bugs with it, and maybe it makes sense broken up into a couple of PRs, but I can go back in soon and polish it up, and then we can do some review.

[Zalastax]

Great, @mindofmatthew! Some polish and we should be good to go 👍

Compared to my suggestion, this doesn't make the whole OSC controller replaceable. Do you agree that we should implement that in addition to the changes you are already introducing? My view is that it would be good flexibility to provide.

[matthewkaney]

I haven't added the functions yet, but this would allow you to attach individual controls to a stream, like so:

-- Add a new listener action (the returned Int can be used to remove the listener action later)
streamAddListener :: Stream -> String -> ([Value] -> IO ()) -> IO Int

-- A BootTidal abstraction
onOSC :: String -> IO () -> IO Int
onOSC name action = streamAddListener tidal name (\_ -> action)

onOSCArgs :: String -> ([Value] -> IO ()) -> IO Int
onOSCArgs = streamAddListener tidal

-- Which could be used like
onOSC "/button_1" $ d1 $ s "bd sn"
onOSC "/button_2" $ d2 $ silence

onOSCArgs "/knob_1" (\v:_ -> all (# lpf v))

I could see the usefulness of swapping out the listener at a higher level, but I don't have a clear sense of how it would fit in with this yet. Can you give me an example of how you're imagining it would be used?

[Zalastax]

I would like to make the OSC controller abstract to the Stream module and not tie it directly to the new Sound.Tidal.OSC.Listener module. The motivation for my suggestion is to make the core simpler. It could also let someone experiment with other ways of implementing OSC controller extensibility or completely replacing the base commands.

The individual controls could still be added like you imagine, but they would not be added to the Stream but instead to the instance of OSCListener. attachListenerActions would need to be received as an argument.

It might be best that you complete your vision of your current change and that I make my changes in a separate PR since it can be difficult to get the idea across this way and you might not be so interested in completing my vision as it likely requires some elbow grease.

[jarmitage]

This seems to be somehow related to: #677

And also tidal-listener? https://github.com/tidalcycles/Tidal/tree/main/tidal-listener

Any thoughts on a way forward here?

[matthewkaney]

The good news and bad news is that all major development now is directed towards a 2.0 release, which means there's an opportunity for serious rewriting, but also it's going to take a while.

Towards the previous point about simplifying the core, my thinking these days is that it's not necessary for the Stream module to implement an OSC listener at all. Except for the bus handshake (which should only be enabled for superdirt targets anyway), the listener just controls the stream through public methods. In the "vanilla" Tidal setup in BootTidal.hs (or whatever) we'd probably want a default listener, but that can be a separate layer of functionality that wraps around a stream.

This means that it's probably possible to do what you're asking, @jarmitage, in a separate module that could be loaded in a BootTidal.hs file and attached to an existing Tidal stream. Are you primarily concerned with a mechanism for mapping OSC paths to Tidal control inputs?

[jarmitage]

Towards the previous point about simplifying the core, my thinking these days is that it's not necessary for the Stream module to implement an OSC listener at all.

Does the same apply for OSC senders?

Ideally there would be a similar UI/API for sending and receiving

[matthewkaney]

Does the same apply for OSC senders?

Yes, inasmuch as the Stream class needs some sort of low-level abstract "target", which would essentially be a function that takes the current events and does something. That would usually be sending OSC messages, but I don't think that should be the only possibility. Currently Tidal builds all of its OSC logic (much of which really only applies to SuperDirt) into the core of the engine, which makes it difficult to extend it to other usages.

That said, I imagine there will be a default interface for defining both OSC listeners and senders, and that the default Tidal configuration would expose all of this. The experience for the typical Tidal user might not be dramatically different than it is now, but the innards would allow a lot more lower-level customization.

[jarmitage]

Ok, I'm out of the loop here... What do you suggest, should we create a new thread specifically about input/output UI (including OSC and possibly other things) in Tidal 2, or is it too early at this point

[Zalastax]

This will be an orthogonal question to most other changes in 2.0 so I see no point in waiting. We need an overview of the use cases, a discussion about the preferred architecture, and that someone takes lead in the implementation.

[jarmitage]

If OSC targets are not going to change in tidal 2, then why not follow the design I described in #677?

[Zalastax]

As @mindofmatthew suggests, we might want to allow for other input/output than OSC. If that's our goal, the described design doesn't take us there. I personally have no use cases around this question at all, so I can only help with evaluating the different architectures we come up with l. It would be great if @mindofmatthew can list some use-cases where being bound to OSC would be detrimental. I like it because it seems cleaner but cleaner is not a great motivation for a large overhaul.

[jarmitage]

In the interests of simplicity and not overhauling things without good reason, wouldn't it be better to assume that whatever IO people want with Tidal can be facilitated with OSC? We used to have tidal-midi but that never worked well. We could have sockets and/or websockets to complement OSC, or serial or something, but I don't see an urgent need for any of those. Anyone needing that connectivity would be able to go via OSC if needed. Adding more abstraction layers and protocols for IO seems like a lot of work for marginal gain, unless I'm missing a big set of use cases.

#677 is a more incremental approach that would be immediately familiar for anyone who has written an OSC target before. @mindofmatthew how does your approach compare?

[matthewkaney]

My approach is ultimately that the Tidal stream wouldn't care what listener you have:

tidal <- startStream config listeners

let myCtl = Controller
-- The default could be @jarmitage's design,
-- and a user could also swap in their own custom implementation

attachController tidal myCtl

Non-OSC IO is definitely a nice outcome, but I agree it's pretty speculative. Setting that aside, here are more immediate use-cases:

1. The Stream.hs module is pretty difficult to work with as currently implemented. The above design would pull the OSC listener code out of that file without needing any new stream code (because the relevant protocol for setting control values already exists in streamSetF and friends)
2. Having more separation of concerns makes it easier to write unit tests of this code, because you can swap in a test implementation of a controller, etc
3. Editors might also want to inject their own controllers (for keyboard shortcuts, custom UI widgets, etc). Ideally, this would happen in code that runs outside of the BootTidal.hs file. If one canonical set of listeners is set in stone at the time the stream is created, this is impossible
4. Someone who's setting up an OSC listener will probably have at least a little debugging or incremental development to do. If you can dynamically attach controllers, then you can attach a controller, make some tweaks to it, replace the previous active controller with your new version, and so on—all without needing to restart Tidal

[matthewkaney]

For context, I'd also like OSC targets to behave similarly in Tidal 2, more like this:

tidal <- startStream config
streamAddTarget tidal superdirt -- or whatever