plmpeg example has choppy audio on the web demo

[frink]

I've seen the note about sample rate conversion in the source. However, I would have expected it to work at least on WASM without problem. Is the problem I'm seeing related to sample rates? The video is crystal smooth so I don't thing internet transmission is a problem... Any ideas?

[floooh]

You can check on the Javascript console, there should be a message:

sokol_audio.h: sample rate  44100

If the displayed sample rate isn't 44100, then the audio artefacts are definitely caused by the missing sample rate conversion, otherwise it must be something else.

FWIW, I just noticed that all WebGL demos are stuttering extremely in Chrome on my Mac currently, but that's most likely caused by the macOS Beta that I'm running.

PS: I want to give miniaudio a try when I can find some time (https://github.com/mackron/miniaudio), this has features that are missing in sokol_audio.h (like filters and sample type and rate conversion). If the WASM binary size doesn't grow much versus sokol_audio.h I might drop sokol_audio.h in favour of miniaudio (and if not, keep sokol_audio.h's feature-limited as it is now, but use miniaudio for more "demanding" scenarios).

[frink]

It says 44100 but I'm still getting pauses sometimes... So not a sample rate thing I guess.

Are you reporting xruns of the audio buffer? Seems like that might be the only way to debug to see if we are getting jittery timing in the event loop. Audio needs absolute consistency to stay glitch free. 44.1k samples a second is a lot less forgiving than 30fps!!!

As to MiniAudio - I'm looking into it for a project I'm starting soon. The interface is very clean and it is easy to think through the workflow as a human. Development is growing a lot right now. They are almost done with the mixing and grouping stuff and talking about doing 3D audio next.

SoLoud looks good too, but I'd like to keep as much development as possible in C. I'm even considering using cimgui to avoid the entire weirdness that is modern C++. You've written quite a few bog posts defending C. I'm curious what your advice would be?

...And thus I totally hijacked my own thread!!! 😛

[floooh]

Can you post your setup (e.g. operating system and browser)? sokol_audio.h currently uses ScriptProcessorNode, which I found to be the most glitch-free solution compared to the alternatives (e.g. buffer queueing which is used by the emscripten wrapper APIs). AFAIK miniaudio also uses ScriptProcessorNode. Ultimately this needs to be ported to Audio Worklets (there's already a PR around), but this is also problematic, because a straightforward solution requires pthread-style threading, which can only be used if the developer has control over the web server configuration (because of specific CORS headers that must be present). So audio on the web still is a veritable shitfest all around, and it doesn't look like this will change anytime soon :/

Does this sample also have audio glitches?

https://floooh.github.io/sokol-html5/saudio-sapp.html

This uses feedback from the audio "thread" to figure out how much new data must be pushed (this means it should work with any sample rate):

And what about this one:

https://floooh.github.io/sokol-html5/modplay-sapp.html

This uses yet another method and "pulls" the required amount of new samples to fill the stream buffer from the modplayer library.

Both are different than the plmpeg sample which simply pushes new samples as soon as they are ready, and depends on the sokol-audio-internal ringbuffer to "smooth things out".

If both of the simple demos are glitchfree on your setup, and the sample rate matches in the plmpeg sample, then another reason might be an incosistent rate at which the frame callback is called by the browser (this may happen even if the rendering appears to be smooth, but the difference is usually much less than a millisecond). Maybe tweaking the streaming buffer size might also help, currently this is 4096 sample frames (however this already twice of of the default which is used by the other demos).

[frink]

The other two demos work flawlessly... I've been testing things on a somewhat older Chromebook Flip (C100P) but the OS is up-to-date. I'm guessing the same thing you mentioned - that there is an inconsistent rate for the frame callback causing glitches. It does not seem to be consistent when and where the glitches are heard. Sometimes its glitch free for 20 seconds. Other times not...

I thought Chrome and Firefox both allowed threading in WASM now. I didn't realize CORS was required to get things working. I'd not seen Audio Worklets before, but they seem like the right way to go. I'd love to see a few examples of MiniAudio handling this stuff. I'm surprised they aren't using Audio Worklets...

[floooh]

The problem with Audio Worklets is that they will only be an improvement over ScriptProcessorNode when the entire audio synthesis can be done from within the audio-thread callback. As soon as the audio synthesis needs to happen on the main thread (such as in my emulators, or with the plmpeg decoder), the sample data must also be pushed to the audio thread through an intermediate ring buffer. This is essentially the same thing that ScriptProcessorNode does anyway, but this time it must be done through user code, and if the intermediate buffer is too small, the audio thread will also starve, so it's still necessary to balance audio latency vs audio glitches.

[frink]

That makes sense...

I thought we got full pthreads in WASM a year ago. Are there still a lot of caveats there?

[floooh]

The SharedArrayBuffer feature (which is the base of WASM pthreads) was disabled in all browsers after the Spectre/Meltdown "incident". In Chrome it had been enabled a little while ago, and in Firefox only with specific HTTP response headers (which basically requires that you have control over the web server configuration)... and it looks like Chrome will have this requirement too. That's why I'm not entirely a fan of the feature ;)

See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer/Planned_changes

[frink]

I see what you mean now. I hadn't dug that deep. This is a real mess. I get the security implications though. You don't want a malicious WASM binary spinning up threads and eating all your memory.

That's not an easy problem to solve at all...

There is no easy way around: You either compile the code into multiple binaries and then have to message between the code. Or you somehow figure out how to fudge things with Audio Worklets. Or you fiddle with headers until you can get SharedArrayBuffer working. Or you try to do something that isn't multi-threaded and hope for the best. Everyone of those options suck big time!!!

I really hate the idea of forcing audio in the main loop...

Of these, the most surefire approach to glitch-free audio seems be to creating two separate WASM apps one for the display loop and one for the audio loop. Then create some sort of message bus in JavaScript between them. But why oh why??? OH THE PAIN!!! - I need a drink!!!!!

And it's laughable because all you're trying to do is play an mpeg for crying out loud!!! We've been able to do that on the web for ages. Why does it have to be so difficult in WASM?

[frink]

Just reading a little more on this... CanIUse shows Chrome, Edge and Opera support threading just fine, but only on desktop. Chrome Mobile and Safari now requires a special flag to enable it after Spectre+Meltdown. Firefox requires special HTTP headers as you mentioned...

So basically half the market supports SharedArrayBuffer and threads in Enscripten out of the box. That's enough to consider supporting it. But you're right this is still very much a mess...