Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer is still processing an 'appendBuffer' or 'remove' operation.

[ItzDerock]

I'm using SFMediaStream for a proximity chat mod for Among Us. I ran into an issue when trying to use it. The issues is:

sfmediastream@latest:8 Uncaught DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer is still processing an 'appendBuffer' or 'remove' operation.
    at l.a.append (https://cdn.jsdelivr.net/npm/sfmediastream@latest:8:4518)
    at d.t.receiveBuffer (https://cdn.jsdelivr.net/npm/sfmediastream@latest:8:3385)
    at f.<anonymous> (https://amongus.derock.dev/play.html?u=UEQXAJ&p=Derock:263:39)
    at f.r.emit (https://cdn.socket.io/socket.io-3.0.1.min.js:6:2341)
    at f.value (https://cdn.socket.io/socket.io-3.0.1.min.js:6:30320)
    at f.value (https://cdn.socket.io/socket.io-3.0.1.min.js:6:30041)
    at f.value (https://cdn.socket.io/socket.io-3.0.1.min.js:6:29648)
    at b.<anonymous> (https://cdn.socket.io/socket.io-3.0.1.min.js:6:33736)
    at b.r.emit (https://cdn.socket.io/socket.io-3.0.1.min.js:6:2341)
    at b.value (https://cdn.socket.io/socket.io-3.0.1.min.js:6:17782)

Here's the code I use for the client:

        socket.on('newBufferHeader', (data) => {
            if(!otherStreams[data.id])
                otherStreams[data.id] = new ScarletsAudioStreamer(200);
            otherStreams[data.id].playStream();
            otherStreams[data.id].setBufferHeader(data.data);
            console.log('[WS] Got headers for ' + data.id)
            // otherStreams[data.id].audioConnect(ScarletsMedia.audioContext.destination);

            if(nextUp) //handleNextBuffer(nextUp, otherStreams[data.id], data.id)
                otherStreams[data.id].receiveBuffer(nextUp);
        })
    
        socket.on('data', (data) => {
            console.log('[AUDIO] Got audio for ' + data.id)
            if(otherStreams[data.id])
                //handleNextBuffer(data.data, otherStreams[data.id], data.id)
                otherStreams[data.id].receiveBuffer(data.data);
            else {
                console.error('[AUDIO] Missing headers for ' + data.id)
                console.log('[WS] Asked for headers')
                socket.emit('requestBuffer', {need: data.id});
                nextUp = data.data;
            }
        })

The client acts like a presenter and a streamer, both delays are synced up (200ms).
For streamer I use the following:

        const presenter = new ScarletsMediaPresenter({
            audio: {
                channelCount: 1,
                echoCancellation: false
            }
        }, 200);

        presenter.onRecordingReady = (packet) => {
            console.log('[AUDIO] Ready to stream audio');
            headers = packet;
            socket.emit('bufferHeader', packet);
            console.log("[WS] Sent headers");
        }
        presenter.onBufferProcess = (packet) => {
            socket.emit('data', packet);
            console.log('[SEND] Audio Sent')
        }

Sometimes it'll work for a few seconds, other times it errors after receiving one chunk.

Any help would be appreciated.

[StefansArya]

Hi!
does it works for 20 seconds before the error?

If yes, maybe the process for removing the old SourceBuffer was pretty slow on the browser.
The internal process for removing the SourceBuffer usually will running after having 20 seconds of audio chunks (line 40) and after the update process was ended (line 16). The reason why it need to be removed is to avoid memory leak.

SFMediaStream/src/MediaBuffer.js

Lines 11 to 44 in b9f895e

	scope.source.onsourceopen = function(){
	sourceBuffer = scope.source.addSourceBuffer(mimeType);
	sourceBuffer.mode = 'sequence';
	sourceBuffer.appendBuffer(bufferHeader);
	sourceBuffer.onupdateend = function(){
	if(removing === false) return;
	removing = false;
	totalTime = 0;
	sourceBuffer.remove(0, removeCount);
	removeCount = 20;
	};
	sourceBuffer.onerror = console.error;
	};
	scope.source.onerror = console.error;
	scope.append = function(arrayBuffer){
	if(sourceBuffer === null)
	return false;
	if(sourceBuffer.buffered.length === 2)
	console.log('something wrong');
	sourceBuffer.appendBuffer(arrayBuffer);
	totalTime += chunksDuration;
	// console.log(totalTime, arrayBuffer);
	if(totalTime >= 20000)
	removing = true;
	return totalTime/1000;
	}

It may possible that the browser was running intensive tasks and slowing down the buffer removal process.

---

If it was error before 20 seconds.

• It may be possible if a single instance of AudioStreamer is receiving multiple buffer from 2 Presenter at the same time and causing the appendBuffer failed to processing 2 task at once.
• But it also possible if the buffer chunks was delayed at the server and then the server sent 2 packet at the same time.

Well because you was found this issue I think the library will need to buffer the buffers 😅.

---

Btw can you try increase the delay for the Presenter and the Streamer?
If it's still produce the same error maybe replacing .receiveBuffer with .realtimeBufferPlay will solve it, but it may have sound gap because it's immediately play the received buffer.

[ItzDerock]

I've tried it with 1 second and 500-millisecond delays, but for my application, having as close to realtime is very important. The error occurs before 20 seconds. The server also does some processing to find the correct person to send the chunk to, so it is possible that it gets delayed, but not by much, all the server does is calculate every player in a certain radius. I'll try realtimeBufferPlay. Is it as simple as swapping receiveBuffer to realtimeBufferPlay, or is there something else I'll need to change?

[ItzDerock]

Using realtimeBufferPlay works! Just a little choppy on the audio.

[StefansArya]

Alright thanks, I think I know how to fix it because of your information.
I'll try to fix it after I finished with something 😄

[StefansArya]

Hi, can you try with this version? https://cdn.jsdelivr.net/npm/[email protected]

Btw I may increase the major version on the future because I want to support WebRTC and modify the player.
Make sure you have specified the version instead of latest.

[ItzDerock]

Hi, can you try with this version? https://cdn.jsdelivr.net/npm/[email protected]

Btw I may increase the major version on the future because I want to support WebRTC and modify the player.
Make sure you have specified the version instead of latest.

Should I try realtimeBufferPlay or receiveBuffer with that version?

[StefansArya]

The receiveBuffer 😅

[ItzDerock]

Alright, I'll test it out when I get the time.

[devkrxspl]

1.2.1 still appears to have this issue

[devkrxspl]

From trial and error though, a good enough fix is just to use realtimeBufferPlay with ~200 ms latency. There is a bit of a delay between when you talk and when you hear your words, but from experience testing with people you don't notice it. (You only notice it if you're streaming to yourself)