Audio delay

[mjb83]

So to start as a question, would 20-30 second audio delay be considered acceptable delay or is the expectation real time? I have 4 cameras, 2x v2, 1x cam pan v1, 1x outdoor v1 and they all have around the same time delay. I can confirm two playback outputs tinycam pro for live monitoring in the kitchen as well I have the feeds all being sent to blue iris for video history / playback. Same delay exists on from both output feeds when watching live, looks to be RTSP not syncing the audio and video.

Not the end of the world but two of them are being used as baby cams so I would of course rather they were real time. Thoughts?

What would you need to diagnose, bridge logs don't seem to show any issue. Maybe I have a setting wrong in the config?

[mrlt8]

Just to confirm, is the delay 20-30s behind real-time or behind the official app/tinycam?

I believe there is an option in the TUTK avclient to request that the data be synchronized, but it ended up causing the video frames to be dropped when I was playing around with it.

I'll have to look into this again and see what they're doing in the official app.

[mjb83]

The video seems to be real time it's just the audio is delayed. So I'll see one of the kids make a noise and then I'll hear it later. I can also hear them upstairs if they're really loud (real time of course) and then I get the 'echo' downstairs on tinycam after.

…

On Mon., May 23, 2022, 12:00 p.m. mrlt8, ***@***.***> wrote: Just to confirm, is the delay 20-30s behind real-time or behind the official app/tinycam? I believe there is an option in the TUTK avclient to request that the data be synchronized, but it ended up causing the video frames to be dropped when I was playing around with it. I'll have to look into this again and see what they're doing in the official app. — Reply to this email directly, view it on GitHub <#388 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWLQ5B7DU2AUUG2XWAARMCTVLOTS5ANCNFSM5WWIFMCQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mjb83]

Any luck turning the data sync back on? I'm seeing considerable delays now on my wyze v3 that I just added. It's delayed about 2-3 minutes now.

[mrlt8]

Does the delay increase over time or is it a constant delay?

Can you post the audio codec for the cam with the 3-min delay? I believe they keep switching the audio codecs around with some of the firmware updates.

[mjb83]

It might be increasing, I'm not exactly sure but I could try to time it?

Log is here, looks like it's using alaw:
2022/07/10 21:04:35 [WyzeBridge] 🎉 Connecting to WyzeCam V3 - on (1/3)
2022/07/10 21:04:38 [] 📡 Getting 120kb/s HD stream (20fps) via LAN mode (WiFi: 57%) FW: 🔒 (DTLS) (2/3)
2022/07/10 21:04:38 [] 🔊 Audio Enabled - ALAW/16,000Hz
2022/07/10 21:04:38 [] WARNING: Skipping smaller frame at start of stream (frame_size=1)

[mrlt8]

@mjb83 Is the audio distorted in any way compared to the stock app?

[mjb83]

Actually... having checked the actual app the sound is kind of garbled... Maybe there is something wrong with the camera?

Strange though as I had a cam pan in there previously that I retired and replaced with the V3. Maybe I need to roll back firmware or something..

[tmeekes]

Can also confirm I get serious audio delay on my Wyze Cam v3 too... haven't dug in to how long or significant it is... I just stopped using the audio because of it.

[mrlt8]

hmm, I'm unable to replicate the delay. I tried comparing the audio from the bridge to the audio from firmware rtsp on a v2/v3 but they're both very similar.

Could this be a firmware and/or camera issue?

[mrlt8]

@tmeekes can you post the FW version your camera is on and what audio codec you're getting?

[tmeekes]

@mrlt8 Let's see here, I've got two cameras here, running 4.36.9.139. I've set the env variable to use AAC.

Interestingly, it's not just the rtsp stream... I popped open the HLS stream and there's delay on there as well. Could it possibly be an issue with computing power? I am running a series of services on this PC... I wonder what it would look like on hw running nothing else... hmmm. At the same time, the video runs with limited delay, so I can only imagine.

Thoughts?

[mrlt8]

Hmm. Could you try removing the AAC option to see if the re-encode is causing any issues?

[JamHeHe]

I've just recently started using wyze-bridge with a pair of Wyze Cam v3 and Zoneminder. I too see a delay in the audio, and have tried both with and without the AAC setting. I've also noticed a pattern of my Linux host becoming nonresponse after some time, typically less than 24 hours. I suspect it's related to the audio issues as I disabled audio yesterday, and things have been stable since. The one time I was able to use top when the system was mostly non- responsive (I already had a console session open), the wyze-bridge container was thrashing the CPU and the io waits were massive.

I'm running OpenSuSE Leap 15.3, Zoneminder 1.36.25, and Wyze Cam v3 4.36.9.139 firmware.

[mrlt8]

Thanks for the data point! I did notice some delay with the RTSP firmware when audio was enabled, so I'll be taking another look at the audio.

[mrlt8]

I think I found the issue and pushed some updates to the ffmpeg command on the dev branch which should reduce the delay.

Would appreciate it if someone could test it out and see if it helps.

[JamHeHe]

Thanks! I should be able to find some time to get it in place today. I'll let you know how it goes.

[JamHeHe]

I pulled the dev branch and enabled audio. My results were that audio was still not synced, and one of the
2 cameras actually wouldn't record video. Went back to the main branch and things are back to normal. I have other things to pay attention to right now, but I will try again in the near future just to make sure I didn't do something wrong.

[mrlt8]

hmm, I'm unable to replicate the out of sync/delay on v4.36.9.139.. My rtsp stream is about 1/2 second off from the Wyze App.

Can you try rebooting the camera?

As for recording, there were some changes in the last release which will use the mov container if the camera is sending PCM audio which isn't compatible with the mp4 container.

[JamHeHe]

Thanks again. I'm stepping back a bit on the audio issue right now as there's something else going on. I'm having a hard time troubleshooting it, but need to square it away first.

Basically, at dusk and dawn I end up with something tied to the video driving my system into high IO Wait, and the whole system becomes unresponsive. I'm going to systematically work through the cameras Night Vision settings and see if anything hits.

Once I can get this squared away I'll go back to the audio piece.

Thanks for your responsiveness, and you're putting this together in the first place. Greatly appreciate your effort!

[JamHeHe]

I have stabilized my system. Turns out it was starved for memory. Upgraded the memory, and everything is much happier. Start with the simple stuff is the moral I guess.

With that, I gave the dev branch another go. It's definitely not happy. The audio is still out of sync, and it frequently causes the video into zoneminder to stop all together or break up at times. I used VLC to test outside of zoneminder and experienced similar issues there. Tried it both without and with the AAC switch.

As you aren't experiencing these issues, could it be tied to the ffmpeg version, of configuration, or something along those lines? I'm on 4.4 with my opensuse 15.3 fully patched up.

[mrlt8]

The container is using ffmpeg-for-homebridge for the default images and FFmpeg-Builds for the images with hardware acceleration enabled, so we should be on the same ffmpeg build inside the container.

When you say out of sync, does the video also fall behind (compared to video only mode) or is it just the audio that's a few seconds behind? Could you also check the audio in the wyze app to make sure everything is good in there?

Do you have any extra options in your docker-compose? Could you try using the minimal config and see if there is a delay in VLC:

docker run \
  -e [email protected] \
  -e WYZE_PASSWORD=password \
  -p 8554:8554 \
  mrlt8/wyze-bridge:latest

[JamHeHe]

I've been poking at it for a few days with no progress made I've tried it with the really simple config as you reference with no change.

What I've noticed is that the logs in Zoneminder show pkt.dts errors. I'm attaching a screenshot of the errors. I've been poking around the Zoneminder forums and see references to others having the same issue with certain cameras, but no fixes.

[mrlt8]

That's most likely caused by the raw h264 from the camera. Might be able to tweak some setting in the ffmpeg command to fix some of them but it may require re-encoding the video.

[tmeekes]

Interestingly (and I'm not sure what's changed) I'm noticing that my audio stream has a more regular delay now (a half second or so). Things seem to be working better... the extreme audio delays seem like they are gone now.

I should add that I am now making more regular restarts on the related containers now... so that might also be helping to hide any possible delays that build over time.

[mrlt8]

@tmeekes Is that with the November firmware?

Still playing around with the buffer/sleep interval, but would appreciate some feedback with the changes on the dev branch.

Not sure if it's just the camera/firmware I'm testing with, but I've noticed that the audio from the bridge occasionally comes a fraction of a second ahead of the wyze app.

I've also noticed that it seems like the audio and video almost want to stay in sync, and that the video and audio will both lag equally when they fall behind.

[mrlt8]

Thanks @carlosnasillo! Any noticeable hit on the CPU?

[WilldabeastHA]

@mrlt8 - getting close!

I updated to the latest Dev around 3pm EST yesterday. it was working great for hours maybe 5-6 hours, after I went to bed this morning around 4-5 am the lag in the video rendered the streams unusable.

I'm going to reboot and see if i can view the logs when the stream becomes unusable.

here is some of the log:

[audio] out of sync
[office] WARNING: clear buffer
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[audio] out of sync
[office] WARNING: Audio pipe closed
[office] Stream stopped
[WyzeBridge] ❌ '/office' stream is down
[WyzeBridge] 📕 Client stopped reading from office
[WyzeBridge] ❌ '/office' stream is down

[mrlt8]

@WilldabeastHA can you try the latest dev container?

[mrlt8]

@carlosnasillo @WilldabeastHA @Albuca Any comments or feedback for the latest dev build?

[carlosnasillo]

@mrlt8 sorry for the radio silence, I am out of town without SSH access. I'll be able to post an update on Sunday once I'm back and having been able to run it for a day 👍🏼

[carlosnasillo]

Ok im running

wyze-bridge  | 🚀 DOCKER-WYZE-BRIDGE v2.6.0 AARCH64 DEV BUILD [2024-01-05t15:02:36.930z] 9bd50ea

So far (<10 m) nothing untoward observed. Only this line in the log that I have not seen before, but behaviour seems OK

wyze-bridge  | [living-room] [audio] out of sync gap=10.615001916885376
wyze-bridge  | [living-room] WARNING: clear buffer
wyze-bridge  | [WyzeBridge] 📖 New client reading from poolside
wyze-bridge  | [WyzeBridge] ✅ '/poolside stream is UP! (3/3)
wyze-bridge  | [WyzeBridge] 📖 New client reading from poolside

Will report tomorrow or earlier if I notice something is off 💪

[carlosnasillo]

@mrlt8 been running it for a day, looks really solid, have not noticed any issues its pretty much bang on. The audio is still ever so slightly ahead of the video (~1s-5s max) but really, thats OK for me. To summarise;

The good:

• A (~1s) / V (~3s) sync holds
• Cameras do not seem to disconnect / retry as often
• Video quality is good

The not-so-good:

• Nothing so far 🥳 💛

The worth mentioning:

• I do notice some cameras timestamp (they have been synced recently) are substantially delayed to others, most of the cameras are within 5s timestamps of each-other, but one of them seems to drag behind as much as 15-20s of realtime. In fairness, I had noticed this issue before with other builds too. Is not an issue for me as everything else holds true. It does not appear to be something that worsens overtime.

Thanks so much @mrlt8 really appreciate all the effort and patience you put into getting to the root of this issue and making WyzeBridge useable again for those of us it affected ❤️ -- let me know if there is anything else you'd like me to test out.

[mrlt8]

Thanks for the feedback @carlosnasillo!

The audio is still ever so slightly ahead of the video (~1s-5s max)

I've noticed this as well, we could potentially delay it further to match the video, but that would push it further behind realtime... 🤔

... one of them seems to drag behind as much as 15-20s of realtime.

Is that just the timestamp or is the actual video/audio delayed that amount? And how are you viewing the stream? I've noticed that the RTSP stream can sometimes drag behind in VLC but will go back to realtime when if re-playing the stream in VLC. WebRTC seems to stay as realtime as possible and has minimal latency.

[WilldabeastHA]

@mrlt8 hi! Sorry for the delay - just getting back from holiday break with the kiddos.

I've tested a more recent dev version, one from a few days ago, 2024-01-02 10:19:20. I can confirm that it seems like the camera footage is 3-5 sec behind the audio. I haven't done much testing, but I did ensure that the v3 camera has a very strong wifi connection to eliminate that as a variable.

let me know if there is anything specific I can provide or test for you.

Thanks again!

[mrlt8]

Thanks @WilldabeastHA! Other the fast audio, are some of your previous issues resolved in the newer build?

[WilldabeastHA]

@mrlt8 - the steam over time seems to remain stable. I haven't noticed the stream becoming unusable. I will monitor this over the next 48 hours and report back.

[mrlt8]

@WilldabeastHA @carlosnasillo Thanks again for the feedback.
I've made a couple of tweaks to the audio sync and ffmpeg command to reduce the buffer.
Would appreciate it if you could see if these changes help!

[carlosnasillo]

@mrlt8 ok just installed the latest will update in <24h 💪

[carlosnasillo]

Been running for a while. Looking great. Tighter A/V sync, low latencies, no disconnects.

Awesome job @mrlt8, thanks a million! 💪 🎖️ 🙏

[1fastt2]

I'd love to be able to test this, is there a way to load the dev branch on HA? I've had audio sync issues in what seems like forever now.

[mrlt8]

Awesome! Thank you again @carlosnasillo will merge this into the main release!

@1fastt2 you can test the dev images by using the edge repo: https://github.com/mrlt8/edge-repo

[theBasher91]

I still have the issue on 2.7.0. Are any specific flags/envVariables supposed to be changed to resolve the delay on this version?

[p2kdev]

@theBasher91 I switched to scrypted & it supports wyze api officially. So everything works flawless including audio. Give it a go.

[Shadester]

I tried using scrypted with the wyze plugin. Doesn't work well at all. No video stream.

[p2kdev]

@Shadester That’s weird. Make sure you are using scrypted docker. It works so well for me that it feels like native homekit camera. Also as per the scrypted dev, this is officially supported from wyze I think.