Welcome to firescrew Discussions!

[chiefMarlin]

👋 Welcome!

We’re using Discussions as a place to connect with other members of our community. We hope that you:

• Ask questions you’re wondering about.
• Share ideas.
• Engage with other community members.
• Welcome others and are open-minded. Remember that this is a community we
  build together 💪.

To get started, comment below with an introduction of yourself and tell us about what you do with this community.

[wehbs]

Does firescrew support CoreML through Docker for object detection?

[chiefMarlin]

I think this can be broken down into 2 parts, is it possible to support coreml adapter - yes, as long as firescrew can connect to it over network, its supported.
Would CoreML work on docker on a mac, i am pretty sure it will not, as docker on mac runs within qemu since mac is not linux and does not support namespaces among the few things required for docker.

To use coreml it would have to run outside of docker (effectively on host mac) and firescrew would have to connect to it using "networkObjectDetectServer": "", parameter.
Im playing with this right now and will let you know if i get anywhere with it.

[chiefMarlin]

So i got the coreml adapter working, its about as fast as using a coral tpu, 30ms with yolov8 model (which is pretty fast). Im still sorting out the box dimensions but i think it should be ready for tomorrows release.

[chiefMarlin]

I have just pushed core with CoreML adapter, its working pretty well on m1 mini, with yolov8s model im getting very good performance with about 20ms avg inference.
What interesting is that i ran 5 instances of recognition on using same adapter simultaneously and inference time only went up to 25ms, meaning you can easily run lots of cameras.

[wehbs]

Very cool! I should run through these steps https://github.com/8ff/firescrew#running-yolov8s-with-coreml-mac-only? Even if running firescrew in Docker container yes?

[chiefMarlin]

Exactly, as long as your networkObjectDetectServer param is set, it will use that

[wehbs]

That works really well! AVG 28ms. But it's because I have NVR software already running on the Mac.

Looks like now the most cpu intensive thing that is occurring is the re encoding to mp4. Is it possible to just record the stream in its original format h264 for example? I notice it captures in .TS but it's unclear to me if this is a necessary technical thing. If I don't re encode chrome won't play the TS video.

[chiefMarlin]

Thats awesome!
So heres a brief on how mp4 reencode works.
All encoding takes resources, so to avoid that, firescrew takes raw hi res stream (typically h264) and stores it into a .ts file.
This process uses almost no cpu and is very quick, however there is a downside, not all browsers will play it.
It works fine on Safari using the WebUI however Chrome doesnt like it.
Thats why there is the "recodeTsToMp4": true option in the config, this will start a background process once a .ts file is created and in the background it will use ffmpeg to create an .mp4 file.
WebUI knows about the above and will check if .mp4 file exists, if it does it will serve that, if it doesnt, it will serve .ts instead.
So to summarize, if you are ok with storing raw stream data and use Safari then you can disable the reencode.

[wehbs]

That's excellent. Just tested, no CPU spike at all in container and video was ready to view almost immediately after snapshot was generated. Amazing work, thanks! I wonder if the same could work with H265 video. I believe Safari has native support. I'm not sure about Chrome.

[chiefMarlin]

There is only one way to find out!
In the meantime i added a warning

if stream.CodecName != "h264" {
      Log("warning", fmt.Sprintf("OnlyRemuxMp4 is enabled but the stream codec is not h264 or h265. Your videos may not play in WebUI.Codec: %s", stream.CodecName))
}

so that should atleast let people know about this.

Thanks for testing it out!

[chiefMarlin]

fyi, i just pushed an update to coreml python adapter, it has a bug fix to avoid crashing

[wehbs]

Got the update. So I just tried my cameras 4K H265 stream and it works great in playing the video back on Chrome! But no video playback on Safari, which means mobile Safari is also borked. But that's odd to me as I see online Safari should be the one having no issues with H265.

[chiefMarlin]

Thanks for the update

[chiefMarlin]

To add to that, i think having a classic webUi that has events is good for some cases but i think it would be much more useful to get away from this classic model and instead having something that allows you to be on the go and get these events on your mobile device in a friendly form.
That is why i am aiming towards structuring these events to be delivered over messaging platform (for example slack) in a form of a message that would should you detailed summary of the event, an image and clicking this image would automatically play the clip.
This is still in the works and would be in addition to everything that exists.
If you have any suggestions or ideas, im all ears!

[wehbs]

I totally agree with this. I think the most important thing would be rich notifications. A snapshot image zoomed in on the prominent object that was detected in the event would be perfect. Would be cool if the Signal messenger app could be used to send the snapshot and clip there. Maybe this project could be used https://github.com/bbernhard/signal-cli-rest-api.

The option to 24/7 record the lower res motion stream would be a neat feature to.

[chiefMarlin]

24/7 lo res recording would be fairly easy to implement i think and useful in some cases, ive added it to the bucket list.
About signal, ive written some tool to send slack messages a few years back - and was really excited to use that for misc daily notifications. But in practise it turned to be unreliable as some of the messages would never make it to the group chat, maybe this improved since then.
Ive been using slack to replace that since and its been solid 100% solid and easy to use with just a url as opposed to tokens (with signal)