Support for video feeds

[dxdc]

It seems like "Microphone" and "Speaker" types are supported, but not cameras. Would it be possible to enable support? It would be terrific to be able to pull a snapshot or 5 sec video feed, for example, for upload/analysis. Not sure how easy this is.

I'm also sharing some example characteristics data from one of my actual cameras, but it didn't work with some kind of default camera set up either using node-red-contrib-homekit-bridged either.

      {
          "iid": 11,
          "type": "110",
          "characteristics": [
            {
              "iid": 12,
              "type": "114",
              "perms": [
                "pr"
              ],
              "format": "tlv8",
              "value": "XXX",
              "description": "Supported Video Stream Configuration"
            },
            {
              "iid": 13,
              "type": "115",
              "perms": [
                "pr"
              ],
              "format": "tlv8",
              "value": "XXX",
              "description": "Supported Audio Stream Configuration"
            },
            {
              "iid": 14,
              "type": "116",
              "perms": [
                "pr"
              ],
              "format": "tlv8",
              "value": "AgEA",
              "description": "Supported RTP Configuration"
            },
            {
              "iid": 15,
              "type": "117",
              "perms": [
                "pr",
                "pw"
              ],
              "format": "tlv8",
              "value": "XXX=",
              "description": "Selected RTP Stream Configuration"
            },
            {
              "iid": 16,
              "type": "120",
              "perms": [
                "pr",
                "ev"
              ],
              "format": "tlv8",
              "value": "AQEA",
              "description": "Streaming Status"
            },
            {
              "iid": 17,
              "type": "118",
              "perms": [
                "pr",
                "pw"
              ],
              "format": "tlv8",
              "value": "AgEC",
              "description": "Setup Endpoints"
            },
            {
              "iid": 18,
              "type": "B0",
              "perms": [
                "pr",
                "pw",
                "ev"
              ],
              "format": "uint8",
              "value": 1,
              "description": "Active",
              "valid-values": [
                0,
                1
              ],
              "maxValue": 1,
              "minValue": 0
            }
          ]
        },

[dxdc]

Thanks for publishing @NorthernMan54. Do you know how to take snapshots or video feeds from these nodes?

[NorthernMan54]

I haven't played with directly accessing the camera's in node red, but am using this to grab 60 seconds of video and upload to google-photos when motion is detected. My goal was to implement some of the features from the Ring Doorbell ( video clips ) without having to pay a subscription fee.

For taking snapshots, I believe you can trigger Homebridge-camera-ffmpeg to take a snapshot and you just need to send the correct message to the node. And I thought I had it also responding with the snapshot image as well. But had abandoned that approach and went with this for my use case.

I'm using Wyze Cameras and they are sluggish to start a feed or take a snapshot so I ended up having the camera cache 60 seconds of video on a RAM drive on my RPI. Then I modified Homebridge-camera-ffmpeg to use that as the snapshot source. And have node-red grab 60 seconds of video starting 20 seconds before the motion sensor is triggered. ( A time machine ).

https://github.com/NorthernMan54/node-red-contrib-homebridge-automation/wiki/Ring-Doorbell-replacement

[dxdc]

That is super cool @NorthernMan54 . I've been thinking about getting some Wyze cams and flashing them with the RTSP firmware to do this. Can you prevent them from uploading to the cloud?

The flow you've laid out here seems a bit different than what you have on your wiki, but maybe the gist is the same. The use of the RAM drive is very clever.

It would be really amazing to upload a snapshot to something like Amazon Rekognition and have it identify people. You could train it to identify your mailman, UPS driver, etc. Just a thought :)

My goal was to implement some of the features from the Ring Doorbell ( video clips ) without having to pay a subscription fee.

Did you have any luck with this part? This is what I'm trying to do more or less, but I'm just not sure how to (or, if it's possible?) connect to the HB Ring feed. I imagine it may also block multiple simultaneous connections?

I don't care about the subscription part, I'm fine with that, I just want to be able to analyze the images and snapshots and maybe make some backup copies as well in an automated way.

[dxdc]

Btw, I also thought it would be fun - one can access the Ring Microphone accessory - to play a sound through there. If you know how to do any of this would be very curious!

[NorthernMan54]

The wiki is just an early view into the final solution. I have been tweaking it over a few months and the screenshot was the latest.

With my wyze cams I'm just ignoring the cloud features ( but occasionally take a look for admin things ).

For recognition I have been using the google photo's recognition engine, and it has been pretty good, but looking at amazon Rekognition the marketing looks impressive. Not sure though how it compares against google photo's.

I had returned my Ring door bell when the free period was expiring, as the comments I got at home on the value was limited. And never looked into the device any further. But I have heard from the Ring Plugin developer that there are rate limits on the Ring API that limit developing more advanced options on the platform. A bit of walled garden.

And changed my efforts to low cost cameras that support direct access, and settled on the Wyze. I had briefly played with the dafang hacks replacement firmware for them, but found that the night time video quality was slightly slower. So have been working towards mimicking the behaviour of the ring, and in particular the video capture capability.

[dxdc]

I may give the Wyze cams a shot. It bothers me a little bit that they are just white-labeled Xiaomi cameras with somewhat loose privacy restrictions on what is being uploaded... but I can't tell if the replacement firmware (with RTSP) disables the cloud or not. They were on special at Home Depot for $13 a few months back.

How did you set up your RAM drive with ffmpeg access? I'm wondering if the homebridge-ring developer would be open to offering that as a config parameter. The ideas could work really similarly in that case. Thanks for your help!

[NorthernMan54]

@dxdc If you install the latest beta, you should now be able to collect a snapshot from HomeKit Camera

To install the beta

npm install node-red-contrib-homebridge-automation@beta

Then place a hs-status node configured to be "Camera control" node for one of your camera, and send it anything, it will output the snapshot as a base64 encoded string. Not sure how useful that will be, but it is a start

[dxdc]

@NorthernMan54 yes, that sounds perfect! I am not yet able to test the beta though b/c I need the pin# bit to make things work on my end. Too many HB instances w/ different pins.

One other thing. For some reason, I see 2 ring cameras per Ring device. This results in 2 "Garage - Camera" type nodes that can't be called independently. I'm wondering if your pending deviceId change would fix this one?

They are both under the same aid with the same "110" type, but different iid. I have no idea why it's showing 2 cameras there. Let me know if you want a more complete dump of this part.

[NorthernMan54]

I saw that as well when I was playing with this, as part of the beta will need to refine the camera discovery a bit

[NorthernMan54]

If you update to the latest, v0.0.76, I tweaked the hb-control node to allow it to output the snapshot image from a camera control node. And have started looking into Amazon’s rekognition engine as well.

This is my poc flow for amazon Rekognition.

[NorthernMan54]

This is the node-red flow

[{"id":"972a3250.146048","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"42cdd4f0.7e8f1c","type":"hb-control","z":"972a3250.146048","name":"Eufy Porch - Dummy","Homebridge":"Indoor Cam 2K","Manufacturer":"Eufy","Service":"Camera Control","device":"Indoor Cam 2K65:7F:96:9B:5F:C2EufyEufy Porch - Dummy00000111","conf":"a96dace6.d5e138","outputs":1,"x":420,"y":100,"wires":[["b55c4bb1.6372a8","cb61133e.5c3fa8"]]},{"id":"9156ee66.96b1c8","type":"debug","z":"972a3250.146048","name":"","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":660,"y":260,"wires":[]},{"id":"b7f786c6.ffc0d","type":"inject","z":"972a3250.146048","name":"Test Button","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":60,"wires":[["42cdd4f0.7e8f1c"]]},{"id":"d0c4187a.3844d8","type":"AWS Rekognition","z":"972a3250.146048","aws":"26bcea45.6e7d36","operation":"DetectText","SourceImage":"","TargetImage":"","CollectionId":"","Input":"","Output":"","Name":"","Settings":"","RoleArn":"","FaceIds":"","Image":"","Id":"","JobId":"","FaceId":"","Video":"","name":"","x":410,"y":260,"wires":[["9156ee66.96b1c8"],[]]},{"id":"b55c4bb1.6372a8","type":"image","z":"972a3250.146048","name":"","width":160,"data":"payload","dataType":"msg","thumbnail":false,"active":true,"pass":false,"outputs":0,"x":660,"y":100,"wires":[]},{"id":"cb61133e.5c3fa8","type":"function","z":"972a3250.146048","name":"Convert Image","func":"msg.Image={Bytes:msg.payload};\nmsg.MinConfidence = 90;\nreturn msg;","outputs":1,"noerr":0,"x":160,"y":260,"wires":[["d0c4187a.3844d8","a3c89e5b.4928c","c63393c2.a7c62"]]},{"id":"c63393c2.a7c62","type":"AWS Rekognition","z":"972a3250.146048","aws":"26bcea45.6e7d36","operation":"DetectLabels","SourceImage":"","TargetImage":"","CollectionId":"","Input":"","Output":"","Name":"","Settings":"","RoleArn":"","FaceIds":"","Image":"","Id":"","JobId":"","FaceId":"","Video":"","name":"","x":410,"y":380,"wires":[["d4ce46f5.8a992","5d503dc1.9b3d24"],[]]},{"id":"d4ce46f5.8a992","type":"debug","z":"972a3250.146048","name":"","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":660,"y":380,"wires":[]},{"id":"a3c89e5b.4928c","type":"AWS Rekognition","z":"972a3250.146048","aws":"26bcea45.6e7d36","operation":"DetectFaces","SourceImage":"","TargetImage":"","CollectionId":"","Input":"","Output":"","Name":"","Settings":"","RoleArn":"","FaceIds":"","Image":"","Id":"","JobId":"","FaceId":"","Video":"","name":"","x":410,"y":320,"wires":[["cf2db6c4.08c09"],[]]},{"id":"cf2db6c4.08c09","type":"debug","z":"972a3250.146048","name":"","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":660,"y":320,"wires":[]},{"id":"61115339.cce764","type":"hb-event","z":"972a3250.146048","name":"Eufy Porch - Dummy","Homebridge":"Indoor Cam 2K","Manufacturer":"Eufy","Service":"Motion Sensor","device":"Indoor Cam 2K65:7F:96:9B:5F:C2EufyEufy Porch - Dummy00000085","conf":"a96dace6.d5e138","x":150,"y":100,"wires":[["42cdd4f0.7e8f1c"]]},{"id":"5d503dc1.9b3d24","type":"split","z":"972a3250.146048","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":150,"y":480,"wires":[["59f91290.df262c"]]},{"id":"59f91290.df262c","type":"split","z":"972a3250.146048","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":290,"y":480,"wires":[["8ef22f79.7ed138"]]},{"id":"8ef22f79.7ed138","type":"switch","z":"972a3250.146048","name":"","property":"payload","propertyType":"msg","rules":[{"t":"jsonata_exp","v":"$contains(payload.Name, 'Vehicle')\t","vt":"jsonata"},{"t":"nnull"}],"checkall":"false","repair":false,"outputs":2,"x":430,"y":480,"wires":[["7fb4cca3.e1187c"],["71d826a5.651128"]]},{"id":"7fb4cca3.e1187c","type":"debug","z":"972a3250.146048","name":"","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":620,"y":460,"wires":[]},{"id":"71d826a5.651128","type":"debug","z":"972a3250.146048","name":"","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","x":620,"y":520,"wires":[]},{"id":"a96dace6.d5e138","type":"hb-conf","z":"","username":"031-45-154"},{"id":"26bcea45.6e7d36","type":"amazon config","z":"","name":"AWS","region":"us-east-1","proxyRequired":false,"proxy":""}]

[dxdc]

wow, this looks awesome! I haven't updated to the latest because of the issue I have with multiple PIN's on different hb instances. has that been resolved? If so, will be glad to test this out!

Btw, this could be a cool additional flow - if you have a Sonos -
https://flows.nodered.org/node/node-red-contrib-sonospollytts

It will announce custom text over your Sonos (text to speech). You could try something like having it say "Vehicle at the back door" out loud between certain hours for ex. Nicer than some of the built in chimes that just play a noise.

I've used it for similar things.