[feat] Add API to configure associations to a final state #186

LordMike · 2021-01-09T22:51:55Z

Is your feature request related to a problem? Please describe.
I maintain a desired state configuration for my z-wave items besides OZW/Z2M/ZJS2M .. this configures my devices (we've discussed templated configurations before) and configures HASS with desired discovery documents. What I haven't currently got, is associations between devices.

Describe the solution you'd like
I'd like an API, where I can configure the associations for a device in one go.

For example, I could post this message to MQTT:

> zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/api/setAssociations/set
{
  "node": 10,
  // Maybe: A property that specifies that any association not in this doc, is removed
  "clearAssociations": false,
  // Maybe: Make it possible to merely add associations, without specifying /all/ desired associations - ie. make this an  "addAssocation" API.
  "appendOnly": false,
  "associations": {
    // Configure group '5' to send to 'node 11 endpoint 1' and 'node 12 endpoint 1'
    // All other nodes set in group 5, should be removed
    "5": ["11.1", "12.1"],
    // Remove all associations for group '7'
    "7": null
    // All other groups are not specified, and aren't touched.
  }
}

Describe alternatives you've considered
I could probably cook something up that does a series of "getAssociation" and add/removes.. But I imagine this could be a precursors to something that's part of ZJS2M.

Additional context
N/A

The text was updated successfully, but these errors were encountered:

LordMike · 2021-01-09T22:53:51Z

I also think that this API could help reduce chatter on the z-wave networks.

It's my impression that the defaults of many devices, is to send all updates to node 1 (the controller).. This makes the networks "just work", but also introduces a lot of potentially unnecessary messages.

Removing the node 1 from many of the groups would go a long way to reducing this. :)

robertsLando · 2021-01-11T09:44:42Z

Removing the node 1 from many of the groups would go a long way to reducing this. :)

@LordMike I don't get this. Without the lifeline association devices will not work, it's used to push changes to controller

LordMike · 2021-01-11T10:26:14Z

Ah yes - but if you don't need the updates of changes. This could be a way to achieve this. A device default could also be the controller in multiple other groups to "make it just work". For my power plugs, I think this is the case - they report three values (power usage W, amperage A and some other thing) in three messages.. Iirc, these came over the lifeline group with no way to choose not to get them.

Additionally, the controller isn't always no. 1. :)

robertsLando · 2021-01-11T11:09:05Z

@LordMike I dunno about the controller id but AFAIK by default all devices need lifeline and I dunno all the possible downsides about not setting this. @AlCalzone ?

AlCalzone · 2021-01-11T11:37:17Z

Additionally, the controller isn't always no. 1. :)

If you look here (and the following lines) you will see that this is taken care of:
https://github.com/zwave-js/node-zwave-js/blob/624fe7820a253c99dbf3e51a4ed7c2bd893ba74d/packages/zwave-js/src/lib/commandclass/AssociationCC.ts#L363

Removing the node 1 from many of the groups would go a long way to reducing this. :)

The first thing when we do this is that 99.99% of our users will complain that the UI/Application they use falls out of sync with the devices. I think the specifications for ZWave+ V2 even require that the devices report all their updates through the lifeline, so they will probably not even get polled anymore after a set. Which means you'd be flying blind without the feedback.

AlCalzone · 2021-01-11T11:41:39Z

I do agree that bulk-configuring associations makes sense, but you can already do that in two commands:

https://github.com/zwave-js/node-zwave-js/blob/624fe7820a253c99dbf3e51a4ed7c2bd893ba74d/packages/zwave-js/src/lib/controller/Controller.ts#L1630-L1634

https://github.com/zwave-js/node-zwave-js/blob/624fe7820a253c99dbf3e51a4ed7c2bd893ba74d/packages/zwave-js/src/lib/controller/Controller.ts#L1751-L1755

We could extend removeAssociations to remove all of them, but as I wrote above, you probably don't want to do that.

In general, getting Associations configured correctly is very tricky, so it is best to let the controller take care of that for you where it can. The following points must all be considered:

Does the device support Multi Channel Associations?
Does the device support Multi Channel Association CC V3+?
Does it support Multi Channel at all?
Does it have multiple endpoints?
Should the Lifeline be a node or multi channel association?
Are there misconfigured associations that are shadowed somehow?

LordMike · 2021-01-11T19:37:01Z

With those two commands, in addition to getAssociations, I think the library has what it needs (separation of concerns and all that).

Now the request is just for zjs2m to have this feature :)

LordMike · 2021-01-11T19:40:27Z

Removing the node 1 from many of the groups would go a long way to reducing this. :)

The first thing when we do this is that 99.99% of our users will complain that the UI/Application they use falls out of sync with the devices. I think the specifications for ZWave+ V2 even require that the devices report all their updates through the lifeline, so they will probably not even get polled anymore after a set. Which means you'd be flying blind without the feedback.

And I would not, of course, at any point advocate for a function that automatically removes every default configuration there is. removeAssocation should do exactly that, remove the specified associations. A high level API, like for ZJS2M, could provide a desired state API, that changes the device into whatever is needed - while doing as few API calls as possible. :)

I can probably make this API myself, if I can figure out how to call get get/remove/add associations over MQTT.

robertsLando · 2021-01-12T08:05:13Z

@LordMike You can call all zwave functions with: https://zwave-js.github.io/zwavejs2mqtt/#/guide/mqtt?id=zwave-apis

robertsLando · 2021-01-13T13:15:00Z

@LordMike Does that solves your problem?

LordMike · 2021-01-17T00:47:42Z

I was caught up in work the past few days.

It might.

For others interested. This is a direct link to the associations API's @ zwave-js. (It was tricky to find .. the search isn't very helpful).

LordMike · 2021-01-17T01:15:27Z

I can call the getNodes without issue. But getAssociations doesn't seem to work.
The logs tell me the following.

2021-01-17 01:11:18.290 INFO MQTT: Message received on zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/controller/getAssociations/set, '{"data":[5]}'
2021-01-17 01:11:18.291 WARN MQTT: Unknown action received -1 _CLIENTS/ZWAVE_GATEWAY-HomeMQTT/controller/getAssociations/set

I've checked bot the zwave-js docs and the source - both place getAssociations in the controller section.

The node is fully interviewed, and the UI can show the associations for node 5:

LordMike · 2021-01-17T01:19:50Z

Ah. Apparently I was missing an api in my request.

https://github.com/zwave-js/zwavejs2mqtt/blob/cf71d1fa6af252c1cff156de68b13b5d7ce21137/lib/MqttClient.js#L220-L232

LordMike · 2021-01-17T01:20:33Z

Nevermind.. I was reading the events docs.. Needed to scroll down.

LordMike · 2021-01-17T01:33:47Z

So I've been publishing to zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/api/getAssociations/set but I can't quite get it work. The response says the request was sucessful, but the results are always empty.

I found this code, which I think is being called, and it's odd - because this differs from the zwavejs docs. In this code, groupId is required, but adding that (0..25) doesn't provide any results either.

https://github.com/zwave-js/zwavejs2mqtt/blob/8371752b9e9f20f596606aec6d9bfedf3d9b0b5d/lib/ZwaveClient.js#L1370-L1372

I've been publishing {"data":[5]} (and other nodes, all with associations in the UI), and ``{"data":[5,4]}` (and other numbers for the group id).

I even iterated the nodes/groups from the getNodes, and couldn't find a pair that produced anything but []'s.

Other notes:

Why restrict the getAssociations to a single group? .. The API from zwave-js gives you all groups at the same time.. :)
getAssociationGroups is also missing from the allowedApis.

robertsLando · 2021-01-17T10:37:50Z

Why restrict the getAssociations to a single group? .. The API from zwave-js gives you all groups at the same time.. :)

You right, I could extend that to return all associations if groupId is not specified, would it be ok for you?

getAssociationGroups is also missing from the allowedApis.

Because that is actually used in getGroups apis to popolate nodes groups. Would you like that api to be accessible too?

LordMike · 2021-01-17T22:09:12Z

Why restrict the getAssociations to a single group? .. The API from zwave-js gives you all groups at the same time.. :)

You right, I could extend that to return all associations if groupId is not specified, would it be ok for you?

Perfect. Do you know why the results I get is always [] ?

getAssociationGroups is also missing from the allowedApis.

Because that is actually used in getGroups apis to popolate nodes groups. Would you like that api to be accessible too?

I'm not interested in the API, I think.. It was more of a jab against the statement on the docs, saying "You have full access to all zwavejs APIs (and more) by simply using MQTT." ... The list of allowed apis kinda limits that :)

... and f.ex. the extra parameter on getAssociations deviates from the zwave-js docs.

robertsLando · 2021-01-18T11:08:14Z

Perfect. Do you know why the results I get is always [] ?

What are the args passed?

I'm not interested in the API, I think.. It was more of a jab against the statement on the docs, saying "You have full access to all zwavejs APIs (and more) by simply using MQTT." ... The list of allowed apis kinda limits that :)

You absolutely right, in dact that comes from the old zwave2mqtt where I simply called the zwave apis without any checks, now I have encapsulated all zwave apis in other functions to better handle them, I have created this as a reminder: #260

I will look at it once I have more free time to improve docs :)

LordMike · 2021-01-18T18:09:33Z

I provided two numerics, a node id and a group id - I also tried with node id alone. But were plain numbers, in the data property.

Example:

Zjs2mqtt has the data. And I tried requesting it with:

Topic: zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/api/getAssociations/set
{"data":[27,0]}
{"data":[27,1]}
etc.. up to endpoint 5

Each time I get a response, with the following:

{"success":true,"message":"Success zwave api call","result":[],"args":[]}

... curiously, I just checked the debug log on the UI:

So... Node 27 is undefined?

Shouldn't this give an error in the response?
How can it be undefined - seems like a bug.

It is node 27 I'm testing at:

LordMike · 2021-01-18T18:36:26Z

Ooh, the code says this message appears when the nodeId in the args is undefined.. I'm seeing other code saying the input is incorrect.

LordMike · 2021-01-18T18:37:27Z

Yeap. I figured it out.

https://github.com/zwave-js/zwavejs2mqtt/blob/a56f044f4abd24d4bb9937485c6fb3c510e7906c/lib/Gateway.js#L445

The actual input is "args". Not "data". I have no idea where I got "data" from.. :|

LordMike · 2021-01-18T18:50:32Z

I'm getting the associations from that.. So yea, it probably works out in the end.

I would suggest that an error is returned, if the call actually doesn't work.

LordMike · 2021-01-18T23:13:07Z

Hey - just wanted to report back that I'm now using get/add/remove associations to setup my home.

In the pic below, my program took a file I created that maps desired associations, and applies them to the result of getAssociations (to get add and remove). What we see here is partly that apparently the lifeline association for the devices has two associations with the controller.. I tested, and only the 1.0 is needed, so I'm cleaning up. And while doing this, I discovered some pairings that were messed up, where a button would never get a report back on a set it would send.

Maybe one day this could be a visual tool... :)

I actually got a bunch of errors from this (besides, my topics were wrong):

Investigating.

LordMike · 2021-01-18T23:23:48Z

The warning I showed before, comes from here - but even though the method threw an exception, the associations were removed from Z2M.

I've refreshed some of the nodes, and the associations are still gone - so I think it came through.

robertsLando · 2021-01-19T09:21:05Z

The warning I showed before, comes from here - but even though the method threw an exception, the associations were removed from Z2M.

Humm really strang. @AlCalzone Do you know why this hjappens ?

@LordMike COuld you send me a log output of zwavejs when you send the removeAssociation command? So we can see what's going on in the lower level

AlCalzone · 2021-01-19T09:40:05Z

What we see here is partly that apparently the lifeline association for the devices has two associations with the controller..

Which one? Which device?

LordMike · 2021-01-19T13:20:14Z

What we see here is partly that apparently the lifeline association for the devices has two associations with the controller..

Which one? Which device?

ZDB5100 from Logic Group. Apparently, there are two associations to node 1 - one targeting endpoint 0, and one targeting endpoint 1. I've now removed the one targeting endpoint 1, and written to the company asking why it might be there. The device works without this association (I still get updates etc..). It's my impression that having two, just makes for double the traffic.

AlCalzone · 2021-01-19T13:34:49Z

I'm not sure where that came from. Devices don't have associations out of the box. These are assigned by controllers, so maybe OZW or something else added them. But an association to the controller, endpoint 1 seems pointless.
zwave-js only auto-adds node associations to the controller device ("1") or the root endpoint of the controller ("1.0")

LordMike · 2021-01-19T18:36:47Z

I'm not sure where that came from. Devices don't have associations out of the box. These are assigned by controllers, so maybe OZW or something else added them. But an association to the controller, endpoint 1 seems pointless.
zwave-js only auto-adds node associations to the controller device ("1") or the root endpoint of the controller ("1.0")

Oh. Well, they've definitely been added using OZW. Re-including with zwave-js did not yield the 1.1 association.

LordMike · 2021-01-19T18:44:33Z

The warning I showed before, comes from here - but even though the method threw an exception, the associations were removed from Z2M.

Humm really strang. @AlCalzone Do you know why this hjappens ?

@LordMike COuld you send me a log output of zwavejs when you send the removeAssociation command? So we can see what's going on in the lower level

zwavejs2mqtt

2021-01-19 18:39:51.561 INFO MQTT: Message received on zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/api/removeAssociations/set, '{"args":[26,14,[{"nodeId":20,"endpoint":2}]]}'
2021-01-19 18:39:51.565 INFO ZWAVE: Assocaitions: Removing associations from Node 26 Group 14: [ { nodeId: 20, endpoint: 2 }, [length]: 1 ]
2021-01-19 18:39:51.801 WARN ZWAVE: Error while removing associations from 26: Node 26 does not support associations!
2021-01-19 18:39:51.802 INFO ZWAVE: Success zwave api call removeAssociations 
2021-01-19 18:39:51.803 INFO MQTT: Message received on zwavejs2mqtt/_CLIENTS/ZWAVE_GATEWAY-HomeMQTT/api/removeAssociations, '{"success":true,"message":"Success zwave api call","args":[26,14,[{"nodeId":20,"endpoint":2}]]}'

zwave-js

18:39:51.577 SERIAL » 0x010d00131a068e040e00140225a2e8                                    (15 bytes)
18:39:51.578 DRIVER » [Node 026] [REQ] [SendData]
                      │ transmit options: 0x25
                      │ callback id:      162
                      └─[MultiChannelAssociationCCRemove]
                          group id:  14
                          node ids:
                          endpoints: 20:2
18:39:51.580 SERIAL « [ACK]                                                                   (0x06)
18:39:51.587 SERIAL « 0x0104011301e8                                                       (6 bytes)
18:39:51.587 SERIAL » [ACK]                                                                   (0x06)
18:39:51.587 DRIVER « [RES] [SendData]
                        was sent: true
18:39:51.603 SERIAL « 0x01070013a20000024b                                                 (9 bytes)
18:39:51.604 SERIAL » [ACK]                                                                   (0x06)
18:39:51.604 DRIVER « [REQ] [SendData]
                        callback id:     162
                        transmit status: OK
18:39:51.611 SERIAL » 0x010a00131a038e020e25a3fb                                          (12 bytes)
18:39:51.612 DRIVER » [Node 026] [REQ] [SendData]
                      │ transmit options: 0x25
                      │ callback id:      163
                      └─[MultiChannelAssociationCCGet]
                          group id: 14
18:39:51.613 SERIAL « [ACK]                                                                   (0x06)
18:39:51.619 SERIAL « 0x0104011301e8                                                       (6 bytes)
18:39:51.620 SERIAL » [ACK]                                                                   (0x06)
18:39:51.620 DRIVER « [RES] [SendData]
                        was sent: true
18:39:51.738 SERIAL « 0x010e0004001a088e030e0500001b017b                                  (16 bytes)
18:39:51.739 SERIAL » [ACK]                                                                   (0x06)
18:39:51.742 CNTRLR   [Node 026] [~] [Multi Channel Association] maxNodes[14]: 5 => 5     [internal]
18:39:51.742 CNTRLR   [Node 026] [~] [Multi Channel Association] nodeIds[14]:  =>         [internal]
18:39:51.743 CNTRLR   [Node 026] [~] [Multi Channel Association] endpoints[14]: [object O [internal]
                      bject],[object Object] => [object Object]
18:39:51.743 DRIVER « [Node 026] [REQ] [ApplicationCommand]
                      └─[MultiChannelAssociationCCReport]
                          group id:           14
                          maximum # of nodes: 5
                          node ids:
                          endpoints:          27:1
18:39:51.781 SERIAL « 0x01070013a300001058                                                 (9 bytes)
18:39:51.781 SERIAL » [ACK]                                                                   (0x06)
18:39:51.782 DRIVER « [REQ] [SendData]
                        callback id:     163
                        transmit status: OK
18:39:51.797 SERIAL « 0x010e0004001a088e030e0500001b017b                                  (16 bytes)
18:39:51.797 SERIAL » [ACK]                                                                   (0x06)
18:39:51.798 CNTRLR   [Node 026] [~] [Multi Channel Association] maxNodes[14]: 5 => 5     [internal]
18:39:51.798 CNTRLR   [Node 026] [~] [Multi Channel Association] nodeIds[14]:  =>         [internal]
18:39:51.798 CNTRLR   [Node 026] [~] [Multi Channel Association] endpoints[14]: [object O [internal]
                      bject] => [object Object]
18:39:51.798 DRIVER « [Node 026] [REQ] [ApplicationCommand]
                      └─[MultiChannelAssociationCCReport]
                          group id:           14
                          maximum # of nodes: 5
                          node ids:
                          endpoints:          27:1

@AlCalzone notice the [object Object] => [object Object] line.. bug? :)

@robertsLando here are the logs. It doesn't seem like zwave-js reports anything odd, perhaps except that the association report comes in twice?

AlCalzone · 2021-01-19T21:57:05Z

It does look like 20:2 was successfully removed. The duplicate report is a result of polling after the command. I'm working on a more general solution for that.
Yeah the [object Object] logging is unfortunate

AlCalzone · 2021-01-19T22:01:16Z

Error while removing associations from 26: Node 26 does not support associations!

And that is totally a bug. Seems like your device only supports Multi Channel Associations, but that should not raise an error.

LordMike · 2021-01-20T20:51:17Z

Should we close this, now that zwave-js has an issue?

AlCalzone · 2021-01-20T21:23:23Z

fixed, the logic had a little bug when trying to remove only multi-channel associations

LordMike added the enhancement New feature or request label Jan 9, 2021

LordMike assigned robertsLando Jan 9, 2021

AlCalzone mentioned this issue Jan 19, 2021

Value change logging for objects prints [object Object] zwave-js/node-zwave-js#1451

Closed

AlCalzone mentioned this issue Jan 19, 2021

Controller methods for managing associations throw when Association CC is not supported but MCA is zwave-js/node-zwave-js#1452

Closed

LordMike closed this as completed Jan 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feat] Add API to configure associations to a final state #186

[feat] Add API to configure associations to a final state #186

LordMike commented Jan 9, 2021

LordMike commented Jan 9, 2021

robertsLando commented Jan 11, 2021

LordMike commented Jan 11, 2021

robertsLando commented Jan 11, 2021

AlCalzone commented Jan 11, 2021 •

edited

Loading

AlCalzone commented Jan 11, 2021

LordMike commented Jan 11, 2021

LordMike commented Jan 11, 2021

robertsLando commented Jan 12, 2021

robertsLando commented Jan 13, 2021

LordMike commented Jan 17, 2021 •

edited

Loading

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021 •

edited

Loading

robertsLando commented Jan 17, 2021

LordMike commented Jan 17, 2021 •

edited

Loading

robertsLando commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021 •

edited

Loading

LordMike commented Jan 18, 2021 •

edited

Loading

robertsLando commented Jan 19, 2021

AlCalzone commented Jan 19, 2021

LordMike commented Jan 19, 2021

AlCalzone commented Jan 19, 2021 •

edited

Loading

LordMike commented Jan 19, 2021

LordMike commented Jan 19, 2021

AlCalzone commented Jan 19, 2021 •

edited

Loading

AlCalzone commented Jan 19, 2021

LordMike commented Jan 20, 2021

AlCalzone commented Jan 20, 2021

[feat] Add API to configure associations to a final state #186

[feat] Add API to configure associations to a final state #186

Comments

LordMike commented Jan 9, 2021

LordMike commented Jan 9, 2021

robertsLando commented Jan 11, 2021

LordMike commented Jan 11, 2021

robertsLando commented Jan 11, 2021

AlCalzone commented Jan 11, 2021 • edited Loading

AlCalzone commented Jan 11, 2021

LordMike commented Jan 11, 2021

LordMike commented Jan 11, 2021

robertsLando commented Jan 12, 2021

robertsLando commented Jan 13, 2021

LordMike commented Jan 17, 2021 • edited Loading

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021

LordMike commented Jan 17, 2021 • edited Loading

robertsLando commented Jan 17, 2021

LordMike commented Jan 17, 2021 • edited Loading

robertsLando commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021

LordMike commented Jan 18, 2021 • edited Loading

LordMike commented Jan 18, 2021 • edited Loading

robertsLando commented Jan 19, 2021

AlCalzone commented Jan 19, 2021

LordMike commented Jan 19, 2021

AlCalzone commented Jan 19, 2021 • edited Loading

LordMike commented Jan 19, 2021

LordMike commented Jan 19, 2021

zwavejs2mqtt

zwave-js

AlCalzone commented Jan 19, 2021 • edited Loading

AlCalzone commented Jan 19, 2021

LordMike commented Jan 20, 2021

AlCalzone commented Jan 20, 2021

AlCalzone commented Jan 11, 2021 •

edited

Loading

LordMike commented Jan 17, 2021 •

edited

Loading

LordMike commented Jan 17, 2021 •

edited

Loading

LordMike commented Jan 17, 2021 •

edited

Loading

LordMike commented Jan 18, 2021 •

edited

Loading

LordMike commented Jan 18, 2021 •

edited

Loading

AlCalzone commented Jan 19, 2021 •

edited

Loading

AlCalzone commented Jan 19, 2021 •

edited

Loading