Z-wave Network Failing to Communicate After Controller Upgrade to SDK 7.19.3

[asayler]

Is your problem within Home Assistant (Core or Z-Wave JS Integration)?

YES, BUT a Home Assistant developer has told me to come here

Is your problem within Z-Wave JS UI (formerly ZwaveJS2MQTT)?

NO, my problem is NOT within Z-Wave JS UI

Checklist

• I have checked the troubleshooting section and my problem is not described there.

• I have read the changelog and my problem was not mentioned there.

Describe the bug

What causes the bug?

Every 6-12 hours, the Zwave controller stops communicating with most (all?) nodes. It seems like some commands are still coming in, but no outbound communication works, and eventually no inbound communication works either. If I leave it long enough, the controller itself seems to lock up and Zwave JS US returns an "Controller is unable to transmit" error. It seems this ultimly locks up HA as well, trigger the watchdog to restart it (although this restart does not resolve the error since the controller is not power cycled in the process).

What do you observe?

I recently upgrade my Zooz ZAC93 controller to the latest firmware 1.20 (SDK 7.19.3). Ever since, the controller seems to cease communicating with nodes every 6-12 hours, requiring a full power cycle to restore it. It's possible this is an issue with the new firmware, but I wanted to check here first to see if it might have something to do with the new SDK version and Zwave-JS compatibility. You can view a side effect of the error in the following graph, where Awake nodes tick up indefinitely until the controller is rebooted:

What did you expect to happen?

The controller to continues transmitting without error.

Steps to reproduce the behavior:

1. Reboot controller
2. Wait 6-12 hours
3. Observe behavior

Device information

Manufacturer: Zooz
Model name: ZAC93
Node ID in your network: 001
FW: 1.2
SDK: 7.19.3

How are you using node-zwave-js?

• zwave-js-ui (formerly zwavejs2mqtt) Docker image (latest)
• zwave-js-ui (formerly zwavejs2mqtt) Docker image (dev)
• zwave-js-ui (formerly zwavejs2mqtt) Docker manually built (please specify branches)
• ioBroker.zwave2 adapter (please specify version)
• HomeAssistant zwave_js integration (please specify version)
• pkg
• node-red-contrib-zwave-js (please specify version, double click node to find out)
• Manually built from GitHub (please specify branch)
• Other (please describe)

Which branches or versions?

I am running the Zwave JS UI add-on, version 1.16.0. I belive this was happening on the prior add-on version as well. This is running atop HAOS 10.5 running HA 2023.9.2.

node-zwave-js: 8.26.0
zwave-js-ui: 11.14.3

Did you change anything?

yes (please describe)

If yes, what did you change?

Upgrade controller to 1.2 FW (SDK: 7.19.3)

Did this work before?

Yes (please describe)

If yes, where did it work?

Seemed to be working fine on the 1.0 version of the controller firmware.

Attach Driver Logfile

The debug logs get really big when this issue is occuring, so I've tried to trim them down to the few minutes before/after it starts to occur. Let me know if you need different time versions, or if you want the full 500MB of logs (and that's just 1 day).

The issue seems to start occuring around 2023-09-22T00:03:07.761Z where the transmit status starts to fail.

zwavejs_2023-09-21.log

[asayler]

It's possible this is the same issue as #6260.

[AlCalzone]

Yeah it is.