Device info lost on restart #875
Replies: 12 comments 39 replies
-
|
@chriswood1001 Could you provide a full zwavejs silly level log file? Also what z2m version are you using? |
Beta Was this translation helpful? Give feedback.
-
|
Did this happen to all of your devices below a certain point, all my nodes up to 30 work as normal everything after that I get the same error you are now. Everything was working fine before. Would a silly log from me help? |
Beta Was this translation helpful? Give feedback.
-
|
After ~48hrs the devices eventually restored themselves and I've not had this issue since, even after restarts. I originally had the problem with 100% of my battery devices, and because I include powered devices first it meant it was the ~2nd half of my list (battery devices) that exhibited the problem. Fortunately I've been stable for ~2weeks after many restarts and not seen the issue again. |
Beta Was this translation helpful? Give feedback.
-
|
This exact same thing happened to me and I'm struggling to find a solution. Fully included about 50 nodes, successfully interviewed and entities created. Suddenly I'm seeing about 3/4 of those devices reverting to Unknown manufacturer 0x031e. It's very strange as most of the devices exhibiting this glitch/failure are Innoveli LZW31. There are a few LZW31 devices in my network that aren't exhibiting this behavior so it can't be specific to the type of device. Attached is the silly level logs for you. Would love to figure out what's happening with this installation |
Beta Was this translation helpful? Give feedback.
-
|
@robbpope Did you try to wait some hours to see if the problem fix itself? |
Beta Was this translation helpful? Give feedback.
-
|
yes I did. This has been an ongoing issue where I pair all of the devices, watch their information populate and set up my behaviors in Node Red. Then, a week later I will come back to check in on the installation (it's at a business) and half of the nodes have lost their parameters and all of their fields are either blank or showing 0x0(etc). I went ahead and rebooted, manually initiated a re-interview on all nodes that are displaying the failure mode described above, and waited overnight with "silly" level logging enabled. I'm attaching these (hopefully) much more complete logs here. Check them out and let me know what other info I can get you to help me troubleshoot. Id be happy to allow you to log into the device via NabuCasa if you'd like. |
Beta Was this translation helpful? Give feedback.
-
|
I was able to restore about half of my devices by deleting db1d31ed.json yesterday only to check again this morning and find them unknown again. Like robbpope I have a ton of inovelli about half work and half dont among other devices. Let me know if there is anything I can do to provide additional logs\screenshots whatever. Before yesterday I let it sit for 3 or 4 days before I did anything and no change. |
Beta Was this translation helpful? Give feedback.
-
|
Just a little more info I am running in docker on an unraid setup using a HUSBZB-1 |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Running into the same issue, had a restart last night and noticed it today that they were all unknown Had this on the 0.9.0 addon, and then upgraded to 0.9.1 and still have the issue |
Beta Was this translation helpful? Give feedback.
-
|
@AlCalzone any clue what could cause this? |
Beta Was this translation helpful? Give feedback.
-
|
Indeed, Node 68 is missing some info. Which version of |
Beta Was this translation helpful? Give feedback.
-
|
zwavejs2mqtt: 3.0.3 Although this started on 7.0.0 |
Beta Was this translation helpful? Give feedback.
-
|
An even more complete log of my installation. |
Beta Was this translation helpful? Give feedback.
-
|
Unraid updated me to the latest version last night and all my nodes after node 34 are dead again here is the latest version I am running: zwavejs2mqtt: 3.1.0 |
Beta Was this translation helpful? Give feedback.
-
|
Any chance you could chime in here with some more info? |
Beta Was this translation helpful? Give feedback.
-
|
Had this same issue and the only way to get the nodes functional was to delete all of the zwaveJS2mqtt config files in the shared volume and have it recreate the devices from the controller. Noticed this issue because the Pi completely locked up sometime last night and had to perform a hard reboot. After the reboot, none of the devices were controllable through HA or zwaveJS2mqtt. Tried re-interviewing the nodes but none of them succeeded. Unfortunately, I found this topic after completely blowing out the config but I do have a complete backup (attached). Using an Aeotec Z-Stick on a Pi4 with zwaveJS2mqtt in a docker container. Home Assistant is also running in a docker container on a different machine. One thing I've noticed that has changed is that zwaveJS2mqtt interviews all nodes at once. In the original version I had installed, not sure what version it was, it would only inspect one device at a time. Not sure if this is related but it does appear to put a significant load on the zwave network while inspecting all nodes at once, whereas before it wasn't as noticeable and the inspection was much faster. |
Beta Was this translation helpful? Give feedback.
-
|
@AlCalzone Unfortunately this happened again. Rebooted the Pi a couple times for items unrelated to zwave and it happened again. This time I was able to save the debug before deleting the files. Please take a look, the logs are very peculiar. Edit: found issue #998 and will try the updated container. |
Beta Was this translation helpful? Give feedback.
-
|
Apologize, bonehead move. Didn't realize zwave was a separate log. Here are the general and zwave logs at silly. Turned on and off node 5 physically at the switch so you can see the communication. |
Beta Was this translation helpful? Give feedback.
-
|
No amount of rebooting, power-cycling, or re-interviewing would bring the devices online. Completely deleted the configuration and reprogrammed, nothing. Even tried downgrading to v3.3.0. As a last-ditch effort, unplugged the Z-Stick while the container was online and plugged it in again, then rebooted. Nodes then started responding. |
Beta Was this translation helpful? Give feedback.
-
The stick was stuck and needed a power cycle.
For reference, don't do that. That can corrupt the stick's memory contents. Best if you shut down the host, wait a few seconds, then pull and replug the stick and boot again. |
Beta Was this translation helpful? Give feedback.
-
|
HI Al, On the other hand, I believe the Z-Stick has an internal battery. Would think this helps prevent internal memory corruption. It's almost like something happened to the Z-Stick while zwaveJS2mqtt was crashed. Like the internal buffer filled and couldn't clear itself, even if the Pi was power cycled. Not sure how unplugging the Z-Stick was different but it definitely resolved the issue. There was also a very odd issue that while it was malfunctioning, the logs showed identical status messages received 14, 15 or 16 times. In the latest logs attached, at 2021-04-19 22:42:07.679, the switch (Node 5) was physically turned off once and 14 "false" messages were received. It's working now but this happened twice on 3.3.0. Hopefully, the SIGTERM or SIGINIT change prevents this from returning. |
Beta Was this translation helpful? Give feedback.
-
Might be, I'm not sure.
Might be that the node was trying to reach the controller and re-tried a few times too many :D |
Beta Was this translation helpful? Give feedback.
-
|
I'm also having this problem. Not running Docker, but running on an Enterprise Linux 8 VM using the packaged version. Likely happening when the power flickers and the ESX host restarts (I have a UPS, but the batteries died and I haven't replaced them yet). |
Beta Was this translation helpful? Give feedback.
-
|
It's possible that this might be fixed for me—I had been on 4.2.1, but upgraded a couple days ago to 4.5.1. Had another power failure today, and all of my devices were still identified on restart. |
Beta Was this translation helpful? Give feedback.
-
|
Yep we worked on a fix for this, the bug shouldn't be present anymore |
Beta Was this translation helpful? Give feedback.
-
|
Running zwave-js 7.7.5 (z2m 5.0.4), I had an unintended power cycle in the RPi and when the system came up (HA, zwavejs2mqtt, both running on docker) all the battery-powered devices showed up in unknown state and I noticed zwave-js was trying to re-interview them. Just migrated to zwave-js and pretty disappointed that I will need to manually wake up each of the sensors (a dozen). Normal container restarts do not exhibit this problem meaning that volume persistence is working ok. It seems the way zwave-js is handling it's cache is a bit weak and it gets corrupted easily. Any known workaround to make z2m more resilient to restarts? |
Beta Was this translation helpful? Give feedback.
-
|
The problem seems more something docker related then zwave-js related. @AlCalzone is working in many different fixes related to cache files but seems the problem is still there... |
Beta Was this translation helpful? Give feedback.
-
|
@arigit The cache files are append-only most of the time. Every now and then, they get compacted which requires re-writing the files. This was always done by writing a separate file, then renaming and finally deleting the original db. However, I've recently learned that Linux might require an That being said - it can also be that your stick is not responsive, which can look a lot like missing cached data. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the fast response. I will retest this scenario after updating to 7.9. Prior to the migration to zwavejs2mqtt I've never noticed this happening before (with many battery sensors going to an unknown state), with the same stick. I did notice something interesting though, while I manually had to wake up all my zooz ZSE40 presence sensors and the aeotec ZWA012 door sensors (after waiting 3-4 hours for sensors to come up on their own, triggering them to no avail), there was one battery sensor that came back up on its own, the aeotec water sensor ZW122, I didn't reset this one nor tried to wake it up). I am not sure if it came back up immediately when the driver restarted or if it took a few hours. Next time this happens I will take a snapshot of state. |
Beta Was this translation helpful? Give feedback.
-
|
I filled some of my breakers yesterday and hard powered off on accident some of my systems. HA+zwave2jsmqtt runs on a NUC and all devices showed up as |
Beta Was this translation helpful? Give feedback.
-
|
Not docker to just running directly on Mint linux using systemd |
Beta Was this translation helpful? Give feedback.
-
|
As always, I need more info to even begin looking at this.
|
Beta Was this translation helpful? Give feedback.
-
|
I'll try and reproduce the problem and give you the cache and driver logs. |
Beta Was this translation helpful? Give feedback.
-
Make sure you update at least to 7.9.0 (z2m 5.1.0). That version contains an important fix (hopefully the last). |
Beta Was this translation helpful? Give feedback.
-
|
@AlCalzone I upgraded recently to newer version and my last forced power restart did not have the prior issues. |
Beta Was this translation helpful? Give feedback.
-
|
@AlCalzone Hello, unfortunately, had the same issue appear that was mentioned in the thread #875 (reply in thread). All devices in the zwavejs2mqtt web GUI were stuck in Unknown, then ProtocolInfo when trying to reinterview. zwavejs2mqtt: 5.4.5 This occurred after a scheduled reboot of the RPi running the docker container. Only way to clear the issue was to unplug the zstick with everything running and reconnect it. Below are the console logs from the container. The reboot happened at 09:04:37 Below are the logs from when the zstick was reconnected Below are the zwavejs logs |
Beta Was this translation helpful? Give feedback.
-
|
@mwolter805 Thanks for the logs. Looks like we need some kind of detection if the Z-Wave stick is stuck and reset it. |
Beta Was this translation helpful? Give feedback.
-
|
That would be great. Not sure what causes it to stop working. Have rebooted the pi dozens of times without issue but this time something went wrong. I could see this deterring users from the integration as the fix is elusive as nothing other than physically reconnecting the zstick resolves the issue. Tried deleting the config, the entire container, etc. nothing else worked. This has been experienced by other users as I've helped someone with the same issue on the HA forums. Wouldn't be far less frustrating if normal reboot fixed it as I could detect the zwave devices down and reboot the pi with an automation or bash script. |
Beta Was this translation helpful? Give feedback.
-
I had successfully included 50+ nodes where interview completed, manufacturer, product and attributes were all working correctly. After a power cycle of my RPI every one of my battery devices (~30) come back as "Unknown manufacturer 0xXXXX" and their characteristics are lost. Trying to re-interview fails, even when waking up the device.
Should this information not be retained and if so where? I have /store mounted where I can see my .json file, node, uses, and settings; however, I don't see manufacture or manufacture ID data in there. Am I doing something wrong?
Beta Was this translation helpful? Give feedback.
All reactions