Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not all values are returned by getDefinedValueIDs #1184

Closed
robertsLando opened this issue Dec 3, 2020 · 44 comments · Fixed by #1192
Closed

Not all values are returned by getDefinedValueIDs #1184

robertsLando opened this issue Dec 3, 2020 · 44 comments · Fixed by #1192
Assignees
Labels
device compatibility Work that needs to be done to support non-compliant or legacy devices

Comments

@robertsLando
Copy link
Member

robertsLando commented Dec 3, 2020

After some digging into this problem I discovered that when the node ready event is triggered if I call zwaveNode.getDefinedValueIDs() that doesn't return all valueids, expecially seems that for this specific device it is missing the valueIds for endpoints > 0:

2020-12-03T08:39:19.525Z z2m:Zwave Node 3 is alive
2020-12-03T08:39:19.555Z z2m:Zwave Node 3: value added 3-94-0-zwavePlusVersion => 1
2020-12-03T08:39:19.556Z z2m:Zwave Node 3: value added 3-94-0-nodeType => 0
2020-12-03T08:39:19.556Z z2m:Zwave Node 3: value added 3-94-0-roleType => 5
2020-12-03T08:39:19.556Z z2m:Zwave Node 3: value added 3-94-0-installerIcon => 4608
2020-12-03T08:39:19.556Z z2m:Zwave Node 3: value added 3-94-0-userIcon => 4609
2020-12-03T08:39:19.557Z z2m:Zwave Node 3: value added 3-112-0-2 => undefined
2020-12-03T08:39:19.557Z z2m:Zwave Node 3: value added 3-112-0-3 => undefined
2020-12-03T08:39:19.557Z z2m:Zwave Node 3: value added 3-112-0-4 => undefined
2020-12-03T08:39:19.557Z z2m:Zwave Node 3: value added 3-112-0-5 => undefined
2020-12-03T08:39:19.557Z z2m:Zwave Node 3: value added 3-112-0-6 => undefined
2020-12-03T08:39:19.558Z z2m:Zwave Node 3: value added 3-112-0-7 => undefined
2020-12-03T08:39:19.558Z z2m:Zwave Node 3: value added 3-112-0-8 => undefined
2020-12-03T08:39:19.558Z z2m:Zwave Node 3: value added 3-112-0-10 => undefined
2020-12-03T08:39:19.558Z z2m:Zwave Node 3: value added 3-112-0-11 => undefined
2020-12-03T08:39:19.558Z z2m:Zwave Node 3: value added 3-112-0-12 => undefined
2020-12-03T08:39:19.559Z z2m:Zwave Node 3: value added 3-112-0-13 => undefined
2020-12-03T08:39:19.559Z z2m:Zwave Node 3: value added 3-112-0-14 => undefined
2020-12-03T08:39:19.559Z z2m:Zwave Node 3: value added 3-112-0-15 => undefined
2020-12-03T08:39:19.559Z z2m:Zwave Node 3: value added 3-112-0-16 => undefined
2020-12-03T08:39:19.559Z z2m:Zwave Node 3: value added 3-112-0-17 => undefined
2020-12-03T08:39:19.560Z z2m:Zwave Node 3: value added 3-112-0-18 => undefined
2020-12-03T08:39:19.560Z z2m:Zwave Node 3: value added 3-112-0-19 => undefined
2020-12-03T08:39:19.560Z z2m:Zwave Node 3: value added 3-112-0-20 => undefined
2020-12-03T08:39:19.560Z z2m:Zwave Node 3: value added 3-112-0-21 => undefined
2020-12-03T08:39:19.560Z z2m:Zwave Node 3: value added 3-114-0-manufacturerId => 411
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-114-0-productType => 3
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-114-0-productId => 515
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-134-0-libraryType => 3
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-134-0-protocolVersion => 6.7
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-134-0-firmwareVersions => 4.0
2020-12-03T08:39:19.561Z z2m:Zwave Node 3: value added 3-134-0-hardwareVersion => undefined
2020-12-03T08:39:19.562Z z2m:Zwave Node 3: value added 3-134-0-sdkVersion => 6.81.6
2020-12-03T08:39:19.562Z z2m:Zwave Node 3: value added 3-134-0-applicationFrameworkAPIVersion => 4.3.0
2020-12-03T08:39:19.562Z z2m:Zwave Node 3: value added 3-134-0-applicationFrameworkBuildNumber => 52445
2020-12-03T08:39:19.562Z z2m:Zwave Node 3: value added 3-134-0-hostInterfaceVersion => unused
2020-12-03T08:39:19.562Z z2m:Zwave Node 3: value added 3-134-0-hostInterfaceBuildNumber => 0
2020-12-03T08:39:19.563Z z2m:Zwave Node 3: value added 3-134-0-zWaveProtocolVersion => 6.7.0
2020-12-03T08:39:19.563Z z2m:Zwave Node 3: value added 3-134-0-zWaveProtocolBuildNumber => 97
2020-12-03T08:39:19.563Z z2m:Zwave Node 3: value added 3-134-0-applicationVersion => 4.0.33
2020-12-03T08:39:19.563Z z2m:Zwave Node 3: value added 3-134-0-applicationBuildNumber => 52445
2020-12-03T08:39:19.563Z z2m:Zwave Node 3: value added 3-94-1-zwavePlusVersion => 1
2020-12-03T08:39:19.564Z z2m:Zwave Node 3: value added 3-94-1-nodeType => 0
2020-12-03T08:39:19.564Z z2m:Zwave Node 3: value added 3-94-1-roleType => 5
2020-12-03T08:39:19.564Z z2m:Zwave Node 3: value added 3-94-1-installerIcon => 4608
2020-12-03T08:39:19.564Z z2m:Zwave Node 3: value added 3-94-1-userIcon => 4609
2020-12-03T08:39:19.564Z z2m:Zwave Node 3: value added 3-50-1-value-65537 => 545.7
2020-12-03T08:39:19.565Z z2m:Zwave Node 3: value added 3-50-1-deltaTime-65537 => 0
2020-12-03T08:39:19.565Z z2m:Zwave Node 3: value added 3-50-1-value-66049 => 0.28
2020-12-03T08:39:19.565Z z2m:Zwave Node 3: value added 3-50-1-deltaTime-66049 => 0
2020-12-03T08:39:19.566Z z2m:Zwave Node 3: value added 3-50-1-value-66561 => 234.8
2020-12-03T08:39:19.566Z z2m:Zwave Node 3: value added 3-50-1-deltaTime-66561 => 0
2020-12-03T08:39:19.566Z z2m:Zwave Node 3: value added 3-50-1-previousValue-65537 => undefined
2020-12-03T08:39:19.567Z z2m:Zwave Node 3: value added 3-50-1-previousValue-66049 => undefined
2020-12-03T08:39:19.567Z z2m:Zwave Node 3: value added 3-50-1-previousValue-66561 => undefined
2020-12-03T08:39:19.567Z z2m:Zwave Node 3: value added 3-64-1-mode => 1
2020-12-03T08:39:19.567Z z2m:Zwave Node 3: value added 3-64-1-manufacturerData => undefined
2020-12-03T08:39:19.568Z z2m:Zwave Node 3: value added 3-66-1-state => 0
2020-12-03T08:39:19.568Z z2m:Zwave Node 3: value added 3-67-1-setpoint-1 => 19
2020-12-03T08:39:19.568Z z2m:Zwave Node 3: value added 3-94-2-zwavePlusVersion => 1
2020-12-03T08:39:19.568Z z2m:Zwave Node 3: value added 3-94-2-nodeType => 0
2020-12-03T08:39:19.568Z z2m:Zwave Node 3: value added 3-94-2-roleType => 5
2020-12-03T08:39:19.569Z z2m:Zwave Node 3: value added 3-94-2-installerIcon => 3328
2020-12-03T08:39:19.569Z z2m:Zwave Node 3: value added 3-94-2-userIcon => 3329
2020-12-03T08:39:19.569Z z2m:Zwave Node 3: value added 3-94-3-zwavePlusVersion => 1
2020-12-03T08:39:19.569Z z2m:Zwave Node 3: value added 3-94-3-nodeType => 0
2020-12-03T08:39:19.569Z z2m:Zwave Node 3: value added 3-94-3-roleType => 5
2020-12-03T08:39:19.570Z z2m:Zwave Node 3: value added 3-94-3-installerIcon => 3328
2020-12-03T08:39:19.570Z z2m:Zwave Node 3: value added 3-94-3-userIcon => 3329
2020-12-03T08:39:19.570Z z2m:Zwave Node 3: value added 3-94-4-zwavePlusVersion => 1
2020-12-03T08:39:19.570Z z2m:Zwave Node 3: value added 3-94-4-nodeType => 0
2020-12-03T08:39:19.571Z z2m:Zwave Node 3: value added 3-94-4-roleType => 5
2020-12-03T08:39:19.571Z z2m:Zwave Node 3: value added 3-94-4-installerIcon => 3328
2020-12-03T08:39:19.571Z z2m:Zwave Node 3: value added 3-94-4-userIcon => 3329
2020-12-03T08:39:19.676Z z2m:Zwave Node 3 ready: ThermoFloor - Heatit Z-TRM3 (Floor thermostat)
2020-12-03T08:39:19.733Z z2m:Zwave Node 3: metadata updated: 49-2-Air temperature
2020-12-03T08:39:19.734Z z2m:Zwave Node 3: value updated: 49-2-Air temperature 24.8 => 24.8
2020-12-03T08:39:20.053Z z2m:Zwave Node 3: metadata updated: 49-3-Air temperature
2020-12-03T08:39:20.054Z z2m:Zwave Node 3: value updated: 49-3-Air temperature 0 => 0
2020-12-03T08:39:21.676Z z2m:Zwave Node 3: metadata updated: 49-4-Air temperature
2020-12-03T08:39:21.676Z z2m:Zwave Node 3: value updated: 49-4-Air temperature 19 => 19

Ref: zwave-js/zwave-js-ui#38 (comment)

@robertsLando robertsLando added the bug Something isn't working label Dec 3, 2020
@AlCalzone
Copy link
Member

for this specific device it is missing the valueIds for endpoints > 0

I do see values for the individual endpoints. Not every endpoint has the same functionality. Which values are expected that aren't present here?

@robertsLando
Copy link
Member Author

robertsLando commented Dec 3, 2020

Those that comes immediatly after the node ready:

2020-12-03T08:39:19.734Z z2m:Zwave Node 3: value updated: 49-2-Air temperature 24.8 => 24.8
2020-12-03T08:39:20.053Z z2m:Zwave Node 3: metadata updated: 49-3-Air temperature
2020-12-03T08:39:20.054Z z2m:Zwave Node 3: value updated: 49-3-Air temperature 0 => 0
2020-12-03T08:39:21.676Z z2m:Zwave Node 3: metadata updated: 49-4-Air temperature
2020-12-03T08:39:21.676Z z2m:Zwave Node 3: value updated: 49-4-Air temperature 19 => 19

I do see values for the individual endpoints

You right, seems this only happens with Multilevel sensors values so

Could it be a cache problem or what else? Why them are not presend in defined values when node is ready?

@AlCalzone
Copy link
Member

Since it is a value updated event, not a value added event, they should already be there. Are they not returned with getDefinedValueIDs?

@robertsLando
Copy link
Member Author

Nope, them are not returned or you would see them also in value added. That's the strange thing

@AlCalzone AlCalzone changed the title Not all values are present when node is ready Not all values are returned by getDefinedValueIDs Dec 3, 2020
@AlCalzone
Copy link
Member

Can you ask for the cache files please, so I can try to reproduce this in a test?

@roflmao
Copy link
Contributor

roflmao commented Dec 3, 2020

zwavejs-cache.zip

@robertsLando
Copy link
Member Author

robertsLando commented Dec 3, 2020

Just for reference

https://github.com/zwave-js/zwavejs2mqtt/blob/master/lib/ZwaveClient.js#L284

Here is where I add all values when I receive the node ready event, and I would expect to have all the valueIds at that point.

@AlCalzone
Copy link
Member

According to the cached information, Node 3 doesn't support Multilevel Sensor CC, therefore getDefinedValueIDs does not return those values.
@roflmao please try re-interviewing it to see if that was a glitch. If it still doesn't work afterwards we need to mark it as supported through the configuration file.

@AlCalzone AlCalzone added device compatibility Work that needs to be done to support non-compliant or legacy devices and removed bug Something isn't working labels Dec 3, 2020
@roflmao
Copy link
Contributor

roflmao commented Dec 3, 2020

@AlCalzone restart or refresh node?

@AlCalzone
Copy link
Member

Refresh node, that resets the information we know about the node, including the supported CCs.

@roflmao
Copy link
Contributor

roflmao commented Dec 3, 2020

zwavejs-cache.zip

@robertsLando
Copy link
Member Author

@AlCalzone Seems only node 2 and 7 are reporting 0x31 as supported CC. So we should add a compact flag for this? @roflmao Could you add the hex identifier of this device? You can find it in control panle when you select a node near the device id

@roflmao
Copy link
Contributor

roflmao commented Dec 3, 2020

Device ID: 411-515-3 (0x019b-0x0203-0x0003)

@AlCalzone
Copy link
Member

Seems like OZW specifically adds the CC and its values through the config files. Wonder if the device will respond to requests though.

@robertsLando
Copy link
Member Author

Wonder if the device will respond to requests though.

Based on my little experience in issues here I say that it may work :) It's like the stopLevelChange that was reported as unsupported and then just by sending it without checks works 🤷‍♂️

@AlCalzone
Copy link
Member

It's like the stopLevelChange that was reported as unsupported

That was different. Sending it with Supervision was unsupported. The command itself is supported or the zwave-js method would throw immediately.

@AlCalzone
Copy link
Member

@roflmao could you try the compat-add-cc branch and see if this resolves your issue?

@roflmao
Copy link
Contributor

roflmao commented Dec 6, 2020

I still get the same in the logs and now the device isnt identified:

 2020-12-06T20:24:56.019Z z2m:Gateway Unable to discover climate device, there is no valid temperature/thermostat mode valueIds                                                                                    
 2020-12-06T20:24:56.021Z z2m:Zwave Node 3 ready: Unknown manufacturer 411 - Unknown product 515 (3)    

zwavejs-cache.zip

@AlCalzone
Copy link
Member

Hmm, seems it doesn't find the device config file now. I'll investigate.

@roflmao
Copy link
Contributor

roflmao commented Dec 6, 2020

@AlCalzone same goes for master

@AlCalzone
Copy link
Member

Hmm, I can't reproduce the behavior in tests where the file is loaded correctly. Only the changes were not applied because I forgot a line, but that shouldn't cause the lookup to fail.

Are you sure that the dependencies are included correctly?

@roflmao
Copy link
Contributor

roflmao commented Dec 6, 2020

EDIT: forgot node_modules doesnt show versions.

node_modules:
node_modules.find.log

Or test with docker image: roflmao/zwavejs2mqtt:compat-add-cc (running node-zwave-js from branch and zwavejs2mqtt from master)

@roflmao
Copy link
Contributor

roflmao commented Dec 6, 2020

npm.ls.log

Maybe this helps (npm ls --json output)

@AlCalzone
Copy link
Member

The libraries seem ok, although I'm not sure how the symlinks in the zwave-js workspace are displayed there.

Since it works on my machine (and I can't test whats going on without your device), I guess you'll need to add some logging:
@zwave-js/config/config/build/Devices.js:

// lines 113-114:
-        if (!(await fs_extra_1.pathExists(filePath)))
-            return;
+        if (!(await fs_extra_1.pathExists(filePath))) {
+            Logger_1.default.config.print(`${filePath} does not exist`, "error");
+            return;
+        }


// before line 111
+    if (!indexEntry) Logger_1.default.config.print(`did not find device config for ${manufacturerId}, ${productType}, ${productId}, ${firmwareVersion}`, "error");

@roflmao
Copy link
Contributor

roflmao commented Dec 7, 2020

Ok, just located an error with the latest Dockerfile.contrib dependency-wise.

Here it is with correct zwave-js branch compat-add-cc and dependencies:

zwavejs-cache.zip

@robertsLando
Copy link
Member Author

@roflmao Could you also give a check to z2m master now? I have submitted a fix that may fix some discovery issues

@roflmao
Copy link
Contributor

roflmao commented Dec 7, 2020

@robertsLando
Copy link
Member Author

@roflmao Could you also tell me something? :) Better ? Worse? same? new errors?

@roflmao
Copy link
Contributor

roflmao commented Dec 7, 2020

@robertsLando Haha, sorry! Its the same it looks to me. First unable to climate, then climate discovered.

@robertsLando
Copy link
Member Author

Ok I see the error is still there but it's a device compatibility error as that CC isn't reported as supported. Once fixed you should not see that error on console :)

@AlCalzone
Copy link
Member

That's what the mentioned branch should fix. I'll take a look at the logs later to see if I can recognize a problem.

@AlCalzone
Copy link
Member

AlCalzone commented Dec 7, 2020

@roflmao it looks a bit better:

16:00:39.704 CNTRLR   [Node 003] device config loaded

but the additional CCs still don't show up in the network cache. Is your test based on the latest commit of the branch?

@AlCalzone
Copy link
Member

AlCalzone commented Dec 7, 2020

So, I tested locally and modified a config file to add a non-supported CC to the list. Here's my MultiSensor 6 cache file after the interview:

...
                "0x51": {
                    "name": "Move To Position Window Covering",
                    "endpoints": {
                        "0": {
                            "isSupported": true,
                            "isControlled": false,
                            "secure": false,
                            "version": 0
                        }
                    }
                },
...

so it does seem to work.

@roflmao
Copy link
Contributor

roflmao commented Dec 7, 2020

Ok my sha tracking has been very much needed so i hacked together something:

root@bender:/usr/src/app# cat package.json  | grep \"version
  "version": "1.0.0-alpha.0+b9a6315",
root@bender:/usr/src/app# cd node_modules/zwave-js/
root@bender:/usr/src/app/node_modules/zwave-js# cat package.json  | grep \"version
  "version": "5.5.0-fe071207+791f5736",

Confirming:

# cat node-zwave-js.version-sha version-sha 
fe071207
b9a6315

Logs:
zwavejs-log-bundle.zip

@AlCalzone
Copy link
Member

Hmmm. I still don't see changes to the network cache file. I guess there's no way that you can step through packages\zwave-js\src\lib\node\Node.ts (lines 1539-1549) with a debugger?

@roflmao
Copy link
Contributor

roflmao commented Dec 7, 2020

Sure, just tell me how :)

@AlCalzone
Copy link
Member

The docker part makes it harder - maybe @robertsLando has some insight?
I usually open a repo in VSCode, build it and execute the main entry point in a Javascript Debug Terminal or start a dedicated task.

@roflmao
Copy link
Contributor

roflmao commented Dec 8, 2020

@AlCalzone Im up for anything, debug statements, remote shell etc.

@mpowell90
Copy link

mpowell90 commented Dec 9, 2020

Not sure if this is related, I also have an Aeotec Multisensor 6, just started testing with it today.
Parameters 111, 112 and 113 (set report group time intervals) aren't included in the getDefinedValueIDs array.
However, after digging through the spec, I was able to set these parameter values successfully.

@robertsLando
Copy link
Member Author

@roflmao https://code.visualstudio.com/docs/containers/debug-node

@AlCalzone
Copy link
Member

AlCalzone commented Dec 9, 2020

@mpowell90

Parameters 111, 112 and 113 (set report group time intervals) aren't included in the getDefinedValueIDs array

They are defined in the config file, so my guess is that reading them from the device failed. However I can't say for sure without logs.

@AlCalzone
Copy link
Member

@roflmao did you have some success with the debugger yet?

@roflmao
Copy link
Contributor

roflmao commented Dec 13, 2020

@AlCalzone Sorry but ive been busy with dead nodes in my network so need to get it up and running first. Also im not so familiar with vscode but ill try when i get the time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
device compatibility Work that needs to be done to support non-compliant or legacy devices
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants