listen node Problem

[savage7]

I can't get the "listen" node to work, it doesn't receive any events.

This mqtt setup works:
I've a working mqtt Env:
UT0311-L0x.223195309.address = 192.168.1.123:60000
UT0311-L0x.223195309.door.1 = Front Door
UT0311-L0x.223195309.door.2 = Side Door

the listener is set to 192.168.1.131:60001
When I open the door, I get an event.

Node red setup:
it runs on the same "machine"/docker.
the listener is set to 192.168.1.131:60002
I choose another port, so that there is no port collision. I also tried it with 60001.
When I open the door, I dont receive an event via the listen node.

I looked at your example. All the other "api" node like get-listener, set-listener work, get-status...
i.e i did set the listener via node.

my door config:

Do a have an error in the config?
I dont get an "error" trace in the debug console, like "cannot bind adress". The listen node also doesnt show an "red error" below the node.
Can I provide some debug output somehow (the controller config is set to debug)?

thxs alot for your help!!!

[uhppoted]

Hi,

If I've understood it correctly:

• 223195309 is your real controller
• uhppoted-mqtt is running on a Docker instance, listening for events on port 60001
• node-red is running on the same Docker instance as uhppoted-mqtt, listening for events on port 60002

Is that correct?

The first step in debugging this is to confirm that the events are getting sent to where you expect them:

1. Can you confirm (using Wireshark or tcpdump or the uhppote-cli listen command) that you are receiving events on port 60002 on the machine that is hosting the Docker instance ? (if you're using uhppote-cli you'll have to do it without the Docker instance running)
2. Can you confirm (using Wireshark or tcpdump or the uhppote-cli listen command) that you are receiving events on port 60002 on the Docker instance ?

[uhppoted]

Hi,

Quick addendum - if you are running NodeRED in a Docker container, have you forwarded UDP port 60002?

docker run -it -p 1880:1880 -p 60002:60002/udp --name uhppoted nodered/node-red

You may also need to set your config listen address to:

0.0.0.0:60002

which is the simplest way of making sure the container's listen node is bound to the internal container network address.

[savage7]

Nodered runs in host network mode, but I tried adding the ports:
-p 60002:60002/udp
Also tried to add tcp too...
--> no difference

I changed the listen address to:
0.0.0.0:60002
--> no difference

Tcpdump:
tcpdump -i any -nn | grep 192.168.1.123
Controller ip is: 192.168.1.123
Docker machine ip: 192.168.1.131
Listener Port: 60002

outside docker on the host machine:
16:49:20.931568 IP 192.168.1.123.60000 > 192.168.1.131.60002: UDP, length 64

Inside the docker container:
17:59:00.330065 IP 192.168.1.123.60000 > 192.168.1.131.60002: UDP, length 64

For me the traces look ok still, the "listen" node doesnt output anything. :(
I did setup a nodered instance on my local machine, not within docker. same I dont get any events :(

[uhppoted]

Hmm ... there may be a problem with NodeRED receiving UDP multicast packets with Docker in host networking mode (Ref. https://groups.google.com/g/node-red/c/g5ZSzr7iNI8) but I'm going to have to dig into it to figure if that's actually the case.

Unfortunately host networking mode doesn't work on a Mac and I don't have a RaspberryPi to hand so it's going to take a little time to get set up. Some things to try while I do that:

1. Try a "normal" container with bridged networking (this worked out of the box for me):

docker run -it -p 1880:1880 -p 60001:60001/udp --name uhppoted nodered/node-red

2. Try uhppote-cli --debug listen

Can you also please post your Dockerfile and the command you use to run it?

[savage7]

Ok I simplified my setup, to remove any error regarding docker.
I created a local node-red installation on my mac (192.168.1.109).

1. uhppote-cli --debug listen
   Works, I get the events.

2. Add the listen node and configured it

• quit uhppote-cli
  --> No events received, I get no message from the listen node.
  I used this configuration on my mac:

I honestly don't know whats wrong.

[uhppoted]

Have to admit I'm fairly baffled as well - there are definitely networking complexities around docker, IPv4/IPv6 and the more recent Linux releases and I'm still digging into those.

But .. I was able to get node-red receiving events from my local development controller on a remote (vultr) Ubuntu 20.10 VPS without doing anything at all special. These were the steps I followed:

1. Configure firewall and startup scripts so allow TCP port 1880 and UDP port 60001 from my development system public IP
2. Start Ubuntu 20.10 instance
3. Login via ssh
4. apt-get update
5. apt-get install npm
6. npm install -g --unsafe-perm node-red
7. node-red
8. Open the VPS node-red page in a browser
9. Install node-red-contrib-uhppoted to the palette
10. Create a flow with a listen node connected to debug node (with the listen address configured as 0.0.0.0:60001)

On my development system:

1. uhppote-cli --debug set-listener <serial#> <VPS address>:60001
2. Swipe card

The swipe event appears in the debug pane of the node-red page. You could try this just to establish a baseline working system as starting point for further debugging - or if you can let me have your public IP address am happy to temporarily set up a system that you can access.

One other thing:

uhppote-cli is the original application and I remember struggling to get it to work correctly with both WiFi and LAN enabled on my Mac, as a result of which it has a somewhat more complex default address detection algorithm i.e. there's a possibility uhppote-cli --debug listen may not be binding to 192.168.1.109.

Some more things to try:

1. uhppote-cli --debug listen and use netstat or lsof to see what address:port it is bound to
2. uhppote-cli --listen 192.168.1.109:60001 --debug listen, to explicity bind to the same listen address:port as node-red
3. Disable WiFi entirely and see if that allows node-red to receive events

[uhppoted]

Hi,

Having experimented some more with a Ubuntu 20.10 VPS I'm able to receive events using NodeRED in a docker container in both host networking mode and normal networking mode without doing anything special.

The only setting that seemed critical was the listen address in the the config node, which must be set to 0.0.0.0:60001.

The command lines used to start NodeRED in a docker container were:

• interactive, host network mode:

docker run --interactive --net=host nodered/node-red

• detached, host network mode:

docker run --detach --net=host nodered/node-red

• interactive, docker proxy (normal):

docker run --interactive --publish 1880:1880 --publish 60001:60001/udp nodered/node-red

• detached, docker proxy (normal):

docker run --detach --publish 1880:1880 --publish 60001:60001/udp nodered/node-red

[uhppoted]

BTW - if you're expecting to receive door relay and push button events, then please be aware of uhppote-core/Issue #2: Door open or closed event not generated.

The CLI has an update that allows you to enable door and push button event recording.

[uhppoted]

Hi,

I'm going to close this issue - NodeRED flows on Ubuntu (both in and outside of Docker containers) were able to receive events without a problem in all configurations I tried. The only critical points to note were:

1. The firewall should be configured to allow UDP events on the listen port e.g.:

   ufw allow from <address> to any port 60000 proto udp
   

2. door open, door close and door button events are only generated if enabled via record-special-events. Support for this function has been added in v0.6.7.

3. The listen address in the config should be 0.0.0.0:<port>

Please feel free to reopen if/when you have any more information to add.

[savage7]

I'll try 0.6.7 again.
Yes sure, I've tried all variants you described but was not able to listen to message from the controller.

Thanks alot for your help and your great work! :)

[uhppoted]

Hmm, I don't think 0.6.7 will make any difference - it only really adds the record-special-events node.

I'm really puzzled because this 'just works' for me - can you maybe upload some Wireshark captures? Maybe I can figure out something from the actual messages..

[savage7]

I traced it on my mac not in docker.

Mac ip: 192.168.1.142
Controller id: 192.168.1.123
Capture filter applied:
ip.src == 192.168.1.123

I opened the web interface on my mac and opened the door --> thats the tcp, http pakets
Then the controller sends a udp message to the Mac.

Trace:
trace.pcapng.zip

Configuration:

[uhppoted]

Thanks! That made it easy - the message bytes are:

   0000   19 20 00 00 ad b0 4d 0d 35 03 00 00 02 01 01 01
   0010   8e 01 a8 c0 20 21 01 04 11 07 08 2c 01 01 00 00
   0020   00 00 00 00 00 11 07 08 00 00 00 00 00 00 00 00
   0030   00 01 00 21 01 04 00 00 00 00 00 00 00 00 00 00

Normally messages start with a 0x17 and that's what the NodeRED codec expects. However, I have seen this before on an older controller and there is a patch for it in the Go code:

// PATCH: accommodates undocumented but seemingly valid 'listen' events that start with 0x19
if (bytes[0] != 0x17) && (bytes[0] != 0x19 || bytes[1] != 0x20) {
    return errors.New(fmt.Sprintf("Invalid start of message - expected 0x17, received 0x%02x", bytes[0]))
}

Which I had forgotten about and missed when doing the port to NodeRED :-(. Apologies .. I should have a fix out in the next day or so.

Out of curiosity - what version is the firmware on your controller?

[savage7]

Oh that is great 😊 thank you!

I don't see a "firmware version" but a "driver version" in the web interface, that is: V6.62.51215

[uhppoted]

Yup, v6.62 was the firmware version of the controller that patch was made for - suspect it was something that got overlooked when they were updating the protocol because it was fixed in later versions.

I'll make a note of it in the README.

[uhppoted]

Hi,

I've pushed an update to the node-red-contrib-uhppoted repo to handle the v6.62 events and also updated the implementation in uhppote-core and uhppote-simulator.

I'm going to hold off on releasing it for a couple of days though, in the hope that NodeRED will release a patch for CVE-2020-28168 in one of their dependencies.

If you want to try out the fix, please use the version in the master branch - otherwise I'll publish a new version early next week.

[savage7]

Sure just take your time, I'll wait for the deployment.

[uhppoted]

Hi,

I've released v0.6.8 with the fix for the listen events and published it to NPM and NodeRED.

Can you give it a try and see if you now receive events?

[savage7]

@uhppoted
Yes it works ❤️
This is really cool, the node red integration is so clean and nicely done :)
Thanks alot.

[uhppoted]

Oh, that's great! Thanks for trying it out .. and thanks for your patience while we got to the bottom of the problem!

Am very glad you're finding the node red integration straightforward - I feel like it just opens up a world of possibilities :-).