MQTT Websocket behind Firewall

[ebnerjoh]

Hi,

This is not a general issue but more a question on the MQTT Websocket topic.

If I understood it correctly, then I could get an automated updated website (current values, not graphs) when using MQTT, correct?
e.g.: there are MQTT Services available, but they are all listening on ports like 8000 or 9001.

If I am correct then your SKIN will not be able to connect to those Brokers, when the firewall of the network (where the client is connected to) only allows 443 outgoing?

Is there a fallback possible like manual refresh or using html refresh instead MQTT?

Br,
Johannes

[poblabs]

Your understanding is correct. If you use MQTT and Websockets the website will update the front page in real time. Every weewx archive interval (weewx defaults to 5 minutes), the graphs will also update automatically.

Your firewall will need to allow a port to the broker behind the firewall. This is dependent on your own situation at home, at the provider, whatever your case is.

If you opt to NOT use MQTT, or MQTT broker is offline or unreachable, the website will function as a static website which is updated every weewx archive interval (weewx defaults to every 5 mins). You will need to refresh the page manually to see the updated information.

[ebnerjoh]

Hi,

I still have troubles with MQTT.

I have the following config in weewx:

[[MQTT]] server_url = mqtt://mqtt:[email protected]:1883/ topic = weather unit_system = METRIC retain = true binding = archive, loop aggregation = individual, aggregate

mqtt_enabled = 1 mqtt_host = "192.168.1.253" mqtt_port = 9001 mqtt_ssl = 0 mqtt_topic = weather disconnect_live_visitor = 1800000

The MQTT broker runs on my LAN and is also configured with "annonymous = true"

If I use hiveMQTT Test-Page, I can connect to the MQTT Websocket, just by adding IP, no user and password. I can also subscribe to the topics and I am getting the updates.

But the skin is says "Connecting to weather station real time data." and not "connected".

Br,
Johannes

[poblabs]

Sorry, I'm confused.

It looks like you're referencing an issue with the MQTT extension? I didn't write the extension, so you may be best to ask the forums about that if you're having trouble.

You need to make sure your broker can handle websockets for the skin to work. The skin would connect to the websockets port that you have setup.

I've written a general guide on setting up MQTT at https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/ - have you read this? It's referenced in the readme on this github repo.

[ebnerjoh]

Hi,

I think my description was very bad, excuse me, I will try it again:

1. Everything is in my local LAN, no Firewall in between
   Weewx, MQTT Broker, Webbrowser

2. I have setup weewx with the MQTT Extension to publish weather information. That works fine. Weewx is posting the values to MQTT and I can read the MQTT Messageswith Home Assistant, Node Red as well as with the HiveMQ Websocket Demo Client (http://www.hivemq.com/demos/websocket-client/)

3. I installed your Skin and configured the MQTT Options under "EXTRAS"

mqtt_enabled = 1
mqtt_host = "192.168.1.253"
mqtt_port = 9001
mqtt_ssl = 0 
mqtt_topic = weather 
disconnect_live_visitor = 1800000

One question is about the mqtt_topic, maybe this is what is breaking my config:

My weewx is publishing to "weather", I tried with mqtt_topic = weather as well as with mqtt_topic = weather/#

Br,
Johannes

[sgrayban]

Are you using the belchertown skin ?

[ebnerjoh]

Yes

[sgrayban]

[[MQTT]]
    server_url = mqtt://user:[email protected]:1883/
    topic = weather/weewx
    unit_system = US
    binding = archive, loop
    aggregation = aggregate

Under.......

    [[Belchertown]]
        HTML_ROOT = /home/weewx/public_html/Nettleton
        skin = Nettleton
        [[[Extras]]]
            mqtt_enabled = 1
            mqtt_host = 0.0.0.0
            mqtt_topic = weather/weewx/loop
            mqtt_port = 9001
            mqtt_ssl = 0

[sgrayban]

This of course only applies if you followed https://obrienlabs.net/how-to-setup-your-own-mqtt-broker/

[ebnerjoh]

The "loop" could be a good hint, I will give it a try.

Thanks,
Br,
Johannes

[ebnerjoh]

Somehow I am not getting it working.

I tried now again with the following procedure:

1. Activated Mosqitto MQTT Broker with Websocket Support on 9001
2. Tested locally on the LAN with an Web MQTT Test Client to register on my internal IP with "weather/weewx/loop" --> Works fine, I am able to see the values updating
3. Tested from remote with an Web MQTT Test Client to register on my external IP (Port forwarding activated) with "weather/weewx/loop" --> Works fine, I am able to see the values updating

If I now activate again the MQTT Support for the Skin with the following extras, then I only see "connecting to..." on the webpage and the values are not updating.

        [[[Extras]]]
            mqtt_enabled = 1
            mqtt_host = "external_ip"
            mqtt_topic = "weather/weewx/loop"
            mqtt_port = 9001
            mqtt_ssl = 0

I have no idea where the issue could be,...

[ebnerjoh]

I think I know now why it is not working.

I have setup MQTT and Websocket without SSL and Google is complaining on that.

[poblabs]

Makes sense if your website is https.

…

On Sat, Feb 2, 2019, 10:21 AM ebnerjoh ***@***.***> wrote: I think I know now why it is not working. I have setup MQTT and Websocket without SSL and Google is complaining on that. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#45 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADUsZwi9MwsIz542NtsfEsSZRCBnb3ihks5vJazqgaJpZM4aIEuf> .

[ebnerjoh]

I think I got it now working.

I was setting up an NGINX ReverseProxy with Letsencrypt with the following options:

    location / {
        proxy_pass http://192.168.1.253:9001/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
    }

So on my public IP I am listening on 443, nginx+letsencrypt is enabling SSL and then the connection is forwared to my MQTT Websocket on 9001.

So far it seems that it is working and also solving my issue, when the client is behind a Firewall which only allows 443 and not 9001.

But I still have two comments:

1. Would it be possible to set a Timestamp of the last received data on the connection string when the connection fails? Currently it only shows a timestamp when it is successfully connected.

2. Is your template supporting username/password for the MQTT Connection?

Br,
Johannes

[poblabs]

Great news! (Off-topic I recently started playing with sslh which is a traffic manager for 443 so you can have multiple things on the same port - sounds like you're sort of the same path as that.)

1. Sure. I have the timestamp in the JavaScript debug console within the browser. I should be able to move it up into the visible page.
2. Not today because typically websockets are available as an anonymous user. So the MQTT broker should have it so anonymous users do not have write and only read access. Plus if you used authentication, anyone who reads your source code would have access to your username and password anyways.

But, some public brokers may not allow anonymous access, and require you to make a readonly account for your topics. So, according to this link, it should be doable. I'd have to investigate making it an option for the skin since not everyone may require it.

[poblabs]

Item 1 is all set and is in the beta test for 0.9. Keep an eye on the Releases page for when 0.9 is released. Hoping to wrap it up in the next few days.