Is HiveMQ MQTT android client an alternative ?

[hipitihop]

I saw mention in original readme about current MQTT client dependency. Is the HiveMQ MQTT client an option ? It seems better maintained.
https://hivemq.github.io/hivemq-mqtt-client/docs/installation/android/ & https://github.com/hivemq/hivemq-mqtt-client

[thanksmister]

The issue with Paho is the Android client has been updated but it’s now incompatible with the Paho library. The Android client for Paho had a background task that was incompatible due to permission changes in Android 9+.

I believe we could fork Paho and make it compatible with the updated MQTT Android client. This should give us a few years reprieve.

We also already support the same commands for HTTP so not all is lost if MQTT is not working on a particular device.

I don’t know about the Hive library, but @TheTimeWalker has already reported on that and he can share his findings.

[TheTimeWalker]

Thanks for making the issue!

While I was looking at the codebase and the known issue, I've been looking around what alternative could be there which could be more maintained. I know that we could fork and work on it manually instead, but it feels like it's quite some work. It makes in my opinion more sense to concentrate our energy to a maintained library.

The HiveMQ client looked like a good choice as it seems to be newer, is being maintained with regular updates (last update being December 2021), supports at minimum Android 4.4 and ticks some nice things like OpenHAB being a user of the library. The documentation also seems simple enough, though I haven't looked too deep into how "drop-in" it is yet.

[thanksmister]

Sounds good to me, I say go for it. I think you will only discover by trying to integrate it into the project.

[hipitihop]

If possible I'd like to see 4.4 support kept but it should not be a show stopper. I have a HTC One X with old Cyanogen Mod which uses Android 4.4.x and LineageOS doesn't seem to support this. So I'm a little stuck. But it should not prevent this project going forward if dropping 4.4 is the only way. At least HiveMQ seems to continue to support it.

[thanksmister]

Is HiveMQ generic? Will it support Home Assistant MQTT or does it require a server side component?

[TheTimeWalker]

As far as I understood, the HiveMQ MQTT client is generic and doesn't have to connect to a HiveMQ server. It can connect to any MQTT 3.11 and 5.0 broker. For Home Assistant you need to use a broker anyway, so it should work with brokers like Mosquitto (the one I personally use).

[hipitihop]

I agree with @TheTimeWalker , I have the same understanding. Any broker or client that complies with the standard should be interchangeable. I also use Mosquitto as my backbone broker. I have Home Assistant, Tasmota and ProjectAlice clients, all integrating using the same broker. I would expect if I decided to use HiveMQ instead of Mosquitto, that things would continue to just work. I would also expect, that HiveMQ client, would be happy to connect to Mosquitto or other brokers complying with the spec/standard

[thanksmister]

Excellent!

[hipitihop]

A further update: I have now done some prelim experiments with wallpanel and pointing it's dashboard URL at my home assistant dashboard (a special minimal one with two buttons), and alas, it stays on "initializing" even though the same url in chrome on same device works. But that's getting off topic and unrelated, but I just wanted to mention it as android 4.4 might have other gotchas for me and I might need something else for basic panel widgets

[thanksmister]

Android SDK does not use the full Chrome browser implementation in its WebView component. Your version of WebView cannot render HA lovelace. Try using a simple url for your tests like google.com.

The solution for you is to not access the lovelace front end but use something simpler like Tileboard to access HA. Tileboard renders well for older versions of WebView.

Alternatively you could try to install a different version of WebView by side loading or from Google Play store, but this only has a small amount of success on older SDKs.

[thanksmister]

WebView compatibility should probably be its own separate issue.

[TheTimeWalker]

Just to give an update to this, I got quite some time yesterday and today and was able to work on this issue. I have made a new branch which replaces Paho with HiveMQ MQTT client.

It was a tiny bit of work replacing some functions to work with the structure of the new library (and honestly the docs are a bit lacking) but connection with MQTT 5.0 to Mosquitto on my end seems to work. There's still work to do but at least it is good to know that the alternative works out.

[sbussinger]

Is the new client intended to fix the MQTT connection issue I'm seeing with Wall Panel? Or should I be creating a new issue? Essentially I can't get Wall Panel to talk to my MQTT server on Home Assistant.

[TheTimeWalker]

Just as a small update: I have done general clean up and testing to see if MQTT works.

Right now, the library creates a MQTT v5 client as opposed to MQTT 3.1.1 with the older Paho client. This allows to connect to any broker that supports MQTT v5. For example, Mosquitto added support for MQTT v5 with 1.6.8. I do plan on adding an option to switch the version between v3.1.1 or v5 for anybody that has an older broker set up that doesn't support v5. This means it will just take a bit more time to reimplement an older client.

As usual, for anybody that can build it locally and wants to try it out, the mentioned branch from before has the latest changes.

[starkillerOG]

@TheTimeWalker thanks for the update!
Could you maybe release a beta version of the new MQTT client?
I can sideload an pre-build app, but I don't know how to build it.

I use a wall-mounted Samsung Galaxy tab A7 with the latest android version and I have frequent crashes of the Wallpanel app/the whole tablet caused by the wallpanel app (it will reboot the tablet).
I have it configured to restart the wallpanel app automatically so it kind of works, but it is not very nice.

[TheTimeWalker]

Sure, that's the plan. I'm first dogfooding myself before publishing as a new version :)

[TheTimeWalker]

It is finally done! The app is now using HiveMQTT and supports both MQTT v3.1.1 and MQTT v5 brokers

I'd be happy if this can be tested, especially on Android 9+ installations: https://github.com/TheTimeWalker/wallpanel-android/releases/tag/v0.10.0.0

Thank you all for your patience!

[npike]

Installed the v0.10.0-beta0 release on my Android 9 tablet. So good so far... but it looks like the MQTT3Service and MQTT5Service both have hardcoded broker URL and ports? Had to tweak those values locally to get everything to connect up.

[TheTimeWalker]

Sorry for the slip up. You absolutely don't have to change the URL and ports locally on your installation to make it work 😄

I have released a new version 0.10.0-beta1 that fixes this: https://github.com/TheTimeWalker/wallpanel-android/releases/tag/v0.10.0.1

[starkillerOG]

@TheTimeWalker I just installed v0.10.0.1 on my samsung galaxy tab A7 running Android 11, with the latest updates of januari 2022 installed.

I did some initial testing and everything seems to be working (just as in the previous versions).

Unfortunately it did crash and rebooted the tablet just now.
I will continue to monitor the tablet in the coming days.

Note that I have motion and face detection enabled on the tablet and a automation in HomeAssistant that turns the screen on when motion or face is detected.
I also have the default screen sleep of the tablet still enabled, so I might test if disabeling that will improve the situation in a few days.

Note that the crashes mostly happen when the tablet has its screen turned off (but of course the majority of time the screen is turned off, so this may not be related).

[npike]

I've been getting some very regular crashes on an Android 9 tablet with this build, seems more frequent than before, but I haven't been able to get a stacktrace yet.

[npike]

If its worth anything, I've had MQTT off (and camera + motion detection on) for 7 days, not a single crash.

[starkillerOG]

@TheTimeWalker could you explain how I can get a stacktrace after the tablet reboots?
Then I can post it here and maybe we can figure out why these crashes are occuring...

[TheTimeWalker]

I have tried to replicate the issue with the app crashing, at least with an Android 10 virtual device, but it seems to work out pretty fine. My Android 8 tablet is also very stable, so either there is an unknown issue with the MQTT library or something else seems to crash this.

@starkillerOG Unfortunately, as far as I know if the whole device reboots there isn't really a way to extract the stacktrace as logcat resets to zero. If there are other ways, I'd be glad if somebody could jump in. One thing that could help is that the app is using Crashlytics which I can see if there are crashes happening. If you haven't blocked crashlyticsreports-pa.googleapis.comvia a network DNS blocker or at least temporarily unblock it, crashes should come in, and I could maybe see what the stacktrace before the reboot happens.

[starkillerOG]

@TheTimeWalker I did not actifly block the crash reports, so I guess they schould be coming in.
crashes are rather unpredictable, sometimes I don't have crashes for 2 days and sometimes it crashes 8 times within 1 hour.
Have not found a specific situation when the crash happens...

[TheTimeWalker]

Hmm, that is very strange. Unfortunately, I haven't received any crash logs yet. One thing, the tablet never in itself reboots, only when WallPanel is running, right?

[starkillerOG]

@TheTimeWalker well I only use the tablet for WallPanel, but no I don't think it reboots withouth WallPanel running.
Where can I check if the crash logs are beeing blocked, is that in the wallpanel settings, tablet settings or router settings?

[starkillerOG]

@TheTimeWalker I just saw that diagnostic information was blocked in the tablets privacy settings, I just turned it on, so hopefully a crash report will be coming in once the next crash occurs. I will let you know when I happen to see it crash again so you can check if the report made it.

[starkillerOG]

@TheTimeWalker I just heard it crash now (14:27 29-04-2022) so hopfully you got a crash report. can you check?

[nodinosaur]

Hi all, I would just like to chip in quickly...

I have recently forked @TheTimeWalker's repo to see if I can compile a version of the app to work with my Nexus 9 running SlimRoms Android 7.1.2 (Nougat) as the most recent version of the app I got from the PlayStore was consistently crashing after a period of time (Version 0.9.5 Build 9).

However, I have found an issue where the App Gradle is set to minSDK 19 (Android 4.4 KitKat) and com.hivemq.client.mqtt requires minSDK 24 (Android 7.0 Lollipop)
This is throwing an IDE error:

Call requires API level 24 (current min is 19): `java.util.Optional#get`

This issue appears in both:
com.thanksmister.iot.wallpanel.network.MQTT5Service
com.thanksmister.iot.wallpanel.network.MQTT3Service

it.publishes(MqttGlobalPublishFilter.ALL) { publish ->
                        listener?.subscriptionMessage(
                            it.config.clientIdentifier.get().toString(), // the Optional is not compatible
                            publish.topic.toString(),
                            Charsets.UTF_8.decode(publish.payload.get()).toString() // the Optional is not compatible
                        )
                    }

Obviously adding the @TargetApi(Build.VERSION_CODES.N) annotation silences the lint, but the problem will still be there.

A potential solution to this would be to fork the HiveMQTT, and replace java.util.Optional with Arrow's Option

[TheTimeWalker]

HiveMQ MQTT specifically mentions that it's compatible with KitKat. It's using RetroFix as the gradle plugin to make it compatible for older devices: https://hivemq.github.io/hivemq-mqtt-client/docs/installation/android/#android-api-levels-below-24

Since the IDE can't know that it's getting backported at compilation time, the linter has to be specifically set to ignore that error. Are you having issues compiling or running on a KitKat device?

[nodinosaur]

I am installing to Nougat and thankfully it is compiling fine.
Good to hear that this is simply a linter issue.