V1.2.0

1.2.0

README.md

@@ -1,6 +1,16 @@
 ![MQTTk](/mqttk/mqttk_splash.png)
 
 - [Introduction](#introduction)
+- [Features](#features)
+  * [Connection profile management](#connection-profile-management)
+  * [Subscribe interface](#subscribe-interface)
+  * [Publish interface](#publish-interface)
+  * [Topic browser](#topic-browser)
+  * [Log tab](#log-tab)
+  * [Import MQTT.fx configuration](#import-mqttfx-configuration)
+- [Planned features](#planned-features)
+  * [V1.3](#v13)
+  * [V1.4](#v14)
 - [Software dependencies](#software-dependencies)
 - [Installation](#installation)
   * [macOS - app](#macos---app)
@@ -20,12 +30,6 @@
     + [Linux - acquiring and installing the package from source](#linux---acquiring-and-installing-the-package-from-source)
     + [Linux - installing it via pip](#linux---installing-it-via-pip)
     + [Linux - Running MQTTk](#linux---running-mqttk)
-- [Using the app](#using-the-app)
-  * [Features](#features)
-  * [Screenshots](#screenshots)
-  * [Planned features](#planned-features)
-    + [V1.2](#v12)
-    + [V1.3](#v13)
 - [Building the app from source](#building-the-app-from-source)
   * [pypi package](#pypi-package)
   * [macOS appimage](#macos-appimage)
@@ -55,6 +59,103 @@ whoever is interested. The project is written in Tk/ttk. I don't have time to le
 fancy-pancy GUI environment, it was quick and easy to knock out, and it should run on anything
 including the kitchen sink without too much pain.
 
+# Features
+## Connection profile management
+MQTTk allows the user to create and manage multiple connection profiles. For each connection profile, the broker
+configuration, the topics that have been subscribed to along with the associated colour, the topics in which messages 
+have been published and the message templates are saved. From these connection profiles, the broker connection 
+configuration and the associated subscribe/publish history and message templates can be exported and imported 
+separately.
+
+Once a connection has been configured, it can be connected to. Upon successful connection, the different interfaces for
+subscription, publish and topic inspection become available.
+
+The configuration files and logs are saved in the following locations:
+
+**macOS**: `~/Library/ApplicationSupport/MQTTk/`
+
+**Windows**: `%LOCALAPPDATA/MQTTk/`
+
+**Linux:** `~/.config/MQTTk/`
+
+
+Configuration interface
+
+![Configuration interface](/assets/configuration.png)
+
+Export subscribe/publish history interface
+
+![Configuration interface](/assets/export.png)
+
+## Subscribe interface
+On the subscribe tab, topics can be subscribed to. The $SYS topics and both the `#` and `+` wildcards are supported.
+Once subscribed, the messages arriving in the topic(s) are listed in the listbox with the time of their arrival, topic,
+QOS, retained state and their ID in MQTTk. Topics and topic patterns subscribed to get a colour assigned to them, 
+messages arriving in these topis appear in the colour associated to the topic pattern. The colour can be changed on the
+fly and the new colour gets applied to all previous messages that arrived in these topics. Activating the `Autoscroll`
+checkbox will cause the last message that arrived to be selected automatically and its details to be shown immediately
+in the message details section of the interface. Topic subscriptions can be temporarily muted using the `Mute` checkbox
+on the subscription widget.
+
+Once messages arrived, they can be selected in the listbox. Selected message details appear in the lower right part of
+the interface. Here, different decoders are available to quickly decode or format the most common message types in
+the message content textbox. So far, a JSON pretty formatter and a hex decoder are available, but decoders can be
+added in the future on demand.
+
+The `Attempt to decompress` option will try to decompress the payload using the most common compression algorithms
+(currently zlib and bz2 are supported, but these can be extended in the future). 
+
+Messages that have arrived, can be exported in .CSV and .JSON formats. Message payload is exported as unicode text if 
+possible, otherwise it is encoded in base64.
+
+Subscribe interface
+
+![Subscribe interface](/assets/subscribe.png)
+
+## Publish interface
+On this interface, messages can be published. Once a topic is input, a message payload can be inserted, the QoS of the
+message selected and if needed be, the message can be made retained. Once a message is published, the topic will be
+saved in the topic drop-down for future use. If a message or a payload is needed often, the message can be saved as a
+template with a custom name, and published by just a click of the publish button on the message template widget.
+Selecting the message template will fill the relevant fields on the interface so the content can be modified and/or
+saved as a new template.
+
+Publish interface
+
+![Publish interface](/assets/publish.png)
+
+## Topic browser
+The topic browser allows to subscribe to a topic pattern and organises all incoming messages in a tree format, split
+by the `/` in message the topic. The most important message information (time of arrival of the last message, QoS, 
+retained status, payload) are also shown. The message payload is decoded into a string if possible, otherwise it remains
+the bytestring as it arrived. Right clicking on the selected message allows the topic and the payload to  be copied on 
+the clipboard. The `Ignore retained messages` option will ignore all retained messages, only freshly arrived messages 
+will make it into the topic browser.
+
+Topic browser
+
+![Topic browser](/assets/topic_browser.png)
+
+## Log tab
+The log may contain useful information in case something isn't working with the app as expected. The log is also output
+in a file, which is in the same directory as the configuration files. The log tab text will change to `* Log *` when
+error or exception level messages get inserted to it, to indicate an issue. Upon clicking on the tab, the text returns
+to normal.
+
+## Import MQTT.fx configuration
+If MQTT.fx was already installed on the computer, the "MQTT.fx config" option in the "Import" will try to find and
+import it. If MQTTk cannot find it, the file can also be browsed for. This feature has only been tested with my MQTT.fx
+configs and although it worked, there may be config files out there that may fail to import.
+
+# Planned features
+## V1.3
+- Broker stats tab
+
+## V1.4
+- option to encrypt the configuration file and decrypt it at application launch or use an alternative unencrypted config
+in the current session
+- option to encrypt exported broker configuration
+
 # Software dependencies
 The project is written in pure python, powered by the below projects: 
 - [python3.7+](https://www.python.org/)
@@ -76,7 +177,7 @@ like any other apps.
 
 The system may complain about not being able to verify the developer. You can find more information
 about me [here](https://mateszabo.com), so you can verify the developer yourself instead. To run the app, follow
-the instructions provided by apple [here](https://support.apple.com/en-ie/guide/mac-help/mh40616/mac).
+the instructions provided by Apple [here](https://support.apple.com/en-ie/guide/mac-help/mh40616/mac).
 
 ## On macOS from source
 ### macOS Dependencies
@@ -194,56 +295,6 @@ $ mqttk-console
 ```
 command. This will leave a console window, which might provide additional debug information when something goes tits up.
 
-# Using the app
-## Features
-MQTTk allows configuration of multiple connection profiles. Once configured, the brokers
-can be connected to, and you can subscribe to topics. Incoming messages are shown in time, their colour
-can be changed and selected message's payloads displayed. There are currently 3 decoders, plain text, json
-pretty formatter and hex decoder to analyse message data. 
-
-You can also publish messages, save message templates and one-click publish them. You can send messages with
-any QoS and retained messages as well.
-
-So far, I added the most useful message decoding features: JSON pretty formatter and hex decoders. There is also an
-option to attempt to decompress the payload before feeding it into the decoder.
-
-If you used MQTT.fx in the past, MQTTk will try to find and import your config. The connection profiles,
-subscribe and publish history will be imported, as well as the saved message templates.
-
-There is a built-in log feature to show any exceptions/debug information, let me know if you see something
-unusual there.
-
-Connection profiles, subscription and publish history and saved message templates can be exported and imported.
-
-Messages can be dumped into .CSV and .JSON formats. Message payload is exported as unicode text if possible,
-otherwise it is encoded in base64.
-
-## Screenshots
-
-Subscribe interface
-
-![Subscribe interface](/assets/subscribe.png)
-
-Publish interface
-
-![Publish interface](/assets/publish.png)
-
-Configuration interface
-
-![Configuration interface](/assets/configuration.png)
-
-Export subscribe/publish history interface
-
-![Configuration interface](/assets/export.png)
-
-## Planned features
-### V1.2
-- tree-style topic inspector where all incoming messages are organised in a tree and the latest payload is shown
-
-### V1.3
-- Broker stats tab
-- MQTT 5.0 features
-
 # Building the app from source
 ## pypi package
 issue the following command in the project root to build the sdist package.

mqttk.spec

@@ -49,4 +49,4 @@ app = BUNDLE(exe,
              name='mqttk.app',
              icon='mqttk.icns',
              bundle_identifier='com.mateszabo.mqttk',
-             version='1.1.0')
+             version='1.2.0')

mqttk/__init__.py

@@ -1 +1 @@
-__version__ = '1.1.0'
+__version__ = '1.2.0'

mqttk/__main__.py

@@ -43,8 +43,9 @@
 from mqttk.widgets.subscribe_tab import SubscribeTab
 from mqttk.widgets.header_frame import HeaderFrame
 from mqttk.widgets.publish_tab import PublishTab
-from mqttk.constants import CONNECT, DISCONNECT
+from mqttk.constants import CONNECT, DISCONNECT, EVENT_LEVELS
 from mqttk.widgets.log_tab import LogTab
+from mqttk.widgets.topic_browser import TopicBrowser
 from mqttk.widgets.dialogs import AboutDialog, SplashScreen, ConnectionConfigImportExport, SubscribePublishImportExport
 from mqttk.widgets.configuration_dialog import ConfigurationWindow
 from mqttk.config_handler import ConfigHandler
@@ -68,9 +69,10 @@ def __init__(self):
         self.add_message_callback = None
         self.message_queue = []
         self.config_handler = None
+        self.notification_callback = None
 
     def add_message(self, message_level, *args):
-        message = "{} - {} ".format(datetime.now().strftime("%Y/%m/%d, %H:%M:%S.%f"), message_level)
+        message = "{} - {} ".format(datetime.now().strftime("%Y/%m/%d, %H:%M:%S.%f"), EVENT_LEVELS.get(message_level))
         message += ", ".join([str(x) for x in args])
         message += os.linesep
         if self.add_message_callback is None:
@@ -83,22 +85,20 @@ def add_message(self, message_level, *args):
                 self.message_queue = []
             self.add_message_callback(message)
             self.config_handler.add_log_message(message)
+            if 1 < message_level and self.notification_callback is not None:
+                self.notification_callback()
 
     def warning(self, *args):
-        message_level = "[W]"
-        self.add_message(message_level, *args)
+        self.add_message(1, *args)
 
     def error(self, *args):
-        message_level = "[E]"
-        self.add_message(message_level, *args)
+        self.add_message(2, *args)
 
     def exception(self, *args):
-        message_level = "[X]"
-        self.add_message(message_level, *args)
+        self.add_message(3, *args)
 
     def info(self, *args):
-        message_level = "[i]"
-        self.add_message(message_level, *args)
+        self.add_message(0, *args)
 
     def on_paho_log(self, _, __, level, buf):
         if level == MQTT_LOG_INFO:
@@ -232,14 +232,22 @@ def __init__(self, root):
         self.publish_frame = PublishTab(self.tabs, self, self.log, self.style)
         self.tabs.add(self.publish_frame, text="Publish")
 
-        # ====================================== Log tab =========================================================
+        # ====================================== Topic browser tab ====================================================
+
+        self.topic_browser = TopicBrowser(self.tabs, self.config_handler, self.log, root)
+        self.tabs.add(self.topic_browser, text="Topic browser")
+
+        # ====================================== Log tab =============================================================
 
         self.log_tab = LogTab(self.tabs)
+        self.log.notification_callback = self.log_tab.notify
         self.tabs.add(self.log_tab, text="Log")
         self.log.add_message_callback = self.log_tab.add_message
         self.log.info("Logger output live")
+        self.tabs.bind("<<NotebookTabChanged>>", self.on_tab_select)
 
         self.subscribe_frame.interface_toggle(DISCONNECT, None, None)
+        self.topic_browser.interface_toggle(DISCONNECT, None, None)
         self.header_frame.interface_toggle(DISCONNECT)
         self.publish_frame.interface_toggle(DISCONNECT)
 
@@ -249,6 +257,7 @@ def on_client_disconnect(self, notify=None):
         self.subscribe_frame.cleanup_subscriptions()
         try:
             self.subscribe_frame.interface_toggle(DISCONNECT, None, None)
+            self.topic_browser.interface_toggle(DISCONNECT, None, None)
             self.header_frame.interface_toggle(DISCONNECT)
             self.publish_frame.interface_toggle(DISCONNECT)
             self.header_frame.connection_indicator_toggle(DISCONNECT)
@@ -257,11 +266,15 @@ def on_client_disconnect(self, notify=None):
 
     def on_client_connect(self):
         self.subscribe_frame.interface_toggle(CONNECT, self.mqtt_manager, self.header_frame.connection_selector.get())
+        self.topic_browser.interface_toggle(CONNECT, self.mqtt_manager, self.header_frame.connection_selector.get())
         self.publish_frame.interface_toggle(CONNECT, self.mqtt_manager, self.header_frame.connection_selector.get())
         self.header_frame.connection_indicator_toggle(CONNECT)
         self.subscribe_frame.load_subscription_history()
+        self.topic_browser.load_subscription_history()
 
     def on_connect_button(self):
+        if self.header_frame.connection_selector.get() == "":
+            return
         self.header_frame.connection_error_notification["text"] = ""
         self.header_frame.interface_toggle(CONNECT)
         self.config_handler.update_last_used_connection(self.header_frame.connection_selector.get())
@@ -326,10 +339,15 @@ def export_messages(self, format):
 
         output_location = filedialog.asksaveasfilename(initialdir=self.config_handler.get_last_used_directory(),
                                                        title="Export {}".format(format),
-                                                       defaultextension="json" if format == "JSON" else "csv")
+                                                       defaultextension="json" if format == "JSON" else "csv",
+                                                       initialfile="MQTTk_messages_{}".format(time.time()))
+        if output_location == "":
+            self.log.warning("Empty file name on export message (maybe the cancel button was pressed?")
+            return
 
         self.log.info("Exporting messages in {} format to {}".format(format,
                                                                      output_location))
+
         self.config_handler.save_last_used_directory(output_location)
 
         try:
@@ -384,6 +402,10 @@ def import_subscribe_publish(self):
     def export_subscribe_publish(self):
         export_dialog = SubscribePublishImportExport(self.root, self.icon, self.config_handler, self.log, False)
 
+    def on_tab_select(self, *args, **kwargs):
+        if "logtab" in self.tabs.select():
+            self.log_tab.mark_as_read()
+
 
 def main():
     app = App(root)

mqttk/config_handler.py

@@ -19,6 +19,7 @@
 
 import sys
 import os
+import traceback
 from pathlib import Path
 import json
 from tkinter import messagebox
@@ -352,6 +353,7 @@ def get_last_used_directory(self):
         return self.configuration_dict["last_used_directory"]
 
     def save_last_used_directory(self, directory):
+        print(directory, type(directory))
         head, tail = os.path.split(directory)
         self.configuration_dict["last_used_directory"] = head
         self.config_file_manager(SAVE)

mqttk/constants.py

@@ -52,5 +52,12 @@
     5: "Not authorised"
 }
 
+EVENT_LEVELS = {
+    0: "[i]",
+    1: "[W]",
+    2: "[E]",
+    3: "[X]"
+}
+
 
 MQTT_VERSION_LIST = list(PROTOCOL_LOOKUP.keys())

mqttk/widgets/configuration_dialog.py

@@ -311,6 +311,8 @@ def save_current_config(self):
     def browse_file(self, target_entry):
         file_path_name = filedialog.askopenfilename(initialdir=self.config_handler.get_last_used_directory(),
                                                     title="Select CA file")
+        if file_path_name == "":
+            return
         self.config_handler.save_last_used_directory(file_path_name)
         target_entry.delete(0, tk.END)
         target_entry.insert(0, file_path_name)
@@ -333,8 +335,8 @@ def connection_selected(self, connection_name):
             try:
                 self.profiles_widgets[self.currently_selected_connection].on_unselect()
             except Exception as e:
-                self.log.error("Exception deselecting profile widget", e,
-                               self.currently_selected_connection, connection_name)
+                self.log.warning("Exception deselecting profile widget, maybe there wasn't one selected?", e,
+                                 self.currently_selected_connection, connection_name)
             try:
                 self.all_config_state_change("normal")
                 self.currently_selected_connection_dict = self.config_handler.get_connection_config_dict(