Add TCP source for lightweight, extended data gathering (#134)

* Add TCP source for lightweight, extended data gathering

Certain Omnik inverters (supposedly with a specific serial number, see
sources below) reply with its statistics as binary data when a "magic"
packet is sent to them on port 8899 over TCP.  This data is more
lightweight than an HTML page or JavaScript source with values embedded,
and provides more statistics then either similar to what can be found on
the LCD display.

The data format and meaning has been extracted from various Python/C
sources (see links below), with some of my own "work" to simplify
checksum calculation and "the reverse hexadecimal serial number" is
nothing more than its little-endian byte representation.  Response data
is parsed at once with a `ctypes.BigEndianStructure` instead of
unpacking individual ranges of bytes through `struct.unpack`.  Only
minimal data massaging is needed to convert the struct to useful
information.

It provides the following extra fields:

- Inverter temperature;
- Voltage and current for the DC input strings (up to 3);
- Voltage, current, frequency and power for all AC outputs (also up to
  3);
- Total number of runtime hours (not sure if this is just the Omnik
  being on, or the time it delivers >0W of power back to the network).

No "device" (network module) data is found in any of the unparsed bits,
nor is `solar_rater_power` available.

Sources:

1. https://github.com/jbouwh/omnikdatalogger/blob/dev/apps/omnikdatalogger/omnik/InverterMsg.py
2. http://www.mb200d.nl/wordpress/2015/11/omniksol-4k-tl-wifi-kit/#more-590
3. https://github.com/Woutrrr/Omnik-Data-Logger
4. https://github.com/t3kpunk/Omniksol-PV-Logger

* tcp: Replace print() with warning() logging

* tcp: Reimplement message parsing to support length, type, and multiple messages

* tests: Separate TCP packet tests into individual functions

* tests: Simplify "Failed to open a TCP connection" test

This variant breaks immediately, instead of blocking for a minute.

* tests: Use OmnikInverterConnectionError type directly

* tcp,tests: read()/recv() exceptions are (re-?)raised in wait_closed()

* Move LOGGER setup to new const.py

* examples: Move TCP changes to a new example

* examples: Replace deprecated get_event_loop() with asyncio.run()

* Remove assert from example again

* Remove now-unused import from dependency

README.md

@@ -35,7 +35,7 @@ A python package with which you can read the data from your Omnik Inverter. Keep
 | Omnik    | Omniksol 2000TL  | JS         |
 | Omnik    | Omniksol 2000TL2 | JSON       |
 | Omnik    | Omniksol 2500TL  | HTML       |
-| Omnik    | Omniksol 3000TL  | JS         |
+| Omnik    | Omniksol 3000TL  | TCP        |
 | Omnik    | Omniksol 4000TL2 | JS         |
 | Ginlong  | Solis-DLS-WiFi   | JSON/HTML  |
 | Hosola   | 1500TL           | JS         |
@@ -72,11 +72,10 @@ async def main():
 
 
 if __name__ == "__main__":
-    loop = asyncio.get_event_loop()
-    loop.run_until_complete(main())
+    asyncio.run(main())
 ```
 
-For the **source type** you can choose between: `javascript` (default), `json` and `html`.
+For the **source type** you can choose between: `javascript` (default), `json`, `html` and `tcp`.
 
 ## Data
 
@@ -93,6 +92,13 @@ You can read the following data with this package:
 - Day Energy Production (kWh)
 - Total Energy Production (kWh)
 
+On the `tcp` source type you can also find:
+
+- Inverter temperature;
+- Voltage and current for the DC input strings (up to 3)
+- Voltage, current, frequency and power for all AC outputs (also up to 3)
+- Total number of runtime hours.
+
 ### Device
 
 - Signal Quality (only with JS)

examples/test_output.py

@@ -7,7 +7,7 @@
 
 
 async def main() -> None:
-    """Test."""
+    """Locally gather statistics using JavaScript."""
     async with OmnikInverter(
         host="examples.com",
         source_type="javascript",
@@ -35,5 +35,4 @@ async def main() -> None:
 
 
 if __name__ == "__main__":
-    loop = asyncio.get_event_loop()
-    loop.run_until_complete(main())
+    asyncio.run(main())

examples/test_output_tcp.py

@@ -0,0 +1,60 @@
+# pylint: disable=W0621
+"""Asynchronous TCP Python client for the Omnik Inverter."""
+
+import asyncio
+import logging
+
+from omnikinverter import Inverter, OmnikInverter
+
+logging.basicConfig(level=logging.DEBUG)
+
+
+async def main() -> None:
+    """Locally gather statistics over TCP."""
+    async with OmnikInverter(
+        host="examples.com",
+        source_type="tcp",
+        serial_number=123456789,
+    ) as client:
+        inverter: Inverter = await client.inverter()
+        # TCP backend (currently?) doesn't provide WiFi module statistics,
+        # so we're not querying .device() here.
+        print(inverter)
+        print()
+        print("-- INVERTER --")
+        print(f"Serial Number: {inverter.serial_number}")
+        print(f"Model: {inverter.model}")
+        print(f"Firmware Main: {inverter.firmware}")
+        print(f"Firmware Slave: {inverter.firmware_slave}")
+        print(f"Current Power: {inverter.solar_current_power}W")
+        print(f"Energy Production Today: {inverter.solar_energy_today}kWh")
+        print(f"Energy Production Total: {inverter.solar_energy_total}kWh")
+
+        optional_fields = [
+            ("solar_rated_power", "Rated Power", "W"),
+            ("temperature", "Temperature", "°C"),
+            ("solar_hours_total", "Solar Hours Total", "h"),
+            ("dc_input_voltage", "DC Input Voltage", "V"),
+            ("dc_input_current", "DC Input Current", "A"),
+            ("ac_output_voltage", "AC Output Voltage", "V"),
+            ("ac_output_current", "AC Output Current", "A"),
+            ("ac_output_frequency", "AC Output Frequency", "Hz"),
+            ("ac_output_power", "AC Output Power", "W"),
+            ("inverter_active", "Inverter Active", ""),
+        ]
+
+        for field, name, unit in optional_fields:
+            if (val := getattr(inverter, field)) is not None:
+                if isinstance(val, list):
+                    values = ", ".join(
+                        f"{v2:.1f}{unit}" for v2 in val if v2 is not None
+                    )
+                    print(f"{name}: {values}")
+                elif isinstance(val, float):
+                    print(f"{name}: {val:.1f}{unit}")
+                else:
+                    print(f"{name}: {val}{unit}")
+
+
+if __name__ == "__main__":
+    asyncio.run(main())

omnikinverter/const.py

@@ -0,0 +1,4 @@
+"""Constants for the Omnik Inverter."""
+import logging
+
+LOGGER = logging.getLogger(__package__)

omnikinverter/exceptions.py

@@ -24,3 +24,11 @@ class OmnikInverterWrongValuesError(OmnikInverterError):
     shows the same value for both day and
     total production.
     """
+
+
+class OmnikInverterPacketInvalidError(OmnikInverterError):
+    """Omnik Inverter replied with invalid data.
+
+    This error appears on the "tcp" source when received
+    data cannot be validated.
+    """

omnikinverter/models.py

@@ -21,6 +21,20 @@ class Inverter:
     solar_energy_today: float | None
     solar_energy_total: float | None
 
+    # TCP only
+    inverter_active: bool | None = None
+    solar_hours_total: int | None = None
+
+    temperature: float | None = None
+
+    dc_input_voltage: list[float] | None = None
+    dc_input_current: list[float] | None = None
+
+    ac_output_voltage: list[float] | None = None
+    ac_output_current: list[float] | None = None
+    ac_output_frequency: list[float] | None = None
+    ac_output_power: list[float] | None = None
+
     @staticmethod
     def from_json(data: dict[str, Any]) -> Inverter:
         """Return Inverter object from the Omnik Inverter response.
@@ -166,14 +180,34 @@ def get_value(position: int) -> Any:
             solar_energy_total=get_value(7),
         )
 
+    @staticmethod
+    def from_tcp(data: dict[str, Any]) -> Inverter:
+        """Return Inverter object from the Omnik Inverter response.
+
+        Args:
+            data: The binary data from the Omnik Inverter.
+
+        Returns:
+            An Inverter object.
+        """
+
+        return Inverter(
+            **data,
+            model=None,  # Not able to deduce this from raw message yet
+            solar_rated_power=None,
+            solar_current_power=sum(
+                p for p in data["ac_output_power"] if p is not None
+            ),
+        )
+
 
 @dataclass
 class Device:
     """Object representing an Device response from Omnik Inverter."""
 
-    signal_quality: int | None
-    firmware: str | None
-    ip_address: str | None
+    signal_quality: int | None = None
+    firmware: str | None = None
+    ip_address: str | None = None
 
     @staticmethod
     def from_json(data: dict[str, Any]) -> Device:

omnikinverter/omnikinverter.py

@@ -13,6 +13,7 @@
 from aiohttp.hdrs import METH_GET
 from yarl import URL
 
+from . import tcp
 from .exceptions import (
     OmnikInverterAuthError,
     OmnikInverterConnectionError,
@@ -31,9 +32,13 @@ class OmnikInverter:
     source_type: str = "javascript"
     request_timeout: float = 10.0
     session: ClientSession | None = None
+    serial_number: int | None = None  # Optional, only for TCP backend
+    tcp_port: int = 8899
 
     _close_session: bool = False
 
+    _socket_mock: None = None
+
     async def request(
         self,
         uri: str,
@@ -117,35 +122,95 @@ async def request(
 
         return raw_response.decode("ascii", "ignore")
 
+    async def tcp_request(self) -> dict[str, Any]:
+        """Perform a raw TCP request to the Omnik device.
+
+        Returns:
+            A Python dictionary (text) with the response from
+            the Omnik Inverter.
+
+        Raises:
+            OmnikInverterAuthError: Serial number is required to communicate
+                with the Omnik Inverter.
+            OmnikInverterConnectionError: An error occurred while communicating
+                with the Omnik Inverter.
+        """
+        if self.serial_number is None:
+            raise OmnikInverterAuthError("serial_number is missing from the request")
+
+        try:
+            if self._socket_mock is not None:
+                reader, writer = await asyncio.open_connection(sock=self._socket_mock)
+            else:  # pragma: no cover
+                reader, writer = await asyncio.open_connection(self.host, self.tcp_port)
+        except OSError as exception:
+            raise OmnikInverterConnectionError(
+                "Failed to open a TCP connection to the Omnik Inverter device"
+            ) from exception
+
+        try:
+            writer.write(tcp.create_information_request(self.serial_number))
+            await writer.drain()
+
+            raw_msg = await reader.read(1024)
+        finally:
+            writer.close()
+            try:
+                await writer.wait_closed()
+            except OSError as exception:
+                raise OmnikInverterConnectionError(
+                    "Failed to communicate with the Omnik Inverter device over TCP"
+                ) from exception
+
+        return tcp.parse_messages(self.serial_number, raw_msg)
+
     async def inverter(self) -> Inverter:
         """Get values from your Omnik Inverter.
 
         Returns:
             A Inverter data object from the Omnik Inverter.
+
+        Raises:
+            OmnikInverterError: Unknown source type.
         """
         if self.source_type == "json":
             data = await self.request("status.json", params={"CMD": "inv_query"})
             return Inverter.from_json(json.loads(data))
         if self.source_type == "html":
             data = await self.request("status.html")
             return Inverter.from_html(data)
-        data = await self.request("js/status.js")
-        return Inverter.from_js(data)
+        if self.source_type == "javascript":
+            data = await self.request("js/status.js")
+            return Inverter.from_js(data)
+        if self.source_type == "tcp":
+            fields = await self.tcp_request()
+            return Inverter.from_tcp(fields)
+
+        raise OmnikInverterError(f"Unknown source type `{self.source_type}`")
 
     async def device(self) -> Device:
         """Get values from the device.
 
         Returns:
-            A Device data object from the Omnik Inverter.
+            A Device data object from the Omnik Inverter. None on the "tcp" source_type.
+
+        Raises:
+            OmnikInverterError: Unknown source type.
         """
         if self.source_type == "json":
             data = await self.request("status.json", params={"CMD": "inv_query"})
             return Device.from_json(json.loads(data))
         if self.source_type == "html":
             data = await self.request("status.html")
             return Device.from_html(data)
-        data = await self.request("js/status.js")
-        return Device.from_js(data)
+        if self.source_type == "javascript":
+            data = await self.request("js/status.js")
+            return Device.from_js(data)
+        if self.source_type == "tcp":
+            # None of the fields are available through a TCP data dump.
+            return Device()
+
+        raise OmnikInverterError(f"Unknown source type `{self.source_type}`")
 
     async def close(self) -> None:
         """Close open client session."""