Align with 080 main

README.md

@@ -1,19 +1,33 @@
 ![GitHub release (with filter)](https://img.shields.io/github/v/release/vingerha/gazpar_2_mqtt)
 
 ## Introduction / status
-
-Reworked from the great repo by yukulehe/gazpar2mqtt (who also provided a large part of the docu), now that GRDF is again without Captcha.
-Main differences are in the login method, now using virtual browser (old method still does not work), and allowing to export to HA Long Term Statistics
-- It is working as a container (tested by 3 people) on collecting data in SQLite, MQTT, InfluxDb, Home Assistant sensor and HA energy-dashboard
-- It is working as an add on (tested by myself and 1 other person)
-- no verification if the now 2+-year old code from yukulehe/gazpar2mqtt is still valid **in its entirety**, bits and pieces may not work perfectly anylonger
-- Not yet tested
-  - Grafana dashboard template
-  - Cost calculation from prices file
+Reworked from the great repo by yukulehe/gazpar2mqtt. Main differences are in the login method, now using virtual browser and allowing to export to HA Long Term Statistics
+- Can be installed as an addon and container; collecting data in SQLite, MQTT, InfluxDb, Home Assistant sensor and HA energy-dashboard (LTS)
+- no 100% verification if the now 2+-year old source from yukulehe/gazpar2mqtt is still valid **in its entirety**, bits and pieces may not work perfectly anylonger
+- Not yet tested: Grafana dashboard template, Cost calculation from prices file
 
 For usage and installation etc. see [DOCUMENTATION](https://github.com/vingerha/gazpar_2_mqtt/wiki)
 
 ## Changelogs :
+- 0.8.0
+  Extending LTS sensors with kWh and price
+  - sensor.[device]_[pce-alias]_consumption_stat : for daily figures in m3
+  - sensor.[device]_[pce-alias]_consumption_kwh_stat : for daily figures in kWh
+  - sensor.[device]_[pce-alias]_consumption_cost_stat : daily cost
+  - sensor.[device]_[pce-alias]_consumption_pub_stat : for periodic figures in m3
+  - sensor.[device]_[pce-alias]_consumption_kwh_pub_stat : for periodic figures in kWh
+  - sensor.[device]_[pce-alias]_consumption_cost_pub_stat : periodic cost
+- 0.7.0 
+  The LTS sensor name can no longer be chosen and is fixed to 
+  - sensor.[device]_[pce-alias]_consumption_stat : for daily figures
+  - sensor.[device]_[pce-alias]_consumption_pub_stat : for periodically 'published' figures 
+  Reason: previously the LTS sensors were added without a regular sensor, this makes their use impossible for e.g. apexcharts who uses the regular sensor-name also for statistics.
+  These 2 sensors will appear both in HA as in HA statistics
+
+- 0.6.5
+  - Allow to select a date from which to collect data from GRDF (max 3y back in time)
+  - Allow import of published measures into Long Term Statistics
+  - Allow to delete Long Term Ststaistics for all PCE
 - 0.6.0
   - Fix issue with double naming in the HA sensor
   - Fix issue with incorrect device_classes for the sensors

VERSION

@@ -1 +1 @@
-0.6.0
+0.8.0

addon/CHANGELOG.md

@@ -1,3 +1,18 @@
+## 0.8.0 (12-05-2024)
+Extending LTS sensors with kWh and price
+sensor.[device]_[pce-alias]_consumption_stat : for daily figures in m3
+sensor.[device]_[pce-alias]_consumption_kwh_stat : for daily figures in kWh
+sensor.[device]_[pce-alias]_consumption_cost_stat : daily cost
+sensor.[device]_[pce-alias]_consumption_pub_stat : for periodic figures in m3
+sensor.[device]_[pce-alias]_consumption_kwh_pub_stat : for periodic figures in kWh
+sensor.[device]_[pce-alias]_consumption_cost_pub_stat : periodic cost
+
+## 0.7.0 (10-05-2024)
+The LTS sensor name can no longer be chosen and is fixed to :
+
+sensor.[device]_[pce-alias]_consumption_stat : for daily figures
+sensor.[device]_[pce-alias]_consumption_stat_pub : for periodically 'published' figures 
+
 ## 0.6.5 (09-05-2024)
 
 Allow import of published measures into Long Term Statistics

addon/Dockerfile

@@ -27,7 +27,7 @@ CMD ["python3", "app/gazpar2mqtt.py"]
 ############
 
 LABEL \
-  io.hass.version="0.6.0" \
+  io.hass.version="0.8.0" \
   io.hass.type="addon" \
   io.hass.arch="armv7|amd64|arm64"
 

addon/README.md

@@ -2,11 +2,10 @@
 
 ## Introduction / status
 
-Reworked from the great repo by yukulehe/gazpar2mqtt (who also provided a large part of the docu).
-Main differences are in the login method, now using virtual browser (old method still does not work), and allowing to export to HA Long Term Statistics
+Reworked from the great repo by yukulehe/gazpar2mqtt.
+Main differences are in the login method and allowing to export to HA Long Term Statistics, values and prices
 - no verification if the now 2+-year old code from yukulehe/gazpar2mqtt is still valid **in its entirety**, bits and pieces may not work perfectly anylonger
 - Not yet tested
   - Grafana dashboard template
-  - Cost calculation from prices file
 
 For usage and installation etc. see [DOCUMENTATION](https://github.com/vingerha/gazpar_2_mqtt/wiki)

addon/config.yaml

@@ -1,6 +1,6 @@
 name: "Gazpar 2 MQTT"
 description: "Extracts GRDF data into MQTT a.o."
-version: 0.6.5
+version: 0.8.0
 slug: "gazpar_2_mqtt"
 init: false
 homeassistant_api: true

app/database.py

@@ -77,11 +77,15 @@ def init(self,g2mVersion,dbVersion,influxVersion):
                         pce TEXT NOT NULL
                         , type TEXT NOT NULL
                         , date TEXT NOT NULL
+                        , periodStart TEXT NOT NULL
+                        , periodEnd TEXT NOT NULL
                         , start_index INTEGER NOT NULL
                         , end_index INTEGER NOT NULL
                         , volume INTEGER NOT NULL
                         , volumeGrossConsumed REAL NOT NULL
                         , energy INTEGER NOT NULL
+                        , energyGrossConsumed REAL NOT NULL
+                        , price REAL NOT NULL
                         , conversion REAL)''')
     self.cur.execute('''CREATE UNIQUE INDEX IF NOT EXISTS idx_measures_measure
                     ON measures (pce,type,date)''')
@@ -313,12 +317,16 @@ def __init__(self,pce,result):
     self.pce = pce
     self.type = result[1]
     self.date = _convertDate(result[2])
-    self.startIndex = result[3]
-    self.endIndex = result[4]
-    self.volume = result[5]
-    self.volumeGross = result[6]
-    self.energy = result[7]
-    self.conversionFactor = result[8]
+    self.periodStart = _convertDateTime(result[3])
+    self.periodEnd = _convertDateTime(result[4])
+    self.startIndex = result[5]
+    self.endIndex = result[6]
+    self.volume = result[7]
+    self.volumeGross = result[8]
+    self.energy = result[9]
+    self.energyGross = result[10]
+    self.price = result[11]
+    self.conversionFactor = result[12]
 
 # Class Measure
 class Threshold():

app/gazpar.py

@@ -332,12 +332,15 @@ def login(self,username,password, download_folder, screenshot: bool = False, ver
         logging.debug("Using Button 2: %s", re_btn)
         re_btn.click()   
         time.sleep(3)   
+
+        if screenshot:
+            self.get_screenshot("04_screenshot_after_password_button.png")
 
         self.__browser.switch_to.default_content()
         time.sleep(3)
 
         if screenshot:
-            self.get_screenshot("03a_screenshot_after_switch_to_default_content.png")
+            self.get_screenshot("05_screenshot_after_switch_to_default_content.png")
 
         isLoggedIn = True
 
@@ -353,7 +356,7 @@ def login(self,username,password, download_folder, screenshot: bool = False, ver
             logging.debug("Using deny_btn: %s", deny_btn)
 
             if screenshot:
-                self.get_screenshot("05_screenshot_after_deny_button.png")                    
+                self.get_screenshot("06_screenshot_after_deny_button.png")                    
 
             try:
                 self.click_in_view(
@@ -371,7 +374,7 @@ def login(self,username,password, download_folder, screenshot: bool = False, ver
                 pass
 
         if screenshot:
-            self.get_screenshot("06_screenshot_after_connexion_path.png")    
+            self.get_screenshot("07_screenshot_after_connexion_path.png")    
 
 
         # When everything is ok
@@ -509,11 +512,7 @@ def getPceMeasures(self,pce, startDate, endDate, type):
             time.sleep(5)
 
         measureList = resp
-
-        # Update PCE range of date
-        #pce.dailyMeasureStart = startDate
-        #pce.dailyMeasureEnd = endDate
-
+
         if measureList:
 
             for measure in measureList[pce.pceId]["releves"]:
@@ -1053,7 +1052,9 @@ def __init__(self, pce, measure,type):
         self.volumeGross = None
         self.volumeInitial = None
         self.energy = None
+        self.energyGross = 0
         self.temperature = None
+        self.price = 0
         self.conversionFactor = None
         self.pce = None
         self.isDeltaIndex = False
@@ -1073,6 +1074,8 @@ def __init__(self, pce, measure,type):
         if measure["energieConsomme"]: self.energy = int(measure["energieConsomme"])
         if measure["temperature"]: self.temperature = float(measure["temperature"])
         if measure["coeffConversion"]: self.conversionFactor = float(measure["coeffConversion"])
+        if measure["coeffConversion"] and measure["volumeBrutConsomme"]: 
+            self.energyGross = float(measure["volumeBrutConsomme"]) * float(measure["coeffConversion"])
         self.pce = pce
 
         # Fix informative volume and energy provided when required
@@ -1103,9 +1106,9 @@ def store(self,db):
             dbTable = "consumption_published"
 
         if self.isOk() and dbTable:
-            logging.debug("Store measure type %s, %s, %s, %s, %s m3, %s m3, %s kWh, %s kwh/m3",self.type,str(self.gasDate),str(self.startIndex),str(self.endIndex), str(self.volume), str(self.volumeGross), str(self.energy), str(self.conversionFactor))
-            measure_query = f"INSERT OR REPLACE INTO measures VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)"
-            db.cur.execute(measure_query, [self.pce.pceId, self.type, self.gasDate, self.startIndex, self.endIndex, self.volume, self.volumeGross, self.energy, self.conversionFactor])
+            logging.debug("Store measure type %s, %s,%s,%s, %s, %s, %s m3, %s m3, %s kWh, %s kWh, %s EUR, %s kwh/m3",self.type,str(self.gasDate),str(self.startDateTime), str(self.endDateTime),str(self.startIndex),str(self.endIndex), str(self.volume), str(self.volumeGross), str(self.energy), str(self.energyGross), self.price, str(self.conversionFactor))
+            measure_query = f"INSERT OR REPLACE INTO measures VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"
+            db.cur.execute(measure_query, [self.pce.pceId, self.type, self.gasDate, self.startDateTime, self.endDateTime, self.startIndex, self.endIndex, self.volume, self.volumeGross, self.energy, self.energyGross, self.price, self.conversionFactor])
 
 
     # Return measure measure quality status