Skip to content

Commit

Permalink
Updated twice a day
Browse files Browse the repository at this point in the history
  • Loading branch information
ARC-MX committed Jul 3, 2024
1 parent cc9f22d commit 4bfd566
Show file tree
Hide file tree
Showing 6 changed files with 71 additions and 73 deletions.
69 changes: 36 additions & 33 deletions .github/workflows/docker-image.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
name: Docker Image CI

on:
push:
paths:
- 'scripts/**'
- 'Dockerfile-for-github-action'
- 'requirements.txt'
- 'example.env'
- '.github/workflows/docker-image.yml'
release:
types: [published, unpublished, created, edited, released, deleted]
paths:
- 'scripts/**'
- 'Dockerfile-for-github-action'
- 'requirements.txt'
- 'example.env'
- '.github/workflows/docker-image.yml'


jobs:

Expand All @@ -16,29 +18,30 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Log into docker hub registry
run: echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin

- name: Build and push Docker image
run: |
PLATFORMS=linux/arm64,linux/amd64
DOCKER_IMAGE=arcw/sgcc_electricity
docker buildx build --platform $PLATFORMS -t $DOCKER_IMAGE:latest -t $DOCKER_IMAGE:1.4.1 --file Dockerfile-for-github-action --push .
# - name: pull Docker image
# run: |
# # PLATFORMS=linux/arm64,linux/amd64
# docker pull --platform $PLATFORMS mongo:4.4.18
# docker tag mongo:4.4.18 registry.cn-hangzhou.aliyuncs.com/arcw/mongo:4.4.18

- name: Log into Aliyun hub registry and push Docker image
run: |
echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.ALIYUN_USERNAME }} --password-stdin registry.cn-hangzhou.aliyuncs.com
PLATFORMS=linux/arm64,linux/amd64
DOCKER_IMAGE=registry.cn-hangzhou.aliyuncs.com/arcw/sgcc_electricity
docker buildx build --platform $PLATFORMS -t $DOCKER_IMAGE:latest -t $DOCKER_IMAGE:1.4.1 --file Dockerfile-for-github-action --push .
-name: Check out from Git
-uses: actions/checkout@v1
-name: Check if Git tag exists
run: echo "::set-env name=HEAD_TAG::$(git tag --points-at HEAD)"

- uses: actions/checkout@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1

- name: Log into docker hub registry
if: env.HEAD_TAG != ''
run: echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin

- name: Build and push Docker image
if: env.HEAD_TAG != ''
run: |
PLATFORMS=linux/arm64,linux/amd64
DOCKER_IMAGE=arcw/sgcc_electricity
docker buildx build --build-arg VERSION=${{ steps.get_version.outputs.VERSION }} --platform $PLATFORMS -t $DOCKER_IMAGE:latest -t $DOCKER_IMAGE:${{ steps.get_version.outputs.VERSION }} --file Dockerfile-for-github-action --push .
- name: Log into Aliyun hub registry and push Docker image
if: env.HEAD_TAG != ''
run: |
echo ${{ secrets.DOCKER_PASSWORD }} | docker login -u ${{ secrets.ALIYUN_USERNAME }} --password-stdin registry.cn-hangzhou.aliyuncs.com
PLATFORMS=linux/arm64,linux/amd64
DOCKER_IMAGE=registry.cn-hangzhou.aliyuncs.com/arcw/sgcc_electricity
docker buildx build --build-arg VERSION=${{ steps.get_version.outputs.VERSION }} --platform $PLATFORMS -t $DOCKER_IMAGE:latest -t $DOCKER_IMAGE:${{ steps.get_version.outputs.VERSION }} --file Dockerfile-for-github-action --push .
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
**重要说明:**原作者@renhai-lab 已于2023年10将项目归档,原仓库不再更新。这个版本是在原仓库基础上更新的。在此向原作者表达谢意和致敬。验证码识别已经从最开始的在线商业API替换成离线神经网络检测版本,请使用本仓库的同学点个小星星,或者打赏鼓励。
### 支付宝&微信 打赏码

<p align="center">
<img src="assets/Alipay.png" width=200 style="margin-right: 70px";/>
<img src="assets/WeiChatpay.png" width=200/>
</p>

# ⚡️国家电网电力获取

Expand Down Expand Up @@ -360,23 +366,17 @@ sensor:

- 增加近30天每日电量写入数据库(默认mongodb),其他数据库请自行配置。
- 添加配置默认增加近 7 天每日电量写入数据, 可修改为 30 天, 因为国网目前[「要签约智能交费才能看到30天的数据,不然就只能看到7天的」](https://github.com/ARC-MX/sgcc_electricity_new/issues/11#issuecomment-2158973048)
- 将间歇执行设置为定时执行: JOB_START_TIME,24小时制,例如"07:00”则为每天早上7点执行,第一次启动程序如果时间晚于早上7点则会立即执行一次。
- 将间歇执行设置为定时执行: JOB_START_TIME,24小时制,例如"07:00”则为每天早上7点执行,第一次启动程序立即执行一次, 每12小时执行一次
- 给last_daily_usage增加present_date,用来确定更新的是哪一天的电量。一般查询的日期会晚一到两天。
- 对configuration.yaml中自定义实体部分修改。

# 支付宝&微信 打赏码

<center class="half">
<img src="assets/Alipay.png" width=200 style="margin-right: 70px";/>
<img src="assets/WeiChatpay.png" width=200/>
</center>

# 重要修改通知

2024-06-13:SQLite替换MongoDB,原因是python自带SQLite3,不需要额外安装,也不再需要MongoDB镜像

2024-06-13:SQLite替换MongoDB,原因是python自带SQLite3,不需要额外安装,也不再需要MongoDB镜像
2024-07-03:新增每天定时执行两次,添加配置默认增加近 7 天每日电量写入数据, 可修改为 30 天。
TO-DO

- [X] 增加离线滑动验证码识别方案
- [ ] 添加默认推送服务
- [ ] 添加默认推送服务,电费余额不足提醒
- [ ] 。。。
5 changes: 2 additions & 3 deletions example.env
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ HASS_URL="http://localhost:8123/"
HASS_TOKEN="eyxxxxx"

## selenium运行参数
# 任务开始时间,24小时制,例如"07:00”则为每天早上7点执行,第一次启动程序如果时间晚于早上7点则会立即执行一次。
# 任务开始时间,24小时制,例如"07:00”则为每天早上7点执行,第一次启动程序如果时间晚于早上7点则会立即执行一次,每隔12小时执行一次
JOB_START_TIME="07:00"

## 其他默认参数
Expand All @@ -30,8 +30,7 @@ RETRY_TIMES_LIMIT=5
# 登录超时时间,秒
LOGIN_EXPECTED_TIME=60
RETRY_WAIT_TIME_OFFSET_UNIT=10
# 第一次运行等待时间,秒
FIRST_SLEEP_TIME=10


## 日志级别
# 例如“DUBUG”可以查看出错情况
Expand Down
23 changes: 11 additions & 12 deletions scripts/data_fetcher.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from onnx import ONNX
import platform

DEBUG = False
DEBUG = True

def __ease_out_expo(sep):
if sep == 1:
Expand Down Expand Up @@ -186,11 +186,11 @@ def __init__(self, username: str, password: str):

if enable_database_storage:
# 将数据存储到数据库
logging.debug("enable_database_storage为true,将会储存到数据库")
logging.debug("enable_database_storage is true, we will store the data to the database.")
self.client = sqlite3.connect(os.getenv("DB_NAME"))
else:
self.client = None
logging.info("enable_database_storage为false,不会储存到数据库")
logging.info("enable_database_storage is false, we will not store the data to the database.")

self.DRIVER_IMPLICITY_WAIT_TIME = int(os.getenv("DRIVER_IMPLICITY_WAIT_TIME"))
self.RETRY_TIMES_LIMIT = int(os.getenv("RETRY_TIMES_LIMIT"))
Expand All @@ -214,18 +214,18 @@ def connect_user_db(self, user_id):
# 创建数据库
self.connect = self.client
self.connect.cursor()
logging.info(f"数据库: {os.getenv('DB_NAME')} 创建成功")
logging.info(f"Database of {os.getenv('DB_NAME')} created successfully.")
try:
# 创建表名
self.db_name = f"daily{user_id}"
sql = f"CREATE TABLE {self.db_name} (date DATE PRIMARY KEY NOT NULL, usage REAL NOT NULL);"
self.connect.execute(sql)
logging.info(f"创建{self.db_name}成功")
logging.info(f"Table {self.db_name} created successfully")
except BaseException as e:
logging.debug(f"{self.db_name}已存在:{e}")
logging.debug(f"Table {self.db_name} already exists: {e}")
# 如果表已存在,则不会创建
except BaseException as e:
logging.debug(f": {self.db_name} 表已存在:{e}")
logging.debug(f"Table: {self.db_name} already exists:{e}")
finally:
return self.connect

Expand All @@ -236,7 +236,7 @@ def insert_data(self, data:dict):
self.connect.execute(sql)
self.connect.commit()
except BaseException as e:
logging.debug(f"数据更新失败:{e}")
logging.debug(f"Data update failed: {e}")

def fetch(self):
"""the entry, only retry logic here """
Expand Down Expand Up @@ -269,7 +269,7 @@ def _fetch(self):
logging.info(f"Login successfully on {LOGIN_URL}")
time.sleep(self.RETRY_WAIT_TIME_OFFSET_UNIT)
user_id_list = self._get_user_ids(driver)
logging.info(f"将获取{len(user_id_list)}户数据,user_id: {user_id_list}")
logging.info(f"There are {len(user_id_list)} users in total, there user_id is: {user_id_list}")
time.sleep(self.RETRY_WAIT_TIME_OFFSET_UNIT)
balance_list = self._get_electric_balances(driver, user_id_list) #
time.sleep(self.RETRY_WAIT_TIME_OFFSET_UNIT)
Expand All @@ -279,7 +279,6 @@ def _fetch(self):
driver.quit()

logging.info("Webdriver quit after fetching data successfully.")
logging.info("浏览器已退出")
return user_id_list, balance_list, last_daily_date_list, last_daily_usage_list, yearly_charge_list, yearly_usage_list, month_list, month_usage_list, month_charge_list

finally:
Expand Down Expand Up @@ -578,9 +577,9 @@ def save_usage_data(self, driver, user_id):
# 插入到数据库
try:
self.insert_data(dic)
logging.info(f"{day}的用电量{usage}KWh已经成功存入数据库")
logging.info(f"The electricity consumption of {usage}KWh on {day} has been successfully deposited into the database")
except Exception as e:
logging.debug(f"{day}的用电量存入数据库失败,可能已经存在: {str(e)}")
logging.debug(f"The electricity consumption of {day} failed to save to the database, which may already exist: {str(e)}")

self.connect.close()

Expand Down
25 changes: 11 additions & 14 deletions scripts/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
import sys
import time
import traceback
from datetime import datetime
from datetime import datetime,timedelta

import dotenv
import schedule
Expand All @@ -16,36 +16,33 @@

def main():
# 读取 .env 文件
dotenv.load_dotenv()
dotenv.load_dotenv(verbose=True)

try:
PHONE_NUMBER = os.getenv("PHONE_NUMBER")
PASSWORD = os.getenv("PASSWORD")
HASS_URL = os.getenv("HASS_URL")
HASS_TOKEN = os.getenv("HASS_TOKEN")
JOB_START_TIME = os.getenv("JOB_START_TIME")
FIRST_SLEEP_TIME = int(os.getenv("FIRST_SLEEP_TIME"))
LOG_LEVEL = os.getenv("LOG_LEVEL")
VERSION = os.getenv("VERSION")

except Exception as e:
logging.error(f"读取.env文件失败,程序将退出,错误信息为{e}")
logging.error(f"Failing to read the .env file, the program will exit with an error message: {e}.")
sys.exit()

logger_init(LOG_LEVEL)
logging.info("程序开始,当前仓库版本为1.4.1,仓库地址为https://github.com/ARC-MX/sgcc_electricity_new.git")
logging.info(f"The current repository version is {VERSION}, and the repository address is https://github.com/ARC-MX/sgcc_electricity_new.git")

fetcher = DataFetcher(PHONE_NUMBER, PASSWORD)
updator = SensorUpdator(HASS_URL, HASS_TOKEN)
logging.info(f"当前登录的用户名为: {PHONE_NUMBER},homeassistant地址为{HASS_URL},程序将在每天{JOB_START_TIME}执行")
schedule.every().day.at(JOB_START_TIME).do(run_task, fetcher, updator)
logging.info(f"The current logged-in user name is {PHONE_NUMBER}, the homeassistant address is {HASS_URL}, and the program will be executed every day at {JOB_START_TIME}.")

if datetime.now().time() < datetime.strptime(JOB_START_TIME, "%H:%M").time():
logging.info(f"此次为首次运行,当前时间早于 JOB_START_TIME: {JOB_START_TIME}{JOB_START_TIME}再执行!")
schedule.every().day.at(JOB_START_TIME).do(run_task, fetcher, updator)
else:
logging.info(f"此次为首次运行,等待时间(FIRST_SLEEP_TIME)为{FIRST_SLEEP_TIME}秒,可在.env中设置")
time.sleep(FIRST_SLEEP_TIME)
run_task(fetcher, updator)
next_run_time = datetime.strptime(JOB_START_TIME, "%H:%M") + timedelta(hours=12)
logging.info(f'Run job now! The next run will be at {JOB_START_TIME} and {next_run_time.strftime("%H:%M")} every day')
schedule.every().day.at(JOB_START_TIME).do(run_task, fetcher, updator)
schedule.every().day.at(next_run_time.strftime("%H:%M")).do(run_task, fetcher, updator)
run_task(fetcher, updator)

while True:
schedule.run_pending()
Expand Down
2 changes: 1 addition & 1 deletion scripts/onnx.py
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ def get_distance(self,image,draw=False):
prediction, org_img = self._inference(image)
boxes = self.get_boxes(prediction=prediction)
if len(boxes) == 0:
print('没有发现物体')
print('No gaps were detected.')
return 0
else:
if draw:
Expand Down

0 comments on commit 4bfd566

Please sign in to comment.