You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For an overview of the yin-yang-ranch distributed computer vision pipeline design, see this repo. The essence of the design is that multiple imagenodes send images and event messages to each imagehub. The imagehub uses Python logging to save the imagenode event messages since it is easy to use and allows easy rotation of log files. In the current version of imagehub, the RotatingFileHandler file handler is used, which means that the log files are named like this at each rotation:
As my development of the librarian code progresses, the current naming convention for the event log files presents 2 problems:
It is not easy to find the event text lines for a specific date; the log files have to be searched.
The librarian uses rsync for backups of imagehub data, since it is very good at efficiently backing up only what changes. But with the current RotatingFileHandler, all the log files are backed up every time because every existing log file is renumbered at each rotation.
To address these problems, I am changing the imagehub logging to use the TimedRotatingFileHandler. This is what the log files will look like after the change:
It solves both of the above problems, and, frankly, it should have been done that way from the start. For anyone currently using imagehub and using a program that reads the event log files, this change will require that those programs be modified.
The actual code change is simple and only involves changing 2 lines in the logging section of imagehub.py. Here is the existing code:
My existing librarian code has been changed and I'll be testing it for a few more weeks. The new imagehub code will be merged into the master branch when the librarian testing is done. This issue is a heads up to anyone currently using imagehub that the change is coming. Feel free to comment.
The text was updated successfully, but these errors were encountered:
I have pushed the above mentioned change to the master branch. The imagehub event message log files now rotate at midnight and look like the TimedRoatingFileHandler example above.
For an overview of the yin-yang-ranch distributed computer vision pipeline design, see this repo. The essence of the design is that multiple
imagenodes
send images and event messages to eachimagehub
. Theimagehub
uses Python logging to save theimagenode
event messages since it is easy to use and allows easy rotation of log files. In the current version ofimagehub
, theRotatingFileHandler
file handler is used, which means that the log files are named like this at each rotation:As my development of the
librarian
code progresses, the current naming convention for the event log files presents 2 problems:librarian
uses rsync for backups ofimagehub
data, since it is very good at efficiently backing up only what changes. But with the currentRotatingFileHandler
, all the log files are backed up every time because every existing log file is renumbered at each rotation.To address these problems, I am changing the
imagehub
logging to use theTimedRotatingFileHandler
. This is what the log files will look like after the change:It solves both of the above problems, and, frankly, it should have been done that way from the start. For anyone currently using
imagehub
and using a program that reads the event log files, this change will require that those programs be modified.The actual code change is simple and only involves changing 2 lines in the logging section of
imagehub.py
. Here is the existing code:which becomes:
My existing
librarian
code has been changed and I'll be testing it for a few more weeks. The new imagehub code will be merged into the master branch when thelibrarian
testing is done. This issue is a heads up to anyone currently usingimagehub
that the change is coming. Feel free to comment.The text was updated successfully, but these errors were encountered: