Logs in UI

[NickM-27]

Here is a demo of the new logs page

Screen.Recording.2022-12-05.at.11.38.35.AM.mov

[netlify]

✅ Deploy Preview for frigate-docs canceled.

Name	Link
🔨 Latest commit	ee8384e
🔍 Latest deploy log	https://app.netlify.com/sites/frigate-docs/deploys/63921a0334ab3f00086f0923

[NickM-27]

@felipecrs I saw you commenting on just-containers/s6-overlay#252 and it seems you'll likely have a better understanding of this than I do. We need to log python, go2rtc, and nginx (and future services) to a file in memory (want to avoid writing logs to disk). It seems s6 is the best way to do this.

This log file in memory can then be read to show the logs in the UI, if you have any ideas I'd be happy to hear them

[felipecrs]

I have to tinker with it a bit. I'll be back in a few hours with some snippets for you. :)

[NickM-27]

I have to tinker with it a bit. I'll be back in a few hours with some snippets for you. :)

I appreciate it

[felipecrs]

I think it's finally time to upgrade to s6 v3... Logging is simpler with it. I'll push a PR.

[NickM-27]

Will be rebased on dev once #4587 is merged

[felipecrs]

I wonder if we should omit nginx and go2rtc logs from the container stdout now that we have this feature. I believe it may turn the add-on logs page very confusing with everything combined.

I also thought about prefixing the logs on stdout with the service name, but I'm not sure how exactly we can do that (just-containers/s6-overlay#447 (comment)).

[NickM-27]

I wonder if we should omit nginx and go2rtc logs from the container stdout now that we have this feature. I believe it may turn the add-on logs page very confusing with everything combined.

I also thought about prefixing the logs on stdout with the service name, but I'm not sure how exactly we can do that (just-containers/s6-overlay#447 (comment)).

That's a question for @blakeblackshear I don't really have an opinion one way or the other.

[felipecrs]

Well, actually it doesn't matter much. With the new UI users won't have to look at the container logs. Only if they want.

[NickM-27]

Well, actually it doesn't matter much. With the new UI users won't have to look at the container logs. Only if they want.

right, I think that's the goal. Just need to make sure there are no downsides to the UI logs

[felipecrs]

One future improvement I can think about is colorizing. :)

[NickM-27]

I'm seeing the color characters for go2rtc just haven't had time to look into how html can be set to change them to colors. If you know the answer I'm all ears

[felipecrs]

On a quick search I found this: https://www.npmjs.com/package/ansi-to-html

[felipecrs]

It would be really nice if https://github.com/emilast/vscode-logfile-highlighter could be reused outside of VS Code. It can colorize the logs automatically.

[felipecrs]

Oh, there's one thing to have in mind that I only realized now:

Since #4587 the log files are written to memory (/dev/shm). Docker by default allocates 64MB for each container, but that can be increased per container with a CLI flag. Unfortunately, HA doesn't support configuring the shm size.

• Refs Allow the shm-size to be configured for an add-on home-assistant/supervisor#1807
• Refs shm-size mories76/ha-addon-frigate#4, it seems that it was once a problem as well.

So, with the current logic, the log files can take up to 60MB. Is anything else writing data to /dev/shm? We can reduce the log files, like keeping no archives but increasing the log size to 15MB, or leaving it as 10MB (and maybe that's plenty already).

[NickM-27]

That's not quite true, the shm size can be increased for HA addons it just has to be set as one value for all addons. (edit: or maybe that was only for /tmp/cache, I can't remember)

And yes, we have the camera frames for detect stored in /dev/shm so that may be a concern for addon users.

[blakeblackshear]

I don't think we need to keep log archives beyond 10MB. Containers aren't really supposed to retain state like that, and we probably don't ever want to mess with trying to read those archives for the UI either. These files are essentially an in memory buffer for the UI to read logs. If users want a longer history of logs, they should do that by capturing the container logs and storing them on the host. Frigate also has fairly quiet logs normally. Even with 10MB, I suspect we will get support issues with "Bus error" crashes.