Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File logging #504

Merged
merged 7 commits into from
Aug 17, 2022
Merged

File logging #504

merged 7 commits into from
Aug 17, 2022

Conversation

mark-prins
Copy link
Collaborator

Closes #344

Logging configurable with the yaml config files - and logging to files with file rollover based on file size.
I've replace env_logger because there is an issue with the Pipe target - which fails silently ( this is last year, but i still have the problem in the latest release )

I was trying to use the custom formatter in env_logger to write using file-rotate which gives better rotation options.. but was worried about performance and the clunky way I had to implement.

So have moved to the async fast_log instead. It doesn't rotate exactly as I'd like - but is good enough I think.
Some notes are in the inline comments - but basically.. a temp file is logged to, and this is split when a size limit is reached, with a max number of files being specified in config. These temp files are created on start, so the history in these is lost when the server starts.

The main log will retain some history - as this is appended to when writing. This also has a max size set to prevent huge logfiles, and will keep a second file too when the size limit is hit. Couldn't see how to create multiple of these extra files though. Ideally I'd like the rotation to be limited to a specified number of files and these to be retained on start.. but what this PR does is good enough for now I think. ( actually, I've retested and there is a temp file retained now?? 🤷 they are temps, so I'd expect them to be removed ) The documentation isn't great for this crate!

@omsupply-bot omsupply-bot bot added back-end Back-end related docs: internal Internal documentation v1 labels Aug 16, 2022
@omsupply-bot omsupply-bot bot added this to the Version 1 milestone Aug 16, 2022
@mark-prins mark-prins requested review from roxy-dao, clemens-msupply and andreievg and removed request for roxy-dao August 16, 2022 00:24
@@ -39,6 +40,7 @@ export const Login = () => {

useEffect(() => {
setPageTitle(`${t('app.login')} | ${t('app')} `);
LocalStorage.removeItem('/auth/error');
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

offroad: I had the server error set, and couldn't shake it! With this - reset the auth error when you load the login page


let (off_switch, off_switch_receiver) = oneshot::channel();
let result = start_server(settings, off_switch_receiver).await;
// off_switch is not needed but we need to keep it alive to prevent it from firing
let _ = off_switch;
result
}

fn logging_init(settings: &LoggingSettings) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably be initialise_logging but I liked thinking of this in a cockney accent.. "it's loggin, innit??"
Could also move this out to a separate file, but thought that it was ok to keep in main 🤷

@github-actions
Copy link

Bundle size difference

Comparing this PR to main

Old size New size Diff
3.9 MB 3.9 MB 41 B (0.00%)

Copy link
Contributor

@clemens-msupply clemens-msupply left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good thanks.

Is fast_log popular enough? As an alternative, there is also file_rotate
https://docs.rs/file-rotate/latest/file_rotate/index.html
but we would need to implement our own Logger to use it.

The Android build is broken right now. There is another comment, I wait for you decision on it and can provide an Android fix.

@@ -26,3 +26,13 @@
# username: "postgres"
# password: "password"
# database_name: "omsupply-database"
# logging:
## one of: Console | File
# destination: Console
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

destination sounds like "file destination" maybe call it logging "mode"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks - will update

@@ -7,6 +8,7 @@ pub struct Settings {
pub server: ServerSettings,
pub database: DatabaseSettings,
pub sync: Option<SyncSettings>,
pub logging: LoggingSettings,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can it be optional Option? i.e. if not specified use the defaults? (don't break existing setups)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I wondered about that. can do - I was thinking that it was nice to have it specified in the config, so that the current logging options were plain to see.

@mark-prins
Copy link
Collaborator Author

Is fast_log popular enough?

Possibly not - it's not hugely used. there are a lot of versions though, which is another indicator.

As an alternative, there is also file_rotate

That's where I started 😆 I like the file handling in it - I was trying to use the env_logger though, and the bug in that threw me. Like you say, we'd have to do more of the implementation ourselves and I thought that there should be a logging crate available already 🤷
The async approach of the fast log appeals too - appending to a temp file and tidying up the file splitting later. Not that we are logging a great deal, but thought it might be helpful.

@mark-prins mark-prins mentioned this pull request Aug 16, 2022
@mark-prins
Copy link
Collaborator Author

thanks for the review @clemens-msupply!
have made the logging settings optional and changed LogDestination to LogMode in eeb8730
( looking at android now.. )

@mark-prins mark-prins removed this from the Version 1 milestone Aug 17, 2022
Copy link
Contributor

@clemens-msupply clemens-msupply left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good thanks!

#[derive(serde::Deserialize, Clone)]
pub struct LoggingSettings {
/// Console (default) | File
pub mode: LogMode,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a Console + File mode ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did wonder about that! will have a quick try and see if I can get that working..

@mark-prins mark-prins merged commit 9236755 into develop Aug 17, 2022
@mark-prins mark-prins deleted the 344-server-logs branch August 17, 2022 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
back-end Back-end related docs: internal Internal documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Server console logs
2 participants