Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CSV logging to LighterLogger #21

Closed
2 of 6 tasks
ibro45 opened this issue Jan 26, 2023 · 3 comments
Closed
2 of 6 tasks

Add CSV logging to LighterLogger #21

ibro45 opened this issue Jan 26, 2023 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@ibro45
Copy link
Collaborator

ibro45 commented Jan 26, 2023

Description

Ideally, match all the loggers that PyTorch Lightning provides.

  • TensorBoard
  • Wandb
  • Comet
  • MLFlow
  • Neptune
  • CSV logging (@kbressem I remember you have something like this in your trainlib, suggestions are welcome)
@ibro45 ibro45 changed the title Add other loggers other than tensorboard and wand` to LighterLogger Add loggers other than tensorboard and wand to LighterLogger Jan 26, 2023
@ibro45 ibro45 self-assigned this Jan 26, 2023
@kbressem
Copy link
Contributor

kbressem commented Jan 28, 2023

Yes, I log train loss on a step level and metric on an epoch level. This allows to create nice graphs, similar to tensorboard. Also, I save individual metrics per file, but currently they are overwritten each epoch, this could be changed. So it is not really logging but more continuously saving results. Still very helpful for me.

Regarding how I do it. I just pull each metric from the engines state and dump it into a file, separating each value with a comma. I do not use pandas or any external classes/functions from monai or ignite. So nothing fancy.

CSV looks like this:

filename class 1 class 2 class n
/path/to/dicom1 0.87123 0.74623 0.12341
/path/to/dicom2 0.45425 0.84234 0.77412

I could imagine that logging every individual metric on an epoch level would also be helpful, to identify cases where the model has problems.

@ibro45
Copy link
Collaborator Author

ibro45 commented Aug 13, 2023

@surajpaib and I discussed - we will not add support for other types of logging, we will have Tensorboard, W&B, and CSV logging support. Still need to implement the CSV one.

@ibro45 ibro45 changed the title Add loggers other than tensorboard and wand to LighterLogger Add CSV logging to LighterLogger Oct 10, 2023
@ibro45 ibro45 added the enhancement New feature or request label Oct 10, 2023
@ibro45
Copy link
Collaborator Author

ibro45 commented Jan 25, 2024

#93 - LighterLogger removed, relying on PL loggers.

Should use https://lightning.ai/docs/pytorch/stable/extensions/generated/lightning.pytorch.loggers.CSVLogger.html now. Haven't tried it ever. Have you guys?

@ibro45 ibro45 closed this as completed Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants