Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

W&B refactor, handle exceptions, CI example #5618

Merged
merged 9 commits into from
Nov 14, 2021

Conversation

AyushExel
Copy link
Contributor

@AyushExel AyushExel commented Nov 11, 2021

This PR:

  • Adds exception handler when calling wandb.log as that is most likely to error out. We can expand the same to other parts based on feedback
  • Adds some useful messages in place
  • Attempts a CI test with a dummy wandb account only meant for CI

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhanced error handling and informative logging for W&B integration.

📊 Key Changes

  • Commented out the direct W&B login command in CI configuration, highlighting a move towards a safer login practice.
  • Added user guidance to install wandb for dataset logging in create_dataset_artifact.
  • Improved the logging experience by replacing direct print statements with LOGGER.info throughout W&B utility functions for more consistent and informative messaging.
  • Implemented error handling during logging to W&B to prevent training disruption and automatically finish the W&B run if errors occur.

🎯 Purpose & Impact

  • 🔒 Increased Security: Obscuring sensitive W&B login details promotes better security practices.
  • 🚀 User Convenience: By prompting users to install wandb, users are more informed about how to enable certain features.
  • 📈 Consistent Logging: Integration of the LOGGER improves the consistency of how information is presented to users.
  • 🛡️ Robustness: The added error handling ensures that training is not halted by logging issues, providing a more resilient training process for users.

@AyushExel
Copy link
Contributor Author

@glenn-jocher can you have a look at this CI test solution. I've created a dummy wandb account and used its API key to log in to wandb from CI workflows. You can replace it with your own CI account if you want to. This is just a demonstration

@glenn-jocher
Copy link
Member

@AyushExel thanks! Looks interesting. I'm going to try to figure out why pre-commit is failing.

@glenn-jocher glenn-jocher changed the title [WIP] W&B: refactor, handle exceptions, try CI fix W&B refactor, handle exceptions, CI example Nov 13, 2021
@glenn-jocher glenn-jocher self-requested a review November 13, 2021 18:54
@glenn-jocher
Copy link
Member

@AyushExel looks good, I've commented the CI to leave as an example for future work, probably in a separate, more in depth CI to come. Is this good to merge?

@AyushExel
Copy link
Contributor Author

@glenn-jocher yes it's good to go for now. We've added a test on wandb side as well so we don't release any update which breaks yolov5. So, a similar test on yolov5's side should be all we need to be foolproof.

@glenn-jocher glenn-jocher merged commit 540ef0d into ultralytics:master Nov 14, 2021
@glenn-jocher
Copy link
Member

@AyushExel PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
* handle exceptions| attempt CI

* update

* Pre-commit manual run

* yaml one-liner

* Update ci-testing.yml

* Comment W&B CI

Leave as example for future separate CI

* Update ci-testing.yml

Co-authored-by: Glenn Jocher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants