Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make CONFIG_DIR configurable per environment variable #4727

Merged

Conversation

joaodiogocosta
Copy link
Contributor

@joaodiogocosta joaodiogocosta commented Sep 9, 2021

WHY

CONFIG_DIR, which is set to general.user_config_dir() is currently hardcoded to be prefixed with Path.home().

This change breaks all calls to YOLOV5 in environments that don't allow disk writes to /home/{...}, such as AWS Lambda and Google Cloud Functions.

CONFIG_DIR = user_config_dir() # Ultralytics settings dir

yolov5/utils/general.py

Lines 106 to 112 in 0d8a184

def user_config_dir(dir='Ultralytics'):
# Return path of user configuration directory (make if necessary)
settings = {'Windows': 'AppData/Roaming', 'Linux': '.config', 'Darwin': 'Library/Application Support'}
path = Path.home() / settings.get(platform.system(), '') / dir
if not path.is_dir():
path.mkdir() # make dir if required
return path

While the subdirectory takes into consideration the host OS defaults for configuration folders, there are systems in which /home/{...} is not writable.

This can happen in any system really, depending on how filesystem access is configured. However, this issue becomes more prevalent for Docker containers, especially in environments like AWS Lambda and Google Cloud Functions.

These kind of environments don't allow disk writes to any directory other than /tmp. Functions are executed in a protected sandbox environment in which all disk writes must happen within /tmp.

Any attempt to write to directories other than /tmp, will throw an exception like the following:

Errno 30] Read-only file system: './.config/folder': OSError
Traceback (most recent call last):
...

HOW

This Pull Request attempts to fix this hardcoding issue by introducing a configuration environment variable YOLOV5_CONFIG_DIR that can be set in the host environment or on a per execution basis.

This approach is similar to what pytorch does with TORCH_HOME.

Matplotlib is another example with their environment variable MPLCONFIGDIR.

I'm happy to change the variable name as you see fit, or switch to a different approach we find a better one.

CLOSING NOTES

I hope this small patch makes life easier for those who are running YOLOV5 in such an environment.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhancement of configuration directory flexibility for YOLOv5.

📊 Key Changes

  • Imported the os module to the plots.py file.
  • Modified CONFIG_DIR to prioritize the YOLOV5_CONFIG_DIR environment variable if set, over the default user configuration directory.

🎯 Purpose & Impact

  • Purpose: This change allows users to specify a custom configuration directory through an environment variable (YOLOV5_CONFIG_DIR), providing greater flexibility and control in different environments or use cases.
  • Impact: Users who need to customize where YOLOv5 settings are stored can now do so easily, which is especially useful for complex deployment scenarios or shared systems. This does not affect users who do not use this environment variable; for them, the default behavior remains unchanged.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👋 Hello @joaodiogocosta, thank you for submitting a 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:

  • ✅ Verify your PR is up-to-date with origin/master. If your PR is behind origin/master an automatic GitHub actions rebase may be attempted by including the /rebase command in a comment body, or by running the following code, replacing 'feature' with the name of your local branch:
git remote add upstream https://github.com/ultralytics/yolov5.git
git fetch upstream
git checkout feature  # <----- replace 'feature' with local branch name
git rebase upstream/master
git push -u origin -f
  • ✅ Verify all Continuous Integration (CI) checks are passing.
  • ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." -Bruce Lee

@joaodiogocosta joaodiogocosta force-pushed the configurable-config-dir branch from e63ab42 to 0b68083 Compare September 9, 2021 13:00
@glenn-jocher glenn-jocher merged commit deb434a into ultralytics:master Sep 9, 2021
@glenn-jocher
Copy link
Member

@joaodiogocosta looks good, thanks for the solution! I also started another PR before seeing this one in #4726. These two don't conflict so we may also merge the second one.

PR is merged. Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐

@joaodiogocosta
Copy link
Contributor Author

Cool! Happy to help ✌️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

User config prefixed by Path.home() breaks in AWS Lambda or any system without /home
2 participants