-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a base DPS image for users #966
Comments
how does this relate to Stacks? |
First pass, we will have a basic conda image as base image with no other packages installed. |
@chuckwondo @wildintellect Please try |
@sujen1412 and I wrangled things in DIT and saw the GEDI Subsetter image go from 10.4GB using the python:v4.0.0 base image down to 4.31GB using the pared down dps image, so we saw ~60% reduction in size. |
@sujen1412, is there more you need to do before you can push the stripped down DPS base image to OPS? |
Yes, there are a few more tests I need to run to ensure that jobs run successfully on the workers. This is done to test user permissions on directories that are mounted within a container when the job runs. |
Currently, users are using (as base image) a pre-built base_image from https://github.com/MAAP-Project/maap-workspaces/tree/main/base_images.
When an algorithm is submitted from registration we are including additional steps for creating a user ops and re-doing all permissions which takes up a lot of time during registration.
TODO: Move up the work required at minimum by DPS to base images to save time during registration.
Future (not covered in this ticket) : How to optimize space taken by these images.
The text was updated successfully, but these errors were encountered: