Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to ubuntu 22.04, gdal 3.4, python 3.10 #14

Merged
merged 2 commits into from
Sep 24, 2024
Merged

Conversation

corviday
Copy link
Contributor

@corviday corviday commented Sep 18, 2024

This branch updates ubuntu to 22.04, gdal to 3.4.1, and python to 3.10. It drops support for python 2.x, which gets tricker in ubuntu 22.04, and I don't think we're using it any more.

The goal was to fix the climate explorer backend build. Here is a demo of SCIP running with the fixed climate explorer backend built from this docker.

Aside from climate-explorer-backend, sandpiper seems to be the only repository currently using this container. It appears to be using a custom branch named "gdal3" and probably won't be affected by this update, though since this update has gdal 3.4, perhaps we should see if it can use this one as well so we have one fewer build to maintain.

@corviday corviday marked this pull request as draft September 18, 2024 02:13
@corviday corviday force-pushed the python-310 branch 2 times, most recently from cb5c563 to 576528d Compare September 18, 2024 21:54
@corviday corviday marked this pull request as ready for review September 23, 2024 18:24
@corviday corviday assigned corviday and unassigned corviday Sep 23, 2024
@corviday corviday changed the title test build branch Update to ubuntu 22.04, gdal 3.4, python 3.10 Sep 23, 2024
@rod-glover
Copy link

Aside from climate-explorer-backend, sandpiper seems to be the only repository currently using this container. It appears to be using a custom branch named "gdal3" and probably won't be affected by this update, though since this update has gdal 3.4, perhaps we should see if it can use this one as well so we have one fewer build to maintain.

Excellent idea.

Copy link

@rod-glover rod-glover left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good.

A couple of small items for thought, especially given the remarks on build time, etc., this morning:

  • One is generally speaking encouraged to reduce the number of layers created in a Dockerfile. Each RUN command, amongst others, creates a new layer. Fewer layers can help reduce the final image size. That's why the original Dockerfile had the one RUN command with so many continuation lines.
  • But more important is to order the RUN commands to exploit layer caching. The most reliable and the most expensive RUNs should come early in the Dockerfile, because when you rebuild an image, Docker uses the layer cache to only build what's necessary. If you are fiddling with some item and it can go towards the end of a Dockerfile, it should, because your (re)builds will be much quicker than if it comes early and invalidates caches from there on down. I don't know if this applies here, because naturally there are also constraints imposed by dependency relationships.

@Nospamas
Copy link

Nospamas commented Sep 23, 2024

Echoing Rod's notes. Rolling the apt get and the caching purge into one line makes that layer overall smaller and seems to be the general best practice. In particular, purging caches before finishing the line lets docker finish layer with those caches already purged.

I was also going to suggest looking for a minimal ubuntu images but it looks like ubuntu images less than 30 megs and don't include a slim version. This is in contrast to something like debian which does publish separate slim images.

EDIT: this is also suggested in the best practices docker docs

@corviday corviday merged commit 3065f1d into master Sep 24, 2024
2 checks passed
@corviday corviday deleted the python-310 branch September 24, 2024 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants