sagemathgh-36716: Free space and repeatability for the Docker Hub wor…

…kflow

    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

This PR implements the suggested changes of
sagemath#36385. Furthermore, it enables the
workflow to detect whether the current tag has already been pushed to
Docker Hub. Thus, if the workflow run on pushing the tag fails due to a
timeout this gives us the possibility to have a scheduler try for a
second or third time (I add a cron for Tuesday and Thursday). Such
timeouts (after 6 hours for a job) seem to happen sporadically, even
though each of the both jobs usually succeeds in less than 4 hours.

I also improve the code structure be refactoring to a reusable workflow
`docker_hub.yml`.

I tested the changes in my fork repository. To prevent from overriding
the existing tags `10.2.rc0` and `develop` I temporarily changed the tag
names to `10.2.rc0t` and `developt` for the test runs. Here is a
[successful run](https://github.com/soehms/sage/actions/runs/6814399805)
that created these tags on Docker Hub. To avoid confusion I've removed
them again. A test-run on existing tags which should skip the subsequent
steps is [this
one](https://github.com/soehms/sage/actions/runs/6828167406).

FYI: I also updated the
[README](https://hub.docker.com/r/sagemath/sagemath) on Docker Hub.
Please have a look and make corrections / additions there if necessary.


### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

- [x] The title is concise, informative, and self-explanatory.
- [x] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36716
Reported by: Sebastian Oehms
Reviewer(s): Matthias Köppe, Sebastian Oehms

.github/workflows/docker_hub.yml

@@ -0,0 +1,91 @@
+name: Reusable workflow for Docker Hub images
+
+on:
+  workflow_call:
+    inputs:
+      dockerhub_repository:
+        default: sagemath-dev
+        type: string
+      dockerfile_target:
+        default: make-build
+        type: string
+
+jobs:
+  build-and-push:
+    name: Build Docker image and push to DockerHub
+    runs-on: ubuntu-latest
+    steps:
+      - name: Maximize build disk space
+        uses: easimon/maximize-build-space@v8
+        with:
+          # need space in /var for Docker images
+          root-reserve-mb:      40000
+          remove-dotnet:        true
+          remove-android:       true
+          remove-haskell:       true
+          remove-codeql:        true
+          remove-docker-images: true
+
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Set tag
+        # docker/metadata-action@v4 is not used since we need to distinguish
+        # between latest and develop tags
+        id: set_tag
+        run: |
+          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
+          TAG_NAME=$(git tag --sort=creatordate | tail -1)
+          REPO=${{ inputs.dockerhub_repository }}
+          # see if the tag has already been pushed
+          # if yes then skip following steps
+          URL="https://registry.hub.docker.com/v2/repositories/sagemath/$REPO/tags?page_size=32"
+          LATEST_TAGS=$(curl -L -s $URL | jq '."results"[]["name"]')
+          JOB_DONE=false
+          for i in $LATEST_TAGS; do if [[ $i == \"$TAG_NAME\" ]]; then JOB_DONE=true; break; fi done
+          echo "JOB_DONE=$JOB_DONE" >> $GITHUB_ENV
+
+          if [[ $JOB_DONE == 'false' ]]
+          then
+            TAG="sagemath/$REPO:$TAG_NAME"
+            TAG_LIST="$TAG, sagemath/$REPO:develop"
+            BASE="sagemath/sagemath-dev:$TAG_NAME"
+            echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
+            echo "TAG=$TAG" >> $GITHUB_ENV
+            echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
+            echo "BASE=$BASE" >> $GITHUB_ENV
+          fi
+          df -h
+
+      - name: Update Tag List
+        id: upd_tag_list
+        run: |
+          REPO=${{ inputs.dockerhub_repository }}
+          TAG_LIST="${{ env.TAG_LIST }}, sagemath/$REPO:latest"
+          echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
+        if: "env.JOB_DONE == 'false' && !contains(env.TAG_NAME, 'beta') && !contains(env.TAG_NAME, 'rc')"
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+        if: env.JOB_DONE == 'false'
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+        if: env.JOB_DONE == 'false'
+
+      - name: Build and push make-build
+        uses: docker/build-push-action@v5
+        with:
+          context: .
+          file: docker/Dockerfile
+          target: ${{ inputs.dockerfile_target }}
+          build-args: |
+            MAKE_BUILD=${{ env.BASE }}
+          push: true
+          tags: ${{ env.TAG_LIST }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+        if: env.JOB_DONE == 'false'

.github/workflows/push_to_docker_hub.yml

@@ -7,106 +7,32 @@ on:
       - 'develop'
   push:
     tags:
-      # Just create image on pushing a tag
-      - '*'
+      # Match all release tags including beta, rc
+      - '[0-9]+.[0-9]+'
+      - '[0-9]+.[0-9]+.[0-9]+'
+      - '[0-9]+.[0-9]+.beta[0-9]+'
+      - '[0-9]+.[0-9]+.[0-9]+.beta[0-9]+'
+      - '[0-9]+.[0-9]+.rc[0-9]+'
+      - '[0-9]+.[0-9]+.[0-9]+.rc[0-9]+'
+
+  schedule:
+    # Recover failed runs each Tuesday and Thursday at one o'clock
+    - cron: '0 1 * * 2,4'
 
 jobs:
   sagemath-dev:
-    name: Build Docker image on target make-build and push to DockerHub sagemath-dev
-    # target make-build replaces former sagemath-dev, see https://github.com/sagemath/sage/pull/36047
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v4
-
-      - name: Set tag
-        # docker/metadata-action@v4 is not used since we need to distinguish
-        # between latest and develop tags
-        id: set_tag
-        run: |
-          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
-          TAG_NAME=$(git tag --sort=creatordate | tail -1)
-          TAG="sagemath/sagemath-dev:$TAG_NAME"
-          TAG_LIST="$TAG, sagemath/sagemath-dev:develop"
-          echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
-          echo "TAG=$TAG" >> $GITHUB_ENV
-          echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
-
-      - name: Update Tag List
-        id: upd_tag_list
-        run: |
-          TAG_LIST="${{ env.TAG_LIST }}, sagemath/sagemath-dev:latest"
-          echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
-        if: "!contains(env.TAG_NAME, 'beta') && !contains(env.TAG_NAME, 'rc')"
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@v3
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Build and push make-build
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          file: docker/Dockerfile
-          target: make-build    # see the corresponding header-note
-          push: true
-          tags: ${{ env.TAG_LIST }}
-          cache-from: type=gha
-          cache-to: type=gha,mode=max
+    uses: ./.github/workflows/docker_hub.yml
+    with:
+      # Build from scratch
+      dockerhub_repository: sagemath-dev
+      dockerfile_target: make-build
+    secrets: inherit
 
   sagemath:
     needs: sagemath-dev
-    name: Build Docker image on target sagemath and push to DockerHub sagemath
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v4
-
-      - name: Set tag
-        # docker/metadata-action@v4 is not used since we need to distinguish
-        # between latest and develop tags
-        id: set_tag
-        run: |
-          git fetch --depth=1 origin +refs/tags/*:refs/tags/*
-          TAG_NAME=$(git tag --sort=creatordate | tail -1)
-          TAG="sagemath/sagemath:$TAG_NAME"
-          TAG_LIST="$TAG, sagemath/sagemath:develop"
-          BASE="sagemath/sagemath-dev:$TAG_NAME"
-          echo "TAG_NAME=$TAG_NAME" >> $GITHUB_ENV
-          echo "TAG=$TAG" >> $GITHUB_ENV
-          echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
-          echo "BASE=$BASE" >> $GITHUB_ENV
-
-      - name: Update Tag List
-        id: upd_tag_list
-        run: |
-          TAG_LIST="${{ env.TAG_LIST }}, sagemath/sagemath:latest"
-          echo "TAG_LIST=$TAG_LIST" >> $GITHUB_ENV
-        if: "!contains(env.TAG_NAME, 'beta') && !contains(env.TAG_NAME, 'rc')"
-
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v3
-
-      - name: Login to Docker Hub
-        uses: docker/login-action@v3
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-
-      - name: Build and push sagemath
-        uses: docker/build-push-action@v5
-        with:
-          context: .
-          file: docker/Dockerfile
-          build-args: |
-            MAKE_BUILD=${{ env.BASE }}
-          target: sagemath
-          push: true
-          tags: ${{ env.TAG_LIST }}
-          cache-from: type=gha
-          cache-to: type=gha,mode=max
+    uses: ./.github/workflows/docker_hub.yml
+    with:
+      # Build from sagemath-dev
+      dockerhub_repository: sagemath
+      dockerfile_target: sagemath
+    secrets: inherit