-
Notifications
You must be signed in to change notification settings - Fork 62
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ML] Retry pulling of Docker images to work around network issues (#1535
) Adds a function to retry pulling Docker images up to 5 times, and calls it to pre-fetch every Docker image that's used as a "FROM" image by our "docker build" commands. "docker build" will pull the "FROM" image if it's not present on the local machine, but will only try once, so is vulnerable to a transient problem with the Docker registry failing the whole build. By retrying "docker pull" several times before running "docker build" we ensure that the image is present locally when "docker build" runs and hence "docker build" never needs to pull it.
- Loading branch information
1 parent
bd14f42
commit 909adec
Showing
10 changed files
with
74 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
#!/bin/bash | ||
# | ||
# Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
# or more contributor license agreements. Licensed under the Elastic License; | ||
# you may not use this file except in compliance with the Elastic License. | ||
# | ||
|
||
# Pull a docker image, retrying up to 5 times. | ||
# | ||
# Making sure the "FROM" image is present locally before building an image | ||
# based on it removes the risk of a "docker build" failing due to transient | ||
# Docker registry problems. | ||
# | ||
# The argument is the path to the Dockerfile to be built. | ||
function prefetch_docker_image { | ||
DOCKERFILE="$1" | ||
IMAGE=$(grep '^FROM' "$DOCKERFILE" | awk '{ print $2 }' | head -1) | ||
ATTEMPT=0 | ||
MAX_RETRIES=5 | ||
|
||
while true | ||
do | ||
ATTEMPT=$((ATTEMPT+1)) | ||
|
||
if [ $ATTEMPT -gt $MAX_RETRIES ] ; then | ||
echo "Docker pull retries exceeded, aborting." | ||
exit 1 | ||
fi | ||
|
||
if docker pull "$IMAGE" ; then | ||
echo "Docker pull of $IMAGE successful." | ||
break | ||
else | ||
echo "Docker pull of $IMAGE unsuccessful, attempt '$ATTEMPT'." | ||
fi | ||
done | ||
} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters