Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retries to CAA image workflow #1339

Closed
wainersm opened this issue Aug 22, 2023 · 0 comments
Closed

Add retries to CAA image workflow #1339

wainersm opened this issue Aug 22, 2023 · 0 comments
Labels
CI Issues related to CI workflows

Comments

@wainersm
Copy link
Member

The CAA image workflow fails rarely (e.g. )https://github.com/confidential-containers/cloud-api-adaptor/actions/runs/5939073597/job/16104756661) and when that happens someone needs to manually re-trigger it, otherwise we might miss some important fixes as it is core to our dev/release pipelines.

That workflow should be like Rocky Balboa: when it is hit and falls, stand up over again to keep fighting! In other words, we could add an auto-retry mechanism to turn it even more reliable.

@wainersm wainersm added the CI Issues related to CI workflows label Aug 22, 2023
wainersm added a commit to wainersm/cc-cloud-api-adaptor that referenced this issue Aug 22, 2023
The CAA image build on push is very important to not fail otherwise
features and fixes don't properly get into devel/release cycle. But
we recently have noticed errors like:

ERROR: failed to solve: DeadlineExceeded: failed to push
quay.io/confidential-containers/cloud-api-adaptor:b277c1ae739ae31892
6ef4f39c88fac16050d13a: no active session for pj7teq3k1b3tqjew1y3tf5mgp:
context deadline exceeded

Error that goes away when the workflow is manually re-trigger after a few
minutes. So this added a retry mechanism that will try again the build
command after two minutes and for the maximum of three tentatives.

Fixes confidential-containers#1339
Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
wainersm added a commit to wainersm/cc-cloud-api-adaptor that referenced this issue Sep 5, 2023
The CAA image build on push is very important to not fail otherwise
features and fixes don't properly get into devel/release cycle. But
we recently have noticed errors like:

ERROR: failed to solve: DeadlineExceeded: failed to push
quay.io/confidential-containers/cloud-api-adaptor:ee0f4cd8a67e41008b
6ef4f39c88fac16050d13a: no active session for pj7teq3k1b3tqjew1y3tf5mgp:
context deadline exceeded

Error that goes away when the workflow is manually re-trigger after a few
minutes. So this added a retry mechanism that will try again the build
command after two minutes and for the maximum of three tentatives.

Fixes confidential-containers#1339
Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
lysliu pushed a commit to lysliu/cloud-api-adaptor that referenced this issue Nov 9, 2023
The CAA image build on push is very important to not fail otherwise
features and fixes don't properly get into devel/release cycle. But
we recently have noticed errors like:

ERROR: failed to solve: DeadlineExceeded: failed to push
quay.io/confidential-containers/cloud-api-adaptor:b277c1ae739ae31892
6ef4f39c88fac16050d13a: no active session for pj7teq3k1b3tqjew1y3tf5mgp:
context deadline exceeded

Error that goes away when the workflow is manually re-trigger after a few
minutes. So this added a retry mechanism that will try again the build
command after two minutes and for the maximum of three tentatives.

Fixes confidential-containers#1339
Signed-off-by: Wainer dos Santos Moschetta <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Issues related to CI workflows
Projects
None yet
Development

No branches or pull requests

1 participant