Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable tolerance to timed out poll requests #579

Closed
raliev12 opened this issue Jun 14, 2021 · 1 comment
Closed

Enable tolerance to timed out poll requests #579

raliev12 opened this issue Jun 14, 2021 · 1 comment
Assignees
Labels
enhancement New feature or request priority/medium Default priority for items size m
Milestone

Comments

@raliev12
Copy link
Contributor

Problem description (if applicable)
There is quite a common situation when apply process interrupts due to timed out poll request, i.e.:

[airshipctl] 2021/06/01 21:09:28 opendev.org/airship/airshipctl/pkg/events/processor.go:84: Received error when applying resources to kubernetes: polling for status failed: etcdserver: request timed out Error events received on channel, errors are: [polling for status failed: etcdserver: request timed out]

However it's not necessary to completely interrupt apply process due to this error, we can try to poll resource's status once again before giving up. In most cases, the next request completes without an error. We can observe such errors in our CI jobs:

https://jenkins.nc.opensource.att.com/job/development/job/Airshipctl/8927/consoleFull
https://zuul.opendev.org/t/openstack/build/336b136065ef4daa95bcc1fc15f1ca5c

Proposed change
We can filter specific errors we want, and therefore apply process won't be interrupted unless we accumulate threshold of these errors (like timed out). To do this, we need to make some changes in our poller implementation and actually use it (right now we use the default one).

Potential impacts
None.

@raliev12 raliev12 added enhancement New feature or request triage Needs evaluation by project members labels Jun 14, 2021
@raliev12
Copy link
Contributor Author

@jezogwza jezogwza added priority/medium Default priority for items and removed triage Needs evaluation by project members labels Jun 16, 2021
@jezogwza jezogwza added this to the v2.1 milestone Jun 16, 2021
@eak13 eak13 added the size m label Jun 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority/medium Default priority for items size m
Projects
None yet
Development

No branches or pull requests

3 participants