Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Serialize start and remove event handling #1334

Merged
merged 2 commits into from
Jul 5, 2023

Conversation

martinpitt
Copy link
Member

Don't start pod and container state updates at the same time. They do asynchronous state updates which sometimes step on each other's toes and reverse the real event order.

This mitigates the problem with getting podman API call results in the wrong order (containers/podman#19124); that causes containers to sometimes appear in a non-current state.


See the above podman issue, this can't be perfect. I stress-tested this in #1324, and at least it got a lot better. E.g. in commit 5a28f0d I just got one failure out of 5x3x13 runs (amplification x affected retries x TEST_OS). But it makes runs a lot easier to get green.

 - Factor out the functionality into two helper methods, and use them in
   testPods as well. That had the same bugs as
   testLifecycleOperationsUser in commit 4ba8fc1.
 - The `Pid` field is "0" while the container is not running yet. So
   don't accept that.
 - Sleep in the waiting loop to avoid excessive polling.
Don't start pod and container state updates at the same time. They do
asynchronous state updates which sometimes step on each other's toes and
reverse the real event order.

This mitigates the problem with getting podman API call results in the
wrong order (containers/podman#19124);
that causes containers to sometimes appear in a non-current state.
@martinpitt martinpitt added the flake unstable test label Jul 5, 2023
// HACK: we don't get a pod event when a container in a pod is removed.
// https://github.com/containers/podman/issues/15408
if (event.Actor.Attributes.podId)
this.updatePodAfterEvent(event.Actor.Attributes.podId, system);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This added line is not executed by any test. Details

Copy link
Member

@jelly jelly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find, would have been impossible without the debug logging.

@martinpitt martinpitt merged commit 7df382f into cockpit-project:main Jul 5, 2023
@martinpitt martinpitt deleted the fixes branch July 5, 2023 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flake unstable test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants