Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force a CNI Delete on refreshing containers #4086

Merged
merged 1 commit into from
Sep 25, 2019

Conversation

mheon
Copy link
Member

@mheon mheon commented Sep 23, 2019

CNI expects that a DELETE be run before re-creating container networks. If a reboot occurs quickly enough that containers can't stop and clean up, that DELETE never happens, and Podman currently wipes the old network info and thinks the state has been entirely cleared. Unfortunately, that may not be the case on the CNI side. Some things - like IP address reservations - may not have been cleared.

To solve this, manually re-run CNI Delete on refresh. If the container has already been deleted this seems harmless. If not, it should clear lingering state.

Fixes: #3759

@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mheon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS labels Sep 23, 2019
@baude
Copy link
Member

baude commented Sep 23, 2019

lgtm, there will be a perf hit on this though ... i don't know what the implications are and what lead us to calling this code so maybe it wont matter.

Copy link
Member

@TomSweeneyRedHat TomSweeneyRedHat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@vrothberg vrothberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM but CI isn't happy yet:

[+0042s] libpod/container_internal.go:626:25: c.runtime.getPodNetwork undefined (type *Runtime has no field or method getPodNetwork)
[+0042s] make: *** [Makefile:178: podman-remote-darwin] Error 2

@mheon
Copy link
Member Author

mheon commented Sep 24, 2019

Ahh. Windows compilation.

CNI expects that a DELETE be run before re-creating container
networks. If a reboot occurs quickly enough that containers can't
stop and clean up, that DELETE never happens, and Podman
currently wipes the old network info and thinks the state has
been entirely cleared. Unfortunately, that may not be the case on
the CNI side. Some things - like IP address reservations - may
not have been cleared.

To solve this, manually re-run CNI Delete on refresh. If the
container has already been deleted this seems harmless. If not,
it should clear lingering state.

Fixes: containers#3759

Signed-off-by: Matthew Heon <[email protected]>
@mheon
Copy link
Member Author

mheon commented Sep 24, 2019

CI is green now

@rhatdan
Copy link
Member

rhatdan commented Sep 25, 2019

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 25, 2019
@openshift-merge-robot openshift-merge-robot merged commit e4835f6 into containers:master Sep 25, 2019
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 26, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rebooted, dangling file in /var/lib/cni/networks/podman prevents container starting
7 participants