Take action on healthcheck failure?

[havenoclu]

Hi Folks - I've been playing around with healthchecks in podman and do not see a way to take an action when a healthcheck fails or meets a certain criteria outside of using something like a script running logic on podman healthcheck run <container> in cron. The action I would like to see is simply restarting the container on a failed healthcheck (maybe if it exhausts the number of retries).

Thoughts on this being a worthy feature request? Thanks :)

[mheon]

@baude Thoughts?

[baude]

well i asked him to start a discussion on this ... so I dont really have thoughts.

[jaymiracola]

https://developers.redhat.com/blog/2019/04/18/monitoring-container-vitality-and-availability-with-podman/ . adding the falg --healthcheck-command or building HEALTHCHECK into the Dockerfile on build should work. Have you tried either?

[jaymiracola]

lol that its @baude article. Would setting HEALTHCHECK and then defining the restart policy not work?

[jaymiracola]

#5680 This looks like exactly what this thread is about . Rebuild that image with buildah using --format=docker as healthchecks are not OCI compliant and are ignored. Thats what I gathered at least!

[mheon]

I'm not opposed to adding the ability to take an action on HC failure. We do similar things for restart policy already, so it should be possible. What actions would we support? Stop container, restart container? Running a user-specified binary would be nice but getting the UI for that right (how do we pass it in by CLI?) could be complicated.

[havenoclu]

@mheon Stop and restart would be fine and better than what I believe is available today. If running something custom turns out to be desired by the masses, maybe that could be a subsequent phase of this enhancement.

[stevenewsted]

I could see some benefit in not only restart/stop, but also to start a different container (ie, so it can throw up some more complex red flags or perform some extra logic).

[ironicbadger]

Executing a script or arbitrary bash command would be insanely useful in this scenario in case there are some esoteric things a user needs to do in the event of a failure (alerting or restarting a service for example).