[flake] remote,rootless: create+start+ps: container does not exist in database

[edsantiago]

In window 1:

$ ./bin/podman system service --timeout=0

In window 2:

$ P() { ./bin/podman-remote "$@"; }
$ P create --name test1 --rm alpine foo;P start test1;P ps -aq
a7a5315b7a4e6d9b40e2955774197644fb771730721b4a9be1ac4128d47c242d
Error: executable file not found in $PATH: No such file or directory: OCI runtime command not found error
Error: container a7a5315b7a4e6d9b40e2955774197644fb771730721b4a9be1ac4128d47c242d does not exist in database: no such container

It's a flake (race condition): don't expect it to happen every time. For me it's one out of 3-5 runs on my laptop.

Rootless only, AFAICT: I could not get it to reproduce with P() { sudo ...}.

master @ 5cedd83 on f32.

While I'm at it, the error message is different between podman and podman-remote, does this matter?

podman: Error: unable to start container "911e628f4232690efc7a7efad770491312c94a4a4f8fb164feb53a1b41ef4a8b": executable file not found in $PATH: No such file or directory: OCI runtime command not found error
remote: Error: executable file not found in $PATH: No such file or directory: OCI runtime command not found error

Only seems to happen when creating an error container (foo, i.e. command not found). Does not happen with true ... except, now, as root!

$ P() { sudo ./bin/podman-remote "$@"; }
$ P create --name test1 --rm alpine true;P start test1;P ps -aq
3c9be0ca2306b17a63040c24a4dbea85a283fa4ff080fc3ad5ce5d1852158d2c
test1
Error: container 3c9be0ca2306b17a63040c24a4dbea85a283fa4ff080fc3ad5ce5d1852158d2c does not exist in database: no such container

(very very infrequently; it takes a lot of repetitions to trigger)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@edsantiago Any update on this issue?

[edsantiago]

Issue is still present; failing about one in ten using reproducer in comment 0 (in a while loop):

fe1e51c49e5839fa85e999dd7fdae8b1ff96c139b827e4a530171b592baf86a4
Error: unable to start container "fe1e51c49e5839fa85e999dd7fdae8b1ff96c139b827e4a530171b592baf86a4": executable file `foo` not found in $PATH: No such file or directory: OCI not found
Error: container fe1e51c49e5839fa85e999dd7fdae8b1ff96c139b827e4a530171b592baf86a4 does not exist in database: no such container

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

I would argue that this is a bug in the code, since the remove should not be happening at all. What is going on here, is the container is being removed some of the time. Not sure why it is not always being removed. I think the behaviour should be to leave the container in place. The --rm should only come into play if the start was successful. But this is difficult for us to implement.