Rootless API not usable right after image is deleted #9251

marusak · 2021-02-06T08:33:29Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

In Cockpit tests we saw one test failing way too many times so I played with it. It is timing issue, possibly some race as sleeping just one second resolves this.

Steps to reproduce the issue:

Prerequisites:

When you start make sure that rootless service is not running
Make sure that you don't have fedora:33 image (nor fedora:latest or anything which points to the same sha. podman rmi fedora:33 in the following script needs to physically remove it, if it just untags it, then it works just fine).

Put the following script into file, chmod u+x it and run it as normal user.

set -x

systemctl --user enable --now podman.socket

echo "With stopped service it times out"
podman pull fedora:33
podman rmi fedora:33
timeout 5s curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info

echo "Even though we never got reply from ^ the service is now running"
systemctl --user is-active podman

echo "Now I do the same, without timeout as it is going to fail"
podman pull fedora:33
podman rmi fedora:33
curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info

echo "If I sleep just a tiny bit after doing `podman rmi` it works fine"
podman pull fedora:33
podman rmi fedora:33
sleep 1
curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info

Describe the results you received:

The first API access just hangs and is killed by timeout.
The second API access fails.
The third API access succeeds.

Describe the results you expected:

All API calls succeed.

Output of podman version:

[admin@m1 ~]$ podman version
Version:      3.0.0-dev
API Version:  3.0.0
Go Version:   go1.16rc1
Built:        Wed Feb  3 07:07:25 2021
OS/Arch:      linux/amd64

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.0.0-0.204.dev.gita086f60.fc34.x86_64

But this is not current regression in podman-3, we see this on current F33 as well.

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

The text was updated successfully, but these errors were encountered:

This call should be very quick. It is the first all that cockpit-podman does. Unfortunately often the first call gets stuck, see: containers/podman#8762 containers/podman#9251 When the call is stuck, the whole page is stuck in `Loading...`. When we just reject it, the `Loading...` state moves on and we are presented with alert to 'Start' the service. Clicking the button does the trick. It is kind of hacky, but I think it is more user friendly then just being stuck on a page.

The first rootless call often gets stuck or fails. In such case we have alert banner to start the service (or just empty state). A real user would just hit the button so lets do the same as this is always getting back to us and we waste too much time reporting to podman with mixed results. Examples: containers/podman#8762 containers/podman#9251 containers/podman#6660

This call should be very quick. It is the first all that cockpit-podman does. Unfortunately often the first call gets stuck, see: containers/podman#8762 containers/podman#9251 When the call is stuck, the whole page is stuck in `Loading...`. When we just reject it, the `Loading...` state moves on and we are presented with alert to 'Start' the service. Clicking the button does the trick. It is kind of hacky, but I think it is more user friendly then just being stuck on a page.

The first rootless call often gets stuck or fails. In such case we have alert banner to start the service (or just empty state). A real user would just hit the button so lets do the same as this is always getting back to us and we waste too much time reporting to podman with mixed results. Examples: containers/podman#8762 containers/podman#9251 containers/podman#6660

mheon · 2021-02-09T18:33:40Z

@baude @jwhonce PTAL

baude · 2021-02-19T22:42:05Z

I think my results are correct?

++ systemctl --user enable --now podman.socket
++ echo 'With stopped service it times out'
With stopped service it times out
++ podman pull fedora:33
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:33...
Getting image source signatures
Copying blob 157ab8011454 done  
Copying config 9f2a560376 done  
Writing manifest to image destination
Storing signatures
9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ podman rmi fedora:33
Untagged: registry.fedoraproject.org/fedora:33
Deleted: 9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ timeout 5s curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.4","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.26-1.fc33.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.26, commit: 777074ecdb5e883b9bec233f3630c5e7fa37d521"},"cpus":12,"distribution":{"distribution":"fedora","version":"33"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.10.14-200.fc33.x86_64","memFree":572014592,"memTotal":33388748800,"ociRuntime":{"name":"crun","package":"crun-0.17-1.fc33.x86_64","path":"/usr/bin/crun","version":"crun version 0.17\ncommit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/usr/bin/slirp4netns","package":"slirp4netns-1.1.8-1.fc33.x86_64","version":"slirp4netns version 1.1.8\ncommit: d361001f495417b880f20329121e3aa431a8f90f\nlibslirp: 4.3.1\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":20989861888,"swapTotal":21059592192,"uptime":"128h 56m 23.16s (Approximately 5.33 days)","linkmode":"dynamic"},"store":{"configFile":"/home/baude/.config/containers/storage.conf","containerStore":{"number":0,"paused":0,"running":0,"stopped":0},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/usr/bin/fuse-overlayfs","Package":"fuse-overlayfs-1.4.0-1.fc33.x86_64","Version":"fusermount3 version: 3.9.3\nfuse-overlayfs: version 1.4\nFUSE library version 3.9.3\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/baude/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":0},"runRoot":"/run/user/1000/containers","volumePath":"/home/baude/.local/share/containers/storage/volumes"},"registries":{"search":["registry.fedoraproject.org","registry.access.redhat.com","registry.centos.org","docker.io"]},"version":{"APIVersion":"3.0.0","Version":"3.1.0-dev","GoVersion":"go1.15.8","GitCommit":"5bd648c6eeb1d37c56bcc33458c84ee7af22adbc-dirty","BuiltTime":"Fri Feb 19 14:21:12 2021","Built":1613766072,"OsArch":"linux/amd64"}}
++ echo 'Even though we never got reply from ^ the service is now running'
Even though we never got reply from ^ the service is now running
++ systemctl --user is-active podman
active
++ echo 'Now I do the same, without timeout as it is going to fail'
Now I do the same, without timeout as it is going to fail
++ podman pull fedora:33
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:33...
Getting image source signatures
Copying blob 157ab8011454 done  
Copying config 9f2a560376 done  
Writing manifest to image destination
Storing signatures
9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ podman rmi fedora:33
Untagged: registry.fedoraproject.org/fedora:33
Deleted: 9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.4","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.26-1.fc33.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.26, commit: 777074ecdb5e883b9bec233f3630c5e7fa37d521"},"cpus":12,"distribution":{"distribution":"fedora","version":"33"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.10.14-200.fc33.x86_64","memFree":572370944,"memTotal":33388748800,"ociRuntime":{"name":"crun","package":"crun-0.17-1.fc33.x86_64","path":"/usr/bin/crun","version":"crun version 0.17\ncommit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/usr/bin/slirp4netns","package":"slirp4netns-1.1.8-1.fc33.x86_64","version":"slirp4netns version 1.1.8\ncommit: d361001f495417b880f20329121e3aa431a8f90f\nlibslirp: 4.3.1\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":20989861888,"swapTotal":21059592192,"uptime":"128h 56m 31.4s (Approximately 5.33 days)","linkmode":"dynamic"},"store":{"configFile":"/home/baude/.config/containers/storage.conf","containerStore":{"number":0,"paused":0,"running":0,"stopped":0},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/usr/bin/fuse-overlayfs","Package":"fuse-overlayfs-1.4.0-1.fc33.x86_64","Version":"fusermount3 version: 3.9.3\nfuse-overlayfs: version 1.4\nFUSE library version 3.9.3\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/baude/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":0},"runRoot":"/run/user/1000/containers","volumePath":"/home/baude/.local/share/containers/storage/volumes"},"registries":{"search":["registry.fedoraproject.org","registry.access.redhat.com","registry.centos.org","docker.io"]},"version":{"APIVersion":"3.0.0","Version":"3.1.0-dev","GoVersion":"go1.15.8","GitCommit":"5bd648c6eeb1d37c56bcc33458c84ee7af22adbc-dirty","BuiltTime":"Fri Feb 19 14:21:12 2021","Built":1613766072,"OsArch":"linux/amd64"}}
+++ podman rmi
Error: image name or ID must be specified
++ echo 'If I sleep just a tiny bit after doing  it works fine'
If I sleep just a tiny bit after doing  it works fine
++ podman pull fedora:33
Resolved "fedora" as an alias (/etc/containers/registries.conf.d/shortnames.conf)
Trying to pull registry.fedoraproject.org/fedora:33...
Getting image source signatures
Copying blob 157ab8011454 done  
Copying config 9f2a560376 done  
Writing manifest to image destination
Storing signatures
9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ podman rmi fedora:33
Untagged: registry.fedoraproject.org/fedora:33
Deleted: 9f2a56037643a68ea81711a8eeb4501428eefd40b000c866ad9745a581c0464d
++ sleep 1
++ curl --unix-socket /run/user/1000/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.4","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.26-1.fc33.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.26, commit: 777074ecdb5e883b9bec233f3630c5e7fa37d521"},"cpus":12,"distribution":{"distribution":"fedora","version":"33"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":1000,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.10.14-200.fc33.x86_64","memFree":566480896,"memTotal":33388748800,"ociRuntime":{"name":"crun","package":"crun-0.17-1.fc33.x86_64","path":"/usr/bin/crun","version":"crun version 0.17\ncommit: 0e9229ae34caaebcb86f1fde18de3acaf18c6d9a\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/usr/bin/slirp4netns","package":"slirp4netns-1.1.8-1.fc33.x86_64","version":"slirp4netns version 1.1.8\ncommit: d361001f495417b880f20329121e3aa431a8f90f\nlibslirp: 4.3.1\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":20989861888,"swapTotal":21059592192,"uptime":"128h 56m 40.67s (Approximately 5.33 days)","linkmode":"dynamic"},"store":{"configFile":"/home/baude/.config/containers/storage.conf","containerStore":{"number":0,"paused":0,"running":0,"stopped":0},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/usr/bin/fuse-overlayfs","Package":"fuse-overlayfs-1.4.0-1.fc33.x86_64","Version":"fusermount3 version: 3.9.3\nfuse-overlayfs: version 1.4\nFUSE library version 3.9.3\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/baude/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":0},"runRoot":"/run/user/1000/containers","volumePath":"/home/baude/.local/share/containers/storage/volumes"},"registries":{"search":["registry.fedoraproject.org","registry.access.redhat.com","registry.centos.org","docker.io"]},"version":{"APIVersion":"3.0.0","Version":"3.1.0-dev","GoVersion":"go1.15.8","GitCommit":"5bd648c6eeb1d37c56bcc33458c84ee7af22adbc-dirty","BuiltTime":"Fri Feb 19 14:21:12 2021","Built":1613766072,"OsArch":"linux/amd64"}}

marusak · 2021-02-23T10:00:56Z

I think my results are correct?

seems so. Have you started with rootless service not running?
But also, this is obvious race (as tiny sleep resolves the problem) so it might work on machine that is faster (to delete the image) or slower (if it takes longer for the subsequent curl).

github-actions · 2021-03-26T00:09:12Z

A friendly reminder that this issue had no activity for 30 days.

rhatdan · 2021-03-26T11:18:15Z

This looks like rootless podman socket is not hanlding socket activation correctly.

rhatdan · 2021-03-26T11:18:53Z

@jwhonce Any idea what is going on here?

rhatdan · 2021-03-26T11:23:12Z

Simpler repeater

systemctl --user restart podman.socket; curl --unix-socket /run/user/3267/podman/podman.sock http://d/v1.24/libpod/info

Hangs

$ systemctl --user restart podman.socket; sleep 1; curl --unix-socket /run/user/3267/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.8","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.27-1.fc34.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.27, commit: "},"cpus":8,"distribution":{"distribution":"fedora","version":"34"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.11.3-300.fc34.x86_64","memFree":1224785920,"memTotal":16411303936,"ociRuntime":{"name":"crun","package":"crun-0.18-1.fc34.x86_64","path":"/usr/bin/crun","version":"crun version 0.18\ncommit: 808420efe3dc2b44d6db9f1a3fac8361dde42a95\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/bin/slirp4netns","package":"slirp4netns-1.1.9-1.fc34.x86_64","version":"slirp4netns version 1.1.8+dev\ncommit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876\nlibslirp: 4.4.0\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":11647287296,"swapTotal":16785596416,"uptime":"376h 10m 37s (Approximately 15.67 days)","linkmode":"dynamic"},"store":{"configFile":"/home/dwalsh/.config/containers/storage.conf","containerStore":{"number":9,"paused":0,"running":0,"stopped":9},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/home/dwalsh/bin/fuse-overlayfs","Package":"Unknown","Version":"fusermount3 version: 3.10.2\nfuse-overlayfs: version 0.7.2\nFUSE library version 3.10.2\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/dwalsh/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":30},"runRoot":"/run/user/3267/containers","volumePath":"/home/dwalsh/.local/share/containers/storage/volumes"},"registries":{"localhost:5000":{"Prefix":"localhost:5000","Location":"localhost:5000","Insecure":true,"Mirrors":null,"Blocked":false,"MirrorByDigestOnly":false},"search":["registry.fedoraproject.org","registry.access.redhat.com","docker.io"]},"version":{"APIVersion":"3.1.0-rc2","Version":"3.1.0-rc2","GoVersion":"go1.16","GitCommit":"","BuiltTime":"Tue Mar 23 21:45:35 2021","Built":1616550335,"OsArch":"linux/amd64"}}

Succeds, the question is whether this is podman issue or systemd. One would figure it is podman.

rhatdan · 2021-03-26T11:31:23Z

More information, if I kill the podman pause container off it works fine.

$ ps -eZ | grep podman
$ systemctl --user restart podman.socket; curl --unix-socket /run/user/3267/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.8","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.27-1.fc34.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.27, commit: "},"cpus":8,"distribution":{"distribution":"fedora","version":"34"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.11.3-300.fc34.x86_64","memFree":1264967680,"memTotal":16411303936,"ociRuntime":{"name":"crun","package":"crun-0.18-1.fc34.x86_64","path":"/usr/bin/crun","version":"crun version 0.18\ncommit: 808420efe3dc2b44d6db9f1a3fac8361dde42a95\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/bin/slirp4netns","package":"slirp4netns-1.1.9-1.fc34.x86_64","version":"slirp4netns version 1.1.8+dev\ncommit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876\nlibslirp: 4.4.0\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":11729338368,"swapTotal":16785596416,"uptime":"376h 18m 5.75s (Approximately 15.67 days)","linkmode":"dynamic"},"store":{"configFile":"/home/dwalsh/.config/containers/storage.conf","containerStore":{"number":9,"paused":0,"running":0,"stopped":9},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/home/dwalsh/bin/fuse-overlayfs","Package":"Unknown","Version":"fusermount3 version: 3.10.2\nfuse-overlayfs: version 0.7.2\nFUSE library version 3.10.2\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/dwalsh/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":30},"runRoot":"/run/user/3267/containers","volumePath":"/home/dwalsh/.local/share/containers/storage/volumes"},"registries":{"localhost:5000":{"Prefix":"localhost:5000","Location":"localhost:5000","Insecure":true,"Mirrors":null,"Blocked":false,"MirrorByDigestOnly":false},"search":["registry.fedoraproject.org","registry.access.redhat.com","docker.io"]},"version":{"APIVersion":"3.1.0-rc2","Version":"3.1.0-rc2","GoVersion":"go1.16","GitCommit":"","BuiltTime":"Tue Mar 23 21:45:35 2021","Built":1616550335,"OsArch":"linux/amd64"}}

Now I wait for the podman service to exit

$ ps -eZ | grep podman
unconfined_u:system_r:container_runtime_t:s0-s0:c0.c1023 230471 ? 00:00:00 podman pause
$ systemctl --user restart podman.socket; curl --unix-socket /run/user/3267/podman/podman.sock http://d/v1.24/libpod/info
HANGS
^C
 curl --unix-socket /run/user/3267/podman/podman.sock http://d/v1.24/libpod/info
{"host":{"arch":"amd64","buildahVersion":"1.19.8","cgroupManager":"systemd","cgroupVersion":"v2","conmon":{"package":"conmon-2.0.27-1.fc34.x86_64","path":"/usr/bin/conmon","version":"conmon version 2.0.27, commit: "},"cpus":8,"distribution":{"distribution":"fedora","version":"34"},"eventLogger":"journald","hostname":"localhost.localdomain","idMappings":{"gidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}],"uidmap":[{"container_id":0,"host_id":3267,"size":1},{"container_id":1,"host_id":100000,"size":65536}]},"kernel":"5.11.3-300.fc34.x86_64","memFree":1288908800,"memTotal":16411303936,"ociRuntime":{"name":"crun","package":"crun-0.18-1.fc34.x86_64","path":"/usr/bin/crun","version":"crun version 0.18\ncommit: 808420efe3dc2b44d6db9f1a3fac8361dde42a95\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"},"os":"linux","security":{"apparmorEnabled":false,"capabilities":"CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT","rootless":true,"seccompEnabled":true,"selinuxEnabled":true},"slirp4netns":{"executable":"/bin/slirp4netns","package":"slirp4netns-1.1.9-1.fc34.x86_64","version":"slirp4netns version 1.1.8+dev\ncommit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876\nlibslirp: 4.4.0\nSLIRP_CONFIG_VERSION_MAX: 3\nlibseccomp: 2.5.0"},"swapFree":11737202688,"swapTotal":16785596416,"uptime":"376h 19m 58.07s (Approximately 15.67 days)","linkmode":"dynamic"},"store":{"configFile":"/home/dwalsh/.config/containers/storage.conf","containerStore":{"number":9,"paused":0,"running":0,"stopped":9},"graphDriverName":"overlay","graphOptions":{"overlay.mount_program":{"Executable":"/home/dwalsh/bin/fuse-overlayfs","Package":"Unknown","Version":"fusermount3 version: 3.10.2\nfuse-overlayfs: version 0.7.2\nFUSE library version 3.10.2\nusing FUSE kernel interface version 7.31"}},"graphRoot":"/home/dwalsh/.local/share/containers/storage","graphStatus":{"Backing Filesystem":"extfs","Native Overlay Diff":"false","Supports d_type":"true","Using metacopy":"false"},"imageStore":{"number":30},"runRoot":"/run/user/3267/containers","volumePath":"/home/dwalsh/.local/share/containers/storage/volumes"},"registries":{"localhost:5000":{"Prefix":"localhost:5000","Location":"localhost:5000","Insecure":true,"Mirrors":null,"Blocked":false,"MirrorByDigestOnly":false},"search":["registry.fedoraproject.org","registry.access.redhat.com","docker.io"]},"version":{"APIVersion":"3.1.0-rc2","Version":"3.1.0-rc2","GoVersion":"go1.16","GitCommit":"","BuiltTime":"Tue Mar 23 21:45:35 2021","Built":1616550335,"OsArch":"linux/amd64"}}

rhatdan · 2021-03-26T11:37:52Z

If I kill the curl and try again it works. It is something about the pause container/joining the user namespace that is getting the confused.
@giuseppe Ideas?

giuseppe · 2021-03-29T09:51:37Z

PR opened here: #9855

if LISTEN_FDS is specified by systemd, we need to use the first fd after the std files (so fd=3) to read from the activation socket instead of manually opening the UNIX socket. [NO TESTS NEEDED] Closes: containers#9251 Signed-off-by: Giuseppe Scrivano <[email protected]>

openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Feb 6, 2021

github-actions bot added the stale-issue label Mar 26, 2021

rhatdan removed the stale-issue label Mar 26, 2021

giuseppe mentioned this issue Mar 29, 2021

service: use LISTEN_FDS #9855

Merged

openshift-merge-robot closed this as completed in #9855 Mar 29, 2021

martinpitt mentioned this issue Sep 14, 2021

Allow using only rootless podman without errors cockpit-project/cockpit-podman#742

Open

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rootless API not usable right after image is deleted #9251

Rootless API not usable right after image is deleted #9251

marusak commented Feb 6, 2021

mheon commented Feb 9, 2021

baude commented Feb 19, 2021

marusak commented Feb 23, 2021

github-actions bot commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

giuseppe commented Mar 29, 2021

Rootless API not usable right after image is deleted #9251

Rootless API not usable right after image is deleted #9251

Comments

marusak commented Feb 6, 2021

mheon commented Feb 9, 2021

baude commented Feb 19, 2021

marusak commented Feb 23, 2021

github-actions bot commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

rhatdan commented Mar 26, 2021

giuseppe commented Mar 29, 2021