[Fedora 33, rootless] podman pod create fails

[djzager]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I am running Fedora 33 Server + Xorg + DWM with multi-user.target as the default target for systemd (ie. no display manager, my .xprofile is included below under additional environment details). I'm not able to create rootless pods without explicitly setting DBUS_SESSION_BUS_ADDRESS. I found this issue #4162 that described a similar issue to mine and this workaround (#4162 (comment)). However, given that the issue is closed I thought it was best to create a new issue.

Steps to reproduce the issue:

1. Run podman pod create --name foobar

Describe the results you received:

$ podman pod create --name foobar
Error: unable to create pod cgroup for pod cb7d67da31a059f670eda0500e063b7d7020517a2b2d80fc4ecda5b26b1167e1: error creating cgroup user.slice/user-libpod_pod_cb7d67da31a059f670eda0500e063b7d7020517a2b2d80fc4ecda5b26b1167e1.slice: Process org.freedesktop.systemd1 exited with status 1

Describe the results you expected:

The result I get after explicitly setting DBUS_SESSION_BUS_ADDRESS:

$ export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus
$ podman pod create --name foobar
5d686a088f7575c6b39dcb8c88e83291790bf6f87a619cccfb7ebb8a0e2a443c

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      2.2.1
API Version:  2.1.0
Go Version:   go1.15.5
Built:        Tue Dec  8 09:37:50 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.18.0
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.21-3.fc33.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.21, commit: 0f53fb68333bdead5fe4dc5175703e22cf9882ab'
  cpus: 8
  distribution:
    distribution: fedora
    version: "33"
  eventLogger: journald
  hostname: dz-thinkpad
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65537
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65537
  kernel: 5.9.14-200.fc33.x86_64
  linkmode: dynamic
  memFree: 28930801664
  memTotal: 33439072256
  ociRuntime:
    name: crun
    package: crun-0.16-1.fc33.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.16
      commit: eb0145e5ad4d8207e84a327248af76663d4e50dd
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  rootless: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.8-1.fc33.x86_64
    version: |-
      slirp4netns version 1.1.8
      commit: d361001f495417b880f20329121e3aa431a8f90f
      libslirp: 4.3.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 21474828288
  swapTotal: 21474828288
  uptime: 37m 53.33s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - registry.centos.org
  - docker.io
store:
  configFile: /home/dzager/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.3.0-1.fc33.x86_64
      Version: |-
        fusermount3 version: 3.9.3
        fuse-overlayfs: version 1.3
        FUSE library version 3.9.3
        using FUSE kernel interface version 7.31
  graphRoot: /home/dzager/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 1
  runRoot: /run/user/1000/containers
  volumePath: /home/dzager/.local/share/containers/storage/volumes
version:
  APIVersion: 2.1.0
  Built: 1607438270
  BuiltTime: Tue Dec  8 09:37:50 2020
  GitCommit: ""
  GoVersion: go1.15.5
  OsArch: linux/amd64
  Version: 2.2.1

Package info (e.g. output of rpm -q podman or apt list podman):

$ rpm -qa|egrep 'podman|runc|crun|slirp|iptables|conmon|systemd' | sort
conmon-2.0.21-3.fc33.x86_64
crun-0.16-1.fc33.x86_64
iptables-1.8.5-4.fc33.x86_64
iptables-libs-1.8.5-4.fc33.x86_64
iptables-nft-1.8.5-4.fc33.x86_64
libreport-plugin-systemd-journal-2.14.0-15.fc33.x86_64
libslirp-4.3.1-3.fc33.x86_64
podman-2.2.1-1.fc33.x86_64
podman-plugins-2.2.1-1.fc33.x86_64
python3-systemd-234-14.fc33.x86_64
rpm-plugin-systemd-inhibit-4.16.0-5.fc33.x86_64
runc-1.0.0-279.dev.gitdedadbf.fc33.x86_64
slirp4netns-1.1.8-1.fc33.x86_64
systemd-246.7-2.fc33.x86_64
systemd-container-246.7-2.fc33.x86_64
systemd-libs-246.7-2.fc33.x86_64
systemd-networkd-246.7-2.fc33.x86_64
systemd-pam-246.7-2.fc33.x86_64
systemd-rpm-macros-246.7-2.fc33.noarch
systemd-udev-246.7-2.fc33.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes. Specifically, https://github.com/containers/podman/blob/master/docs/tutorials/rootless_tutorial.md

Additional environment details (AWS, VirtualBox, physical, etc.):

My .xprofile:

#!/usr/bin/env sh

# Fix Gnome Apps Slow  Start due to failing services
# Add this when you include flatpak in your system
dbus-update-activation-environment --systemd DBUS_SESSION_BUS_ADDRESS DISPLAY XAUTHORITY

# https://wiki.archlinux.org/index.php/Qt
export QT_QPA_PLATFORMTHEME="qt5ct"

setbg &                     # Set the background
sxhkd &                     # Bind keys
dunst &                     # Notification daemon
picom &                     # Transparency
xset dpms 600 &             # Sleep after 10 minutes
xset r rate 300 50 &        # Speed xrate up

[djzager]

This may very well end up being a documentation effort (change in configuration for me). I remember adding dbus-update-activation-environment --systemd DBUS_SESSION_BUS_ADDRESS DISPLAY XAUTHORITY to get Gnome apps and my notification daemon (dunst) to run nicely in my window manager. However, the manner in which I am doing it may be problematic for reasons I don't yet know. Additionally, I attempted adding export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus to my .xprofile and still did not get the desired behavior.

The reason for creating the issue is three-fold:

1. Googleable for others that may run into this in the future.
2. For me to learn the correct way to address this issue.
3. Call attention to documentation gaps in the case this isn't a bug in podman itself.

[mheon]

For reference, you can also swap Podman's cgroup manager from systemd to cgroupfs - there will be some missing features if you are rootless on a cgroupsv2 system (resource limits will not work without systemd) but core functionality will still function.

[djzager]

For reference, you can also swap Podman's cgroup manager from systemd to cgroupfs - there will be some missing features if you are rootless on a cgroupsv2 system (resource limits will not work without systemd) but core functionality will still function.

Seems reasonable. I get the feeling that not running in Gnome is part of the problem for me; which is kind of expected given the need to start everything myself. Knowing how to properly startup my window manager such that podman is happy to run with systemd (given I am already using systemd 😎) is what I want to know -- and for others to know when they go to setup podman for theirself in the future.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

I don't believe this is an issue, but a user modification required.

[djzager]

@rhatdan I don't agree that this should be closed but I could be missing something.

It seems odd to me that I can install Fedora Server, install podman, and it doesn't work without tweaks that aren't listed anywhere. I am aware of a workaround that gets me moving but it still isn't clear to me if export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus is the right way to fix this (I hope it isn't since I have to run it each time podman fails and I say "Oh yeah, it doesn't work without this"). Additionally, it was mentioned:

For reference, you can also swap Podman's cgroup manager from systemd to cgroupfs - there will be some missing features if you are rootless on a cgroupsv2 system (resource limits will not work without systemd) but core functionality will still function.

But that doesn't strike me as wholly helpful as Fedora Server is using systemd.

Seems reasonable. I get the feeling that not running in Gnome is part of the problem for me; which is kind of expected given the need to start everything myself. Knowing how to properly startup my window manager such that podman is happy to run with systemd (given I am already using systemd ) is what I want to know -- and for others to know when they go to setup podman for theirself in the future.

I suspect this comment of mine may have made it out to be more of a user problem than it is. If that's the case, I'm sorry.

From my perspective, if I install podman on Fedora Server, then podman should work without tweaks. If tweaks are required, I would expect them to be easy to find here. What do you think?

[rhatdan]

@giuseppe @vrothberg FYI ^^

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@giuseppe @vrothberg FYI ^^ again.

[giuseppe]

how was the user session created? Through su?

If DBUS_SESSION_BUS_ADDRESS is not set, Podman tries to automatically detects it given that unix:path=/run/user/$(id -u)/bus exists.

Does that file exist when you encounter the issue?

Is DBUS_SESSION_BUS_ADDRESS already set to a different value?

[djzager]

Here is my .xprofile:

#!/usr/bin/env sh

# Fix Gnome Apps Slow  Start due to failing services
# Add this when you include flatpak in your system
dbus-update-activation-environment --systemd DBUS_SESSION_BUS_ADDRESS DISPLAY XAUTHORITY

# https://wiki.archlinux.org/index.php/Qt
export QT_QPA_PLATFORMTHEME="qt5ct"

setbg &                     # Set the background
sxhkd &                     # Bind keys
dunst &                     # Notification daemon
picom &                     # Transparency
xset dpms 600 &             # Sleep after 10 minutes
xset r rate 300 50 &        # Speed xrate up

And my xinitrc:

#!/bin/sh

if [ -d /etc/X11/xinit/xinitrc.d ]; then
  for f in /etc/X11/xinit/xinitrc.d/*; do
    [ -x "$f" ] && . "$f"
  done
  unset f
fi

[ -f $HOME/.xprofile ] && . $HOME/.xprofile

while :; do
	#dbus-launch --exit-with-session dwm || break
	ssh-agent dwm || break
done

X is started via this line in my profile:

[ "$(tty)" = "/dev/tty1" ] && ! pgrep -x Xorg >/dev/null && exec startx

Quick runthrough:

$ echo $DBUS_SESSION_BUS_ADDRESS
unix:abstract=/tmp/dbus-E7790MdGA0,guid=6136c15b192b4fb601f25e0b6058eb72
$ podman pod create --name foobar
Error: unable to create pod cgroup for pod 492e6e51fafb4ed5da0e51f3ee53db1ccabe51e1b4259a3f289f8b474c3cfa1e: error creating cgroup user.slice/user-libpod_pod_492e6e51fafb4ed5da0e51f3ee53db1ccabe51e1b4259a3f289f8b474c3cfa1e.slice: Process org.freedesktop.systemd1 exited with status 1
$ export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus
$ podman pod create --name foobar
907679f48bb517cb1be4bd209d45d886bd24ada55c857ee3aa34c06a8af7b4be

[giuseppe]

could you force the export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus before dbus-update-activation-environment ?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Since we got no response I guess @giuseppe suggestion worked, I am going to close the issue. Reopen if I am mistaken.

[djzager]

Apologies. I got wrapped up in other things. Clearly not a backbreaking problem 😎. If I run into the issue after fresh install of Fedora 34, I'll attempt what @giuseppe mentioned and report back.

Thank you all for your time.

[larrycai]

we met the similar issue as well, and noticed some x-session needs to set DBUS_SESSION_BUS_ADDRESS like

DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-P0Hy0VSGn1,guid=f0fd41e472aa4b4f5a28f37a6323d8ds8a

while podman wants to

DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus

there could be conflict

also I checked #9727

[giuseppe]

Podman won't try to set the env variable if it is an abstract path, in this case you must make sure the env variable is correctly set before using Podman

[larrycai]

just curious to check whether we have other better solution since it is conflict with Citrix/VDA setup which I can't change

In Citrix environment, we have system script like below

eval `dbus-launch --sh-syntax --exit-with-session`

it will generate

DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-P0Hy0VSGn1,guid=f0fd41e472aa4b4f5a28f37a6323d8ds8a

Can we use other variable or setup this in local configuration?