podman with options userns keep-id and --restart always doesn't restart like expected

[econogit]

Issue Description

The container after a crash or kill doesn't restart with the option keep-id. It restarts like expected without this option.

Steps to reproduce the issue

$ podman run -d --userns keep-id --name nginx --restart always nginx:latest
8d79d08ffbb2a376d13fa62b0c70f77fc2367291daabe31dc7f6d834283545ca
$ podman inspect nginx --format '{{ .State.Pid }}'
2568968
$ kill -9 2568968
$ podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
8d79d08ffbb2 localhost/nginx:latest 14 seconds ago Exited (137) 4 seconds ago nginx

Describe the results you received

In addition, in the journald log, we have a "Started Process Core Dump".
I suspect the command "podman container cleanup nginx" who do a "Core Dump" with the error :
"panic: runtime error: invalid memory address or nil pointer dereference"

Describe the results you expected

$ podman run -d --userns host --name nginx --restart always nginx:latest
516dad881f0f0ebf9f98b4063cf4a09df3499b48d02a61c35cbf122fcf5160cd
$ podman inspect nginx --format '{{ .State.Pid }}'
2566680
$ kill -9 2566680
$ podman ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
516dad881f0f localhost/nginx:latest 51 seconds ago Up 3 seconds nginx

podman info output

$ podman info 
host:
  arch: amd64
  buildahVersion: 1.31.3
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.8-1.module+el8.9.0+20326+387084d0.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: 21e9be52d14128789284e1bbf54d8c25b4820215'
  cpuUtilization:
    idlePercent: 97.89
    systemPercent: 0.9
    userPercent: 1.21
  cpus: 4
  databaseBackend: boltdb
  distribution:
    distribution: '"rhel"'
    version: "8.8"
  eventLogger: journald
  freeLocks: 2047
  hostname: xxxx
  idMappings:
    gidmap:
    - container_id: 0
      host_id: xxxx
      size: 1
    - container_id: 1
      host_id: 231072
      size: 65536
    uidmap:
    - container_id: 0
      host_id: xxxx
      size: 1
    - container_id: 1
      host_id: 231072
      size: 65536
  kernel: 4.18.0-513.11.1.el8_9.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 2508718080
  memTotal: 16481193984
  networkBackend: cni
  networkBackendInfo:
    backend: cni
    dns:
      package: podman-plugins-4.6.1-4.module+el8.9.0+20326+387084d0.x86_64
      path: /usr/libexec/cni/dnsname
      version: |-
        CNI dnsname plugin
        version: 1.3.1
        commit: unknown
    package: containernetworking-plugins-1.3.0-4.module+el8.9.0+20326+387084d0.x86_64
    path: /usr/libexec/cni
  ociRuntime:
    name: runc
    package: runc-1.1.9-1.module+el8.9.0+20326+387084d0.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.1.9
      spec: 1.0.2-dev
      go: go1.20.6
      libseccomp: 2.5.2
  os: linux
  pasta:
    executable: ""
    package: ""
    version: ""
  remoteSocket:
    path: /run/user/xxxx/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_SYS_CHROOT,CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.1-1.module+el8.9.0+20326+387084d0.x86_64
    version: |-
      slirp4netns version 1.2.1
      commit: 09e31e92fa3d2a1d3ca261adaeb012c8d75a8194
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.2
  swapFree: 18253078528
  swapTotal: 18253602816
  uptime: 65h 48m 20.00s (Approximately 2.71 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
store:
  configFile: /users/xxxx/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /containers/xxxx/containers/storage
  graphRootAllocated: 80491315200
  graphRootUsed: 14373953536
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 5
  runRoot: /run/user/xxxx/containers
  transientStore: false
  volumePath: /containers/xxxxx/containers/storage/volumes
version:
  APIVersion: 4.6.1
  Built: 1696868155
  BuiltTime: Mon Oct  9 18:15:55 2023
  GitCommit: ""
  GoVersion: go1.20.6
  Os: linux
  OsArch: linux/amd64
  Version: 4.6.1

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

No

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[Luap99]

Do you have the full stack trace output?

[econogit]

@Luap99, here is the stack trace for the "container cleanup" command.
stack trace.txt