play kube: memory limit for pod kind behaves differently than podman's '-m' option and also differently then k8s

[ydayagi]

/kind bug
Description
play kube: memory limit for pod kind behaves differently than podman's '-m' option and also differently then k8s
Steps to reproduce the issue:
pod YAML:

apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  containers:
  - name: c1
    image: quay.io/ydayagi/memoryeater
    command: ["/stam/a.out", "400"]
    resources:
      limits:
        memory: 32Mi

1. Use pod YAML above to create a pod with 'podman play kube'

2. Use pod YAML above to create a pod in k8s cluster

3. run a container with podman run -m 60m -d --entrypoint '["/stam/a.out","200"]' quay.io/ydayagi/memoryeater:latest

Describe the results you received:
For 'play kube' the container's RSS is limited to approximately the limit in the YAML. However, virtual memory is whatever the container consumes.
For k8s pod the cluster kills the pod and sets the 'OOMKilled' status.
For 'podman run' the container exists with exit code 137 but OOMKilled is false.

Describe the results you expected:
I expect options 2 and 3 to be the same. I do not see a reason not to. After all, 'play kube' and 'run' are just 2 different input methods for the same flow/operation.

Output of podman version:
I am using the latest podman code

Client:       Podman Engine
Version:      4.0.0-dev
API Version:  4.0.0-dev
Go Version:   go1.16.8
Git Commit:   f203a2cf1c65fe7e9c3d79934406365127fa79c6
Built:        Sun Jan 30 21:19:05 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.24.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 8
  distribution:
    distribution: fedora
    variant: workstation
    version: "34"
  eventLogger: journald
  hostname: fedora
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.11.12-300.fc34.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 18163752960
  memTotal: 33436639232
  networkBackend: cni
  ociRuntime:
    name: crun
    package: crun-0.18-5.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.18
      commit: 808420efe3dc2b44d6db9f1a3fac8361dde42a95
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 8199831552
  swapTotal: 8589930496
  uptime: 43h 5m 47.55s (Approximately 1.79 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/ydayagi/.config/containers/storage.conf
  containerStore:
    number: 3
    paused: 0
    running: 2
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.5.0-1.fc34.x86_64
      Version: |-
        fusermount3 version: 3.10.2
        fuse-overlayfs: version 1.5
        FUSE library version 3.10.2
        using FUSE kernel interface version 7.31
  graphRoot: /home/ydayagi/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 16
  runRoot: /run/user/1000/containers
  volumePath: /home/ydayagi/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1643570345
  BuiltTime: Sun Jan 30 21:19:05 2022
  GitCommit: f203a2cf1c65fe7e9c3d79934406365127fa79c6
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 4.0.0-dev

[rhatdan]

@umohnani8 PTAL

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@umohnani8 Any chance you have looked at this?

[umohnani8]

yup, working on it.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@umohnani8 Ping.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@cdoern PTAL

[cdoern]

-m for pods was just implemented last week. I think play kube applies the memory limit to each container in the pod rather than pulling from the cgroup parent as podman pods are supposed to. This might explain the differences in settings.

not sure if this is a bug or just a quirk of different implementations.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@cdoern what do you think we should do?
@giuseppe @vrothberg @nalind Opinions?

[rhatdan]

@baude @mheon WDYT?

[mheon]

The memory limit being per-container looks correct - that's what the YAML is requesting.

The actual bug appears to be that OOMKilled is not being set properly, from what I'm reading? Also, potentially that our virtual memory consumption is in excess of the requested amount - which is probably an issue of mapping a K8S memory limit to our resource limit primitives.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.