podman optimize relabel of mount point

[akostadinov]

Issue Description

I'm using podman rootless to run a seafile server with many files. On startup it appears as if it is relabeling for a long time before container starts.

I read that podman optimizes relabeling when directory was once labeled with z but in this case I'm using a systemd service via Quadlet. So each startup a new container is created. It is not starting the same one.

I think it is important to support this use case.

Any ideas for workarounds are also appreciated.

Steps to reproduce the issue

Steps to reproduce the issue

1. create a podman container user service with quadlet with a mount point with a lots of files
2. start service
3. stop and start service again

Describe the results you received

it is slow on every startup

Describe the results you expected

second startup does not relabel all files

podman info output

host:
  arch: amd64
  buildahVersion: 1.37.2
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.12-2.fc40.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.12, commit: '
  cpuUtilization:
    idlePercent: 98.29
    systemPercent: 1.21
    userPercent: 0.5
  cpus: 2
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    version: "40"
  eventLogger: journald
  freeLocks: 2038
  hostname: [REDACTED]
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1111
      size: 1
    - container_id: 1
      host_id: 655360
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1111
      size: 1
    - container_id: 1
      host_id: 655360
      size: 65536
  kernel: 6.10.9-200.fc40.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 6317486080
  memTotal: 16447782912
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.12.2-2.fc40.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.12.2
    package: netavark-1.12.2-1.fc40.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.12.2
  ociRuntime:
    name: crun
    package: crun-1.15-1.fc40.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/user/1111/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20240906.g6b38f07-1.fc40.x86_64
    version: |
      pasta 0^20240906.g6b38f07-1.fc40.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: false
    path: /run/user/1111/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 74h 37m 50.00s (Approximately 3.08 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /home/containers/.config/containers/storage.conf
  containerStore:
    number: 8
    paused: 0
    running: 8
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /media/disk3/podman_data/containers
  graphRootAllocated: 2146418302976
  graphRootUsed: 59442978816
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 14
  runRoot: /run/user/1111/containers
  transientStore: false
  volumePath: /media/disk3/podman_data/containers/volumes
version:
  APIVersion: 5.2.2
  Built: 1724198400
  BuiltTime: Wed Aug 21 03:00:00 2024
  GitCommit: ""
  GoVersion: go1.22.6
  Os: linux
  OsArch: linux/amd64
  Version: 5.2.2

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response

[Luap99]

Are you actually using z or Z (upper case)? In the later case we have to relabel each new container because each container must have a unique label.

I know the issue you are talking about, I also have a volumes that takes minutes to relabel so I very much interested in ideas to improve this but I do not see the current behavior as bug.

[akostadinov]

My bad, I use Z. I mixed up the two.

I don't know how selinux stuff works now. But based on a wild guess, the only thing that comes to mind is deriving the private id from some container properties and when the root of the mount matches, then don't relabel.

Or just read what is already there in the mount root. And in case no other container uses that id already, use that id otherwise relabel.

I'm planning to upload a lot of data. I think startup times will become unacceptable very soon as I've just uploaded a sample of everything I want to keep there.

I guess I can switch to z and see. But it defeats the purpose of the protection as what I want to do is to isolate containers from one another.

[rhatdan]

If you chcon -R -t container_file_t PATHTODIR/

Then don't add the :z or :Z then SELinux should work fine with the quadlet without any relabeling.

As long as the directory tree does not get removed it will stay labeled container_file_t and all content added to the tree will get that label.

If you remove the tree for any reason, then when you recreate the tree do the mkdir PATODIR/; chcon -t container_file_t PATHTODIR and start copying the content, it will all get labeled correctly and the containers will be able to read/write the content.

[Luap99]

But that does not label it private for only the single container, i.e. it does what z does, no? And AFAIK we do not relabel in that case when the label matches,

podman/libpod/container_internal_common.go

Lines 3022 to 3024 in b997841

	if label == mountLabel {
	return nil
	}

[akostadinov]

Another idea, again I don't know whether it is possible or not, to label just the root as private but all else to be shared. This should prevent other containers to read the files inside in normal conditions.

In this case on every restart only a single label will be updated. It could be a separate flag like zZ for example.

[akostadinov]

Thank you, I don't think I can ensure there is a container at all times with quadlet. Maybe make quadlet not remove container until starting again. Anyway, I think accepting the risk of accidental match is more reasonable than waiting excessive time on startup.

I don't understand still these pairs though. I've read https://www.redhat.com/en/blog/how-selinux-separates-containers-using-multi-level-security and my understanding from it is that this is a set of .. "categories"? that container is expected to be able to access.

So I don't understand how using 2 values is more secure. If you have any resources about that, I will appreciate.

[rhatdan]

It is all about numbers, if you use one number you have 1024 categories. If you use 2 you have
(1024*1024/2 - 1024) =~ 500,000 unigue pairs.
cX,X== cX
cX,CY==cY,CX

So we want to have as many unigue pairs as possible, at some point we might need to add a third option.
Realize that MCS is about dominance.

s0:C123,c456 Is allowed to read write files with s0:C123,c456 as well as s0:c123 and s0:c456 and s0.
So you need to have all of the categories to dominate the other label.
s0:c123,c456 is not allowed to read/write s0:c123,c4 since it does not dominate the c4 category.

s0 is writable by all MCS labels since it has no categories, This is why we use this label for :z

[akostadinov]

Now I've got it. Thanks a lot @Luap99 and @rhatdan for the valuable insights! I think this is a reasonable solution to the relabeling problem. Would perhaps be useful to document it.

[rhatdan]

Sure, where would you like it documented? Under quadlet?

[akostadinov]

$ man podman run
...
Relabeling walks the file system under the
volume and changes the label on each file, if the volume has thousands
of inodes, this process takes a long time, delaying the start of the
container.
...

I think somewhere around there would be a good place to be documented or have a reference for more information.