MacOS file system events don't pass through to containers

[catmeme]

Problem Description

Compare with Docker for Mac's feature, osxfs

Test case

docker-compose.yml

version: '3.7'
services:
  bash:
    entrypoint: bash /inotify.sh
    image: bash
    volumes:
      - ./inotify.sh:/inotify.sh
      - ./watch_me:/watch_me

inotify.sh

#!/usr/bin/env bash
apk add inotify-tools

inotifywait -m /watch_me -e create -e modify |
    while read directory action file; do
      echo $action $file
    done

mkdir watch_me
docker compose up

# in a new terminal tab:
echo "test" > watch_me/test.txt

Expected Results

it should output CREATE test.txt and MODIFY test.txt if the file system events pass from the host to the container.

Attaching to fse-test
fse-test  | fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
fse-test  | fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
fse-test  | (1/1) Installing inotify-tools (3.20.11.0-r0)
fse-test  | Executing busybox-1.34.1-r5.trigger
fse-test  | OK: 6 MiB in 18 packages
fse-test | Setting up watches.
fse-test  | Watches established.
fse-test  | CREATE test.txt
fse-test  | MODIFY test.txt

The file watcher in the container can be verified to work with:

docker compose exec bash bash
echo "test" > /watch_me/test.txt

Proposed Solution

Compare with Docker for Mac's feature, osxfs

Additional Information

This is a killer feature in Docker for Mac.

Preflight Checklist

• I have searched the issue tracker for a feature request that matches the one I want to file, without success.

[jandubois]

Compare with Docker for Mac's feature, osxfs

Unfortunately osxfs is a proprietary (closed-source) component of Docker Desktop. I'm not aware of any open-source network protocol implementation that supports forwarding of filesystem events.

For Lima we are currently investigating 9p as an alternative to reverse-sshfs, but that too does not support filesystem events (and would be non-trivial to implement).

Maybe one day we can switch to the Apple virtualization framework and use virtfs.

[catmeme]

Thanks for the information and explanation. Shame osxfs is proprietary, but I get it. Looking forward to a possible alternative in the future.

For now I have a hacky workaround, maybe someone can build a better hack off it for their use case.

I use a filewatcher on the host to exec a touch of the file on the app container and add a 10 second latency to avoid the filewatchers playing ping pong. The awk is to clean up the difference in the absolute path on the host and relative path on the container.

brew install fswatch
fswatch packages/app/src --event Updated --event Created --exclude ".*" --include "\\.tsx$" -l 10 | xargs \
  -I{} -n1 sh -c "echo {} && docker compose exec app touch \$(echo {} |awk -F "/packages/" '{ print \"packages/\"\$2 }')"

[jandubois]

Thanks for sharing your workaround!

[ellyxc]

Docker desktop is switching to VirtioFs now and still having the inotify, i tried to enabled VirtioFS in Rancher but still don't have the events sent to container.

[chris-ng-scmp]

Me too, tried VirtioFS in Rancher but still no luck :(
This is the only missing piece avoiding us adopt Rancher Desktop :(

[chris-ng-scmp]

@jandubois the Lima project do support inotify lima-vm/lima#1913, maybe there should be a way to support it for Rancher too?

[jandubois]

@jandubois the Lima project do support inotify lima-vm/lima#1913, maybe there should be a way to support it for Rancher too?

Yes, of course. It requires an update to the bundled Lima version, and probably a checkbox to make it opt-in because the way it works is that the host-agent will collects all the inotify events on the host, send them to the guest-agent, which will then "touch" all the files that had an event, which in turn will trigger the kernel to send an inotify event inside the guest.

So this is definitely an experimental configuration that can affect the files on the host.