Cleanup and major refactor (#1)

* add persistence, clean structure

* add systemd configurations

* improve setup and packaging

.gitignore

@@ -1,6 +1,20 @@
 # Python cache
-**/*.pyc
-**/__pycache__
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Environments
+.env
+.venv
+env/
+venv/
+env.bak/
+venv.bak/
+
+# Python distribution/packaging
+*.egg-info
+dist/
+build/
 
 # IDE configuration
 .vscode

MANIFEST.in

@@ -0,0 +1 @@
+graft systemd

README.md

@@ -1,11 +1,120 @@
 # `easyfuse` - simple FUSE volume driver for Docker
 
-Simple FUSE-based volume driver based on the local driver systax. By offloading volume mounting to a simple `mount -t fuse` subprocess call, the typical problem of https://github.com/moby/moby/issues/27103 does not apply.
+Simple FUSE-based docker volume driver based on the local driver systax. By offloading volume mounting to a simple `mount -t fuse` subprocess call, the typical problem of https://github.com/moby/moby/issues/27103 does not apply.
 
 The plugin uses legacy architecture (mounting as a unix socket) to expose to the user full host system capabilites of `mount`. This way, user can e.g. fully specify `uid` and `gid` volume mapping, use local system resources (e.g. SSH keys), something that is an annoying shortcoming of plugins such as `vieux/sshfs`.
 
 Initially, the plugin was developed to allow using SSHFS in scenario, where `vieux/sshfs` simply wouldn't work, and after discovering _why_ the `local` driver would not work with fuse mount type. Hovever, in theory this plugin should work with any `fuse`-mounted filesystem (and in future, possibly even others).
 
+### Permissions
+
+Currently, since easyfuse uses `mount` for creating actual fuse volumes, `easyfuse` must be ran as root. This may change in future versions.
+
+## Package dependencies
+
+It's relatively easy to run `easyfuse` without actually installing the package, nevertheless it 
+requires some non-standard third party Python libraries. To start, simply install module dependencies 
+(see [requirements.txt]), either with
+
+```
+sudo python3 -m pip install -r requirements.txt
+```
+
+or (preferably) with your system package manager, e.g. for Debian/Ubuntu:
+
+```
+sudo apt install python3-aiohttp
+```
+
+When using `pip`, make sure the packages are installed globally, as `easyfuse` currently needs to be ran as root.
+
+## Local installation in virtual environment
+
+`easyfuse` should have no problems working out of a python virtual environment.
+Simply initialize a virtual environment and install the package locally:
+
+```
+$ python3 -m venv venv
+$ source venv/bin/activate
+(venv) $ pip install -e .
+```
+
+This will pull all local requirements, without interfering with the system and system's package
+managers.
+
+When running from venv with `sudo`, you may need to manually point to the venv's python as virtual
+environment variables are not inherited by `sudo`-ed process, see:
+
+```
+# locally installed aiohttp in version 3.5.1, venv installed aiohttp in version 3.7.3:
+(venv) $ python3 -c 'import aiohttp; print(aiohttp.__version__)'
+3.7.3
+(venv) $ sudo python3 -c 'import aiohttp; print(aiohttp.__version__)'
+3.5.1
+```
+
+Instead, simply use `venv/bin/python` explicitly:
+
+```
+(venv) $ sudo venv/bin/python -c 'import aiohttp; print(aiohttp.__version__)'
+3.7.3
+```
+
+## Running manually (without installation)
+
+To run in a stand-alone mode (preferred for development), start in the main directory and simply run
+
+```
+sudo python3 -m easyfuse -s /run/docker/plugins/easyfuse.sock
+```
+
+see `python3 -m easyfuse -h` (no `sudo` required) for full list of available options.
+
+## Running with `systemd` (with or without installation)
+
+`systemd` folder contains basic systemd unit files for socket activation, either for global and local
+setup.
+Assuming `/etc/systemd/system` as the configuration path of choice, typical activation for a local setup would be:
+
+```
+$ SYSTEMD_UNIT_TARGET=/etc/systemd/system
+$ sudo cp systemd/easyfuse.socket $SYSTEMD_UNIT_TARGET/
+$ WORKDIR=$PWD envsubst '$WORKDIR' < systemd/easyfuse.service.dev | sudo tee $SYSTEMD_UNIT_TARGET/easyfuse.service
+$ sudo systemctl daemon-reload
+$ sudo systemctl start easyfuse.socket
+```
+
+When using virtualenv, use the appropriate config:
+
+```
+(venv) $ SYSTEMD_UNIT_TARGET=/etc/systemd/system
+(venv) $ sudo cp systemd/easyfuse.socket $SYSTEMD_UNIT_TARGET/
+(venv) $ envsubst '$VIRTUAL_ENV' < systemd/easyfuse.service.venv | sudo tee $SYSTEMD_UNIT_TARGET/easyfuse.service
+(venv) $ sudo systemctl daemon-reload
+(venv) $ sudo systemctl start easyfuse.socket
+```
+
+For a global installation (i.e. if you can call `python3 -m easyfuse -h` as root from anywhere, without virtual environment), simply use:
+
+```
+$ SYSTEMD_UNIT_TARGET=/etc/systemd/system
+$ sudo cp systemd/easyfuse.socket  $SYSTEMD_UNIT_TARGET/
+$ sudo cp systemd/easyfuse.service $SYSTEMD_UNIT_TARGET/
+$ sudo systemctl daemon-reload
+$ sudo systemctl start easyfuse.socket
+```
+
+### Automatic systemd setup
+
+`easyfuse` provides a simple automatic `systemd` setup via `easyfuse.systemd_setup` script-module. 
+See `python3 -m easyfuse.systemd_setup -h` for reference. Remember that when using `sudo` with `venv`,
+you need to use the `venv` symlink instead of `python3`:
+
+```
+(venv) $ sudo python3 -m easyfuse.systemd_setup venv          # <- this will install using the global python
+(venv) $ sudo venv/bin/python3 -m easyfuse.systemd_setup venv # <- this will install using the local venv
+```
+
 ## Using `easyfuse` with docker volume
 
 Note: when using SSHFS, make sure the host key is accepted by the root user (or whatever user is running the mount command). This can be done by simply doing `sudo ssh user@my-ssh-host` and verifying the public key, or with `ssh-keyscan >> /root/.ssh/known_hosts`.
@@ -120,7 +229,3 @@ Removing examples_mytest_1 ... done
 Removing network examples_default
 Removing volume examples_nas
 ```
-
-# Permissions
-
-Currently, since easyfuse uses `mount` for creating actual fuse volumes, `easyfuse` must be ran as root. This may change in future versions.

easyfuse/Driver.py

@@ -0,0 +1,107 @@
+'''
+easyfuse - simple FUSE volume driver for Docker
+Copyright (C) 2020  Marcin Słowik
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import os
+import subprocess
+
+from .MountDatabase import MountDatabase, VolumeSpec
+
+
+class DriverError(Exception):
+    pass
+
+
+class Driver:
+    def __init__(self, opts):
+        self.mntpath = opts.mntpt
+        self.mntdb = MountDatabase(opts.mntdb)
+        dbpath = os.path.dirname(opts.mntdb)
+        os.makedirs(self.mntpath, mode=0o777, exist_ok=True)
+        os.makedirs(dbpath, mode=0o777, exist_ok=True)
+
+    def get_path_for(self, name: str):
+        return os.path.join(self.mntpath, name)
+
+    async def is_mounted(self, name):
+        async with self.mntdb:
+            return self.mntdb.get_ro(name).is_mounted
+
+    @property
+    async def volumes(self):
+        """
+        Provides a read-only view of the mntdb
+        """
+        async with self.mntdb:
+            return {key: self.mntdb.get_ro(key) for key in self.mntdb.keys()}
+
+    async def volume_create(self, name: str, opts: dict):
+        async with self.mntdb:
+            if name in self.mntdb:
+                raise DriverError(
+                    f"Volume {name} already exist, remove it first.")
+            self.mntdb[name] = VolumeSpec(name, set(), opts)
+
+    async def volume_remove(self, name: str):
+        async with self.mntdb:
+            del self.mntdb[name]
+
+    async def volume_mount(self, name: str, vid: str):
+        async with self.mntdb:
+            try:
+                vol = self.mntdb[name]
+            except KeyError:
+                raise DriverError(f"Volume {name} not found.")
+            vol.instances.add(vid)
+            if not vol.is_mounted:
+                # mount -t fuse -o [opts.o] [opts.device] [mountpoint]
+                cmd = ("mount", "-t", "fuse")
+                try:
+                    cmd += ("-o", vol.opts["o"])
+                except KeyError:
+                    pass
+                try:
+                    cmd += (vol.opts["device"], )
+                except KeyError:
+                    raise DriverError(f"Volume {name} missing option: device")
+                mntpt = self.get_path_for(name)
+                cmd += (mntpt, )
+                try:
+                    os.makedirs(mntpt, mode=0o777, exist_ok=True)
+                    subprocess.run(cmd, check=True)
+                    vol.is_mounted = True
+                except subprocess.CalledProcessError as e:
+                    raise DriverError(e)
+
+    async def volume_unmount(self, name: str, vid: str):
+        async with self.mntdb:
+            try:
+                vol = self.mntdb[name]
+            except KeyError:
+                raise DriverError(f"Volume {name} not found.")
+            try:
+                vol.instances.remove(vid)
+            except KeyError:
+                raise DriverError(f"Volume ID {vid} not found.")
+            if not vol.instances and vol.is_mounted:
+                mntpt = self.get_path_for(name)
+                try:
+                    cmd = ('umount', mntpt)
+                    subprocess.run(cmd, check=True)
+                    vol.is_mounted = False
+                except subprocess.CalledProcessError as e:
+                    raise DriverError(e)