ZFS filesystem support

[galindro]

I've made a simple modification of _pslinux.py based on master branch downloaded at Sep 21 19:27 UTC.

The bellow modification allow users to list ZFS filesystems on Linux (http://zfsonlinux.org/) through psutil.disk_partitions()

# diff _pslinux.py-original _pslinux.py
666,667c666,669
<             if not line.startswith("nodev"):
<                 phydevs.append(line.strip())

---
>             if line.startswith("nodev") and line.find('zfs') == -1:
>                 continue
>             else:
>                 phydevs.append(line.replace('nodev','').strip())

I've made tests and it is working fine.

This is the result:

>>> import psutil
>>> psutil.disk_partitions()
[sdiskpart(device='/dev/xvda1', mountpoint='/', fstype='ext4', opts='rw'), sdiskpart(device='datastore_var/var', mountpoint='/var', fstype='zfs', opts='rw'), sdiskpart(device='datastore_backup/backup', mountpoint='/backup', fstype='zfs', opts='rw,noatime,xattr')]

This is my contribution with this nice project. I hope you include it in future versions.

:)

[curlup]

Is there a PR? I can't find one.

What about other systems (not linux) - will it work too?

[galindro]

curlup, sorry for delay... I don't tested this feature in other OS. Only in Linux with ZFSonLinux.

[mrvn]

Any updates on this? This is something I would like to use too.

[giampaolo]

Is this correct? Can someone test this patch?

diff --git a/psutil/_pslinux.py b/psutil/_pslinux.py
index be443ef..3157384 100644
--- a/psutil/_pslinux.py
+++ b/psutil/_pslinux.py
@@ -671,7 +671,7 @@ def disk_partitions(all=False):
     phydevs = []
     with open("/proc/filesystems", "r") as f:
         for line in f:
-            if not line.startswith("nodev"):
+            if not line.startswith("nodev") or not line.startswith("zfs"):
                 phydevs.append(line.strip())

     retlist = []

[galindro]

@giampaolo I could confirm that it works as I said when ticket was opened.

[giampaolo]

OK, I'm gonna make a release soon. Sorry for the big delay (I've been traveling).

[giampaolo]

@galindro can you please post the output of cat /proc/filesystems (with an active zfs partition of course)?

[galindro]

nodev   sysfs
nodev   rootfs
nodev   ramfs
nodev   bdev
nodev   proc
nodev   cgroup
nodev   cpuset
nodev   tmpfs
nodev   devtmpfs
nodev   debugfs
nodev   securityfs
nodev   sockfs
nodev   pipefs
nodev   anon_inodefs
nodev   devpts
    ext3
    ext2
    ext4
nodev   hugetlbfs
    vfat
nodev   ecryptfs
    fuseblk
nodev   fuse
nodev   fusectl
nodev   pstore
nodev   mqueue
nodev   zfs
    iso9660
    xfs
    msdos
    ufs
    btrfs

[giampaolo]

I should have fixed this in 31258b5. Can you please try latest psutil tests?
Please do:

git clone [email protected]:giampaolo/psutil.git
cd psutil
make test

[galindro]

Done!

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.disk_partitions() 
[sdiskpart(device='/dev/xvda1', mountpoint='/', fstype='ext4', opts='rw'), sdiskpart(device='datastore/var', mountpoint='/var', fstype='zfs', opts='rw'), sdiskpart(device='datastore/tmp', mountpoint='/tmp', fstype='zfs', opts='rw,noatime'), sdiskpart(device='datastore/var_log', mountpoint='/var/log', fstype='zfs', opts='rw,noatime')]
>>> 

[giampaolo]

Wonderful! I should release a new version soon.
Did you run tests? Do they pass?

[galindro]

Yes, over 10 months aproximatelly.
Em 11/07/2015 16:32, "giampaolo" [email protected] escreveu:

Wonderful! I should release a new version soon.
Did you run tests?

—
Reply to this email directly or view it on GitHub
#534 (comment).

[giampaolo]

Hey, you know I do this for free, right? And you know there's a button called "make a pull request" and another one called "fork"? Next time you desperately need a feature you can either:

• 1. make a pull request including tests
• 2. fork psutil and do it yourself if you see I'm absent
• 3. pay me

Usually 1) is the best way to include the change you need as soon as possible because to me that means just pressing the "merge pull request" button. And by the way, by "tests" I mean the tests I added 12 hours ago in 31258b5, not the ones you didn't provide 10 months ago.

[galindro]

I'm new in github and at that time, I was never worked with git, as you can
se in the issue's conversation. In feature releases, if I need some
improvement like this one, be sure that I will make it using the correct
way.

Thanks again.
Em 12/07/2015 11:13, "giampaolo" [email protected] escreveu:

Hey, you know I do this for free, right? And you know there's a button
called "make a pull request" and another one called "fork"? Next time you
desperately need a feature you can either:

• 1. make a pull request including tests
• 2. fork psutil and do it yourself if you see I'm absent
• 3. pay me Usually 1) is the best way to include the change you need
     as soon as possible because to me that means just pressing the "merge pull
     request" button. And by the way, by "tests" I mean the tests I added 12
     hours ago in 31258b5
     31258b5,
     not the ones you didn't provide 10 months ago.

—
Reply to this email directly or view it on GitHub
#534 (comment).

[mrvn]

Zfs also has real block devices that can be partitioned further. This should be tested as well. Since they appear under /dev I think they will work already but better safe than sorry.

[giampaolo]

What do you mean? What should we test exactly? Can you can up with some code sample or output of how /proc/partitions or /dev would look like in that case?

[mrvn]

A zfs filesystem does not show in /proc/partitions as it has no user reachable block device. But it does show up in df output:

% df -h
Filesystem Size Used Avail Use% Mounted on
lvm-zfs/test 830M 128K 830M 1% /lvm-zfs/test

A zfs block device on the other hand shows up in /proc/partions:

% cat /proc/partitions
230 0 2097152 zd0
230 1 2096128 zd0p1
230 16 1925120 zd16
230 17 1919736 zd16p1
230 32 102400 zd32

% ls -lh /dev/zvol/lvm-zfs/
total 0
lrwxrwxrwx 1 root root 10 Jul 16 16:10 test-block -> ../../zd32
lrwxrwxrwx 1 root root 9 Jun 3 12:52 trusty-fai -> ../../zd0
lrwxrwxrwx 1 root root 11 Jun 3 12:52 trusty-fai-part1 -> ../../zd0p1
lrwxrwxrwx 1 root root 10 Jun 3 12:52 usb2g -> ../../zd16
lrwxrwxrwx 1 root root 12 Jun 3 12:52 usb2g-part1 -> ../../zd16p1

The block device can have a filesystem and be mounted too, just like any other block device. I'm fairly certain that just works out of the box already. Formated as ext4 and mounted it shows in df:

% df -h
/dev/zd32 93M 1.6M 85M 2% /mnt/zfs-block-ext4

% python3
Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.

import psutil
psutil.disk_partitions()
[sdiskpart(device='/dev/zd32', mountpoint='/mnt/zfs-block-ext4', fstype='ext4', opts='rw')]

and it does work in psutil. Might be nice to list /dev/zvol/lvm-zfs/test-block as device name for devcies named zdXY but that probably goes the same for raid and lvm devices.

[giampaolo]

Closing as fixed.