Report merged reads and writes per disk

[bobrik]

I have spinning disks and during raid-10 rebuild I noticed this:

With a theoretical maximum of ~200 IOPS having 2000+ IOPS looks a bit surreal. I dug a bit and found out that these are merged reads and the do not mean much for monitoring, since you don't measure how hard you hit your disks.

I suggest reporting merged reads and writes:

• https://www.kernel.org/doc/Documentation/ABI/testing/procfs-diskstats

It also looks like time spent in IO for spinning disks could be more than 1 second per 1 second of real time. This also looks like a science fiction, you can't have one disk head in many locations at once, that's witchcraft^W quantum computing.

I suggest also reporting 14th column from the document above, that seems to hold the right metric.

[giampaolo]

So you're suggesting to add 5 - reads merged and 9 - writes merged.
What about 14 - weighted time spent doing I/Os (ms)? Why do you think it would be useful? Is it what is it being suggested in #523?

[bobrik]

Yes, let's add 5 and 9.

#523 seems to do what I want to calculate %util.

[giampaolo]

Why do you need 5 and 9? They definitively exist on Linux only and I would avoid them if possible. Isn't 10 ms_spent_doing_io enough?

[bobrik]

I need them because I care about actual writes on disk, not about write() syscalls. With spinning disks these are even more important.

Does FreeBSD (or other systems) provide you with raw or merged writes?

[giampaolo]

It is not possible to determine merged r/w on BSD and Windows (didn't check OSX and Solaris yet). They either don't have this capability of merging 2 adjacent r/w calls into one or the kernel does not collect/expose this info.
OK, I'm convinced this is needed, even if it's Linux only.

[giampaolo]

As for the other stats:
12 - I/Os currently in progress: I'm gonna add it as part of #523.

As for:

13 - time spent doing I/Os (ms)
14 - weighted time spent doing I/Os (ms)

...I may consider adding them if we can come up with some real-world app use case. Right now I'm not sure (don't understand) how can they be useful.

[bobrik]

@giampaolo real world example for 14 is util% from iostat, #523 aims to provide that.