Example Scan Performance
This is the typical performance of a scan of the isilon system at CRG. We use 128 mpi workers. The slope is initially steep since some of the nodes in our isilon cluster are equipped with SSDs to improve metadata performance. The scan quickly chomps through the metadata held on these nodes but then slows down as it reaches metadata that is not on SSD. If we had SSD metadata in all the nodes of the cluster the full scan would likely only take around 4 hours.
This is the typical scan progress on the lustre system at CNAG. This uses 32 workers. Interestingly, the slope is roughly the same as the fast part of the isilon system above - around 100 million inodes per hour. Note that the metadata queries hit the OSTs not the MDTs since the size data can only be retrieved from running an lstat on the underlying OST filesystem (ext4 in our case).
