Needed Development: SimpleStorageService using S4U Storage

[henricasanova]

Update the implementation of the SimpleStorageService so that it uses the Storage abstraction provided by S4U, whenever available/documented. One issue to pay attention to is that pipelining of network transfer and disk writes (a "store-and-forward" approach is really not realistic).

[henricasanova]

Did a refactoring of the storage service (to be merged soon into master), that will make it possible to finally move forward on this. SimGrid should provide new XML tags and APIs by the end of September 2019, at which point we'll be able to modify the Storage Service as follows:

• "disk" tags in the XML will specify the disks attached to a host:

     <host id="alice" speed="1Gf">
      <disk id="Disk1" read_bw="200MBps" write_bw="80MBps">
        <prop id="size" value="500GiB"/>
        <prop id="mount" value="/scratch" />
      </disk>
    </host>

• I/O operations to those disks will be simulated by SimGrid (some refactoring in WRENCH makes that immediate)

• StorageService constructors will no longer require capacities, just disk IDs

  • Question: If I start a storage service on a host with 2+ disks, does the storage service handle all disks?

  • More general question: Can a StorageService handle more than one disk?

• The notion of "partition", which was like a directory, will just become a disk / mount point

  • Question We had a notion of directory (which were partitions). Now it seems that to deal with files at a storage service we have a mount point (if a storage service can handle more than one disk), a directory (which we use, e.g., for jobs' temporary directories but can be used by the WMS at will), and a file. To handle that complexity Henri is thinking of implementing a FileSystem class that handles all that, so that a disk simply has a set of file systems. Otherwiwe, welcome to x[y].first.at(x).second.front() :)

[henricasanova]

Current todo list:

• Get the storage branch to compile
• Asynchronous I/O in FileTransferThread::copyFileLocally() ?
• Update wrench_init if need be