[LibOS] Add Recovery feature to Encrypted files

[dimakuv]

Description of the feature

Intel SGX SDK's recovery code must be ported to Gramine. Kailun's team can work on this, but after the v1.8 release.

See #1972

Why Gramine should implement it?

Need recovery of partially-written files (before crash).

[kailun-qin]

Basically, SGX SDK's file recovery is implemented based on:

• a recovery file to store all the data that is about to change;
• an update flag in the metadata node to indicate that a write transaction have begun.

This "shadow" recovery file has a dump of the cached file blocks. Before we start to save the main file contents we will set the update_flag in the file's metadata node. It'll be cleared only when the transaction is over -- when the main file contents are fully dumped and the updated metadata node is written. If we open an encrypted file and find that the update_flag is set, we know that an error occured and a recovery process will start, and we will revert all the partial changes (the original data is in the recovery file) and return to the last known good state. Specifically:

• During file flush (https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_tprotected_fs/file_flush.cpp#L73-L127):
  • save all the data that is about to change to the recovery file (pairs of address + 4KB node): https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_tprotected_fs/file_flush.cpp#L130-L201
    • metadata always changes as it saves the GMAC and key of the first node (recall: encrypted files representation on host storage and in SGX enclave memory)
    • update_flag in the saved metadata node is always clear
    • if we crash in the middle or right after this stage, when opening the file we won’t do anything, the flag hasn't been set yet -- the recovery file will be ignored
  • set update_flag in the file’s metadata node: https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_tprotected_fs/file_flush.cpp#L204-L233
    • only set on the host storage, stays clear in memory
    • if we crash after this stage and before the process finishes, we will do a recovery when we will re-open the file
  • encrypt data and crypto nodes and update all the GMACs; encrypt the metadata node's encrypted part and update the plaintext part
  • write all the changes to the file
    • clear the flag (happens automatically when the updated metadata node is written)
• During file open: https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_tprotected_fs/file_init.cpp#L417-L437:
  • check update_flag in the metadata node, if it's set:
    • close the file and start the recovery process (done in the untrusted): https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_uprotected_fs/sgx_uprotected_fs.cpp#L360-L508:
      • open the main file and the recovery file
      • copy all the original data from the recovery file over the existing data in the appropriate locations in the file
      • clear the flag (happens automatically when the original metadata node is restored)
      • close both files
    • re-open the file and re-read the metadata node
    • verify that the flag is clear now
• During file close, erase the recovery file: https://github.com/intel/linux-sgx/blob/sgx_2.21/sdk/protected_fs/sgx_tprotected_fs/file_init.cpp#L587

[mkow]

Doesn't this make the I/O to encrypted files significantly slower?