SGX compute committees

[bennetyee]

Fully support TEE-based (SGX) transactions as a choice in the network.

Details

We should fully support TEE-based (SGX) committees, the BFT registry should allow registration of TEE-enabled nodes and the BFT roothash should validate that results are actually being computed in TEEs. We should be able to test hardware SGX as part of our CI process.

Linked Issues

• Support an SGX compute committee
• sgx: Periodically re-attest with IAS
• E2E tests with multiple runtimes - 2d
• Should TEE attestations etc be included with commitments in the block header
• Roothash should validate commit attestation for TEE-enabled nodes

Integration:

• run compute committee only
  • cross-cluster communication pr
• Buidlkite SGX Agents
• gateway support for accessing other runtimes issue
  • (unknown, possibly more than one task)
• benchmark cross-cluster testnet
• figure out ibm node storage requirements
• monitoring & logs for ibm clusters
• Bootstrap: registering the sgx entity & runtime
• setup production ibm cluster (https://github.com/oasislabs/private-ops/issues/662)
  • note: this one is pretty straightforward but should be done last, as it will require spawning another SGX machine therefore increasing the costs

Fidelity:

• tying runtime actions/output to attestation (the "tie") issue
• incorporating the tie into ledger issue
• validating the tie in...
  • validators issue
  • gateway (?)
  • web3c clients (?)
• make sure key manager adequately controls access issue
  • (unknown, possibly many, many tasks)

Improvements:

• removing high-privilege workarounds
  • worker sandbox <-> aesmd issue
  • container <-> device issue
  • Figure out if Intel SGX breaks sandbox isolation (Figure out if Intel SGX breaks sandbox isolation. #1433)
• continuous delivery of compute committee in sync with compatible rest-of-the-network issue
• scan for crosscluster nodeport access
• ibmcloud ks cluster-config --export workaround

Actual production-ness:

• production IAS credentials issue
• non-debug launch authorization issue
• write policy on acceptable attestations issue (repurposed)
• More goes here

[pro-wh]

@ryscheng asked me to add some items to the issue, so I'll edit the description

[ptrus]

wanted to create a separate aws-ibm epic issue, but saw that some related issues we're already here - so i've addded some more issues under this one