Complete spec for sector sets and related datastructures

[ZenGround0]

A lot of this work seems close to landing. This issue calls out the need to get all of this great work written up in the spec and canonicalized to facilitate development storage protocol faults.

• Sector Sets. These are currently unspecified. However there is a lot of work in this issue that looks close to landing.
• Sector IDs. In the above issue the discussion indicates that we can give the miner full selection over the sector ID. This is in conflict with the current spec for CommitSector. We need to settle on a sector ID assignment scheme.
• Bitfields are currently not speced (encoding still TBD). However this proposal looks accepted. And the chosen rle+ encoding scheme is referenced in the spec. We just need to close the loop and specify how bitfields are encoded.
• Bitfield/FaultSet/SectorSet Operations. These show up frequently unspecified in SubmitPoSt. This is discussed in enough detail here that implementors can probably make progress with only a Bitfield and SectorSet spec. For completeness implementers need to be able to write:
  • ValidateFaultSets([]FaultSet, Bitfield, Bitfield)
  • AggregateBitfileds([]FaultSet)
  • Subtract(Bitfield, Bitfield)
  • Filter(SectorSet, []FaultSet)
  • Sizeof(SectorSet)
• FaultSets. These are mostly specified when a Bitfield spec is merged but the encoding still TBD implies there is a little work to do after this happens. Also again looking at Why's comments here it seems like there is still some uncertainty on how to timestamp faults.

[pooja]

cc @pooja @dignifiedquire

[laser]

@ZenGround0

In the above issue the discussion indicates that we can give the miner full selection over the sector ID. This is in conflict with the current spec for CommitSector. We need to settle on a sector ID assignment scheme.

FYI: The spec for CommitSector has recently changed to accommodate miners selecting sector IDs.

[dignifiedquire]

spoke with @ZenGround0 and he will be working with me on the Sector Sets

[whyrusleeping]

The bitfield operations in submit post should be clearer with #310

[pooja]

Let's review this PR: #333
Also related is: #116

[mishmosh]

@ZenGround0 would you consider sector sets spec'd to your liking now? (if yes please close)