Can a sector be re-committed after done?

[anorth]

Can a sector previously committed and proven, and subsequently either declared "done" or as a fault, be re-committed without re-sealing? Is there any need for the challenge embedded in the seal/proof to be recent?

As it stands at the moment, the actor spec contains no mechanism to detect or prevent committing a sector that has previously be part of a miner's sector set but subsequently removed. This may be fine, but I think we should be quite explicit about it. If not, we need to remember sectors forever or require some recency in the commitment.

If allowed, a faulted sector can be recovered and re-committed for only the gas cost. This is probably great for storage reliability as perceived by deal clients: a miner is more likely to still be storing their data in the future. With a grace period before deal arbitration, miners could recover from a brief outage at no risk to storage deal collateral. It reduces on-chain storage of all sectors forever. It does mean there is essentially no friction to a miner (without deals) rapidly scaling their commitments up or down as costs or returns fluctuate, though (e.g. with FIL price and opportunity cost of pledge collateral).

If disallowed, data in a faulted sector would need to be re-sealed before being re-committed. Since sealing is an expensive operation this represents a non-trival cost and penalty to failure (see also #407). Miners will likely have some maximum sealing rate that would limit the rate at which they could re-seal sectors after a large failure (e.g. a missed PoSt due to network cut → need to re-seal entire datacentre). At large scale, the cost of lost block rewards would be very significant. Depending on the expected rate of failures of various kinds, this would reduce profitability expectations. It's not so great for clients either: if their sector fails, their data is gone for good. However, it could incentivise physical distribution of large-scale operations in order to reduce correlated failures.

Originally asked in slack. cc @laser @sternhenri @dignifiedquire @whyrusleeping

[sternhenri]

It does mean there is essentially no friction to a miner (without deals) rapidly scaling their commitments up or down as costs or returns fluctuate, though (e.g. with FIL price and opportunity cost of pledge collateral).

What do you mean by this? How might a miner scale commitments down and then recommit without SEALing. My understanding is that SEALing is necessary in order to obtain the replica through which the commitment is proven. I take "scales commitments down" to mean "clears space on their drives," in which case there will be no way to recover from a fault (i.e. by proving the sector) without access to the appropriate replica, which is to say without SEALing.

Perhaps one could think up partial drops of sectors, but given the randomness in the inclusion proofs, assuming an appropriate query rate (proving period), that's just asking for trouble and would be irrational.

As always, I defer to @whyrusleeping @dignifiedquire here.

[anorth]

I take "scales commitments down" to mean "clears space on their drives,"

I didn't mean that, I meant just un-pledge storage without removing data. A miner might reduce their commitment, e.g., in order to recover pledge collateral and stake that value on some other crypto network that temporarily has a higher ROI, just leaving drives idle.

[sternhenri]

So as I understand it, the issue specifically revolves around declared faults. A first point there, as I understand it, declaring faults does not mean you can recover your storage collateral, even temporarily (ie it's not the same as "uncommitting": it's a fault, you have to stay collateralized and prove the storage quickly too, lest you lose collateral after a grace period, and have to reseal after a few more).

As it stands at the moment, the actor spec contains no mechanism to detect or prevent committing a sector that has previously be part of a miner's sector set but subsequently removed.

So perhaps we need to add an explicit note saying that collateral cannot be removed for a declared fault (though that PR has not quite shipped yet I think).

Now, I'm not quite sure what you mean by "done", and in fact am not sure how "deals" are detected as being complete (I would assume set at onset). So I can't speak to that. But on the issue of re-using the SEAL from a gracefully completed deal, we provide some protection from that in the form of ticket sampling for SEALING.

[whyrusleeping]

A sector can be resubmitted to the chain after being removed, for now. The idea of pulling in recent chain randomness for generating seals has been talked about, and that would prevent this (outside of the randomness time bounds). This hasnt made it into the spec yet (cc @nicola )

[nicola]

The current spec does not allow resubmission of sectors without re-sealing (from what I understand).

As part of the new PR that @dignifiedquire is writing, we are proposing a ~5 days period where you can introduce faulty sectors; math to ensure that part is secure will come before making that choice.

@whyrusleeping had an attack if this period would be larger