Skip to content

Latest commit

 

History

History
130 lines (94 loc) · 13.2 KB

fip-0025.md

File metadata and controls

130 lines (94 loc) · 13.2 KB
Raw
fip title author discussions-to status type category created spec-sections
0025
Handle Expired Deals in ProveCommit
ZenGround0 (@ZenGround0)
Deferred
Technical
Core
2021-09-06
specs-actors

FIP-0025: Handle Expired Deals in ProveCommit

Simple Summary

Add a new parameter to ProveCommitSector and ProveCommitAggregate so that storage providers can commit sectors with some expired deals.

Abstract

Today storage providers are unable to commit sectors with expired deals because the sector's data commitment cannot be determined on chain with deal PieceCID and PieceSize info removed from state. To make this possible the ProveCommit and ProveCommitAggregate messages can be adapted to take in the PieceInfos of the expired deals. Then the data committment can be computed from the provided expired piece info and the market actor's data.

Change Motivation

Occasionally deals expire between PreCommit and ProveCommit of a sector and in these cases sectors containing these deals will fail to commit. This is bad for storage providers committing sectors with deals and bad for the non-expired deals that would become active if the sector were committed. While offchain logic can help in many cases there is always a small risk of deals expiring even with optimal offchain logic as long as there is a delay between precommit and provecommit.

This change is deferred (1) because the design adds a lot of complexity for a small useability gain (2) there is not broad agreement on what the storage provider SLA should be exactly and it is possible this change takes the network slightly in the wrong direction.

Specification

Introduce a new type ExpiredDealInfo for use in method parameters

type ExpiredDealInfo struct {
     DealID    abi.DealID
     PieceCID  cid.Cid
     PieceSize abi.PaddedPieceSize
}

ProveCommitSector

  1. ProveCommitSector takes in an array of ExpiredDealInfos as a new parameter.
  2. If all deals are expired the method fails and this sector cannot be committed.
  3. Market actor ComputeDataCommitment is extended to take in an array of ExpiredDealInfos. It validates that all expired deals are actually expired by asserting that each dealID is less than the global next dealID and the deal is not found on chain. If any expired deal fails validation the method will error, failing the caller. After validating ComputeDataCommitment then uses expired deal information to correctly compute the data commitment for ProveCommitSector.
type SectorDataSpec struct {
     DealIDs      []abi.DealID
     SectorType   abi.RegisteredSealProof
     ExpiredDeals []ExpiredDealInfo
}

type ComputeDataCommitmentParams struct {
     Inputs []*SectorDataSpec
}

ConfirmSectorProofsValid

  1. Market actor ActivateDeals error semantics are changed. It does not error if some deals are not found. Instead it returns a bitfield of the indexes of input deals that were successfully activated. If no deals can be activated the method will error.
  2. Additionally ActivateDeals is modified to return the weight of the sector being activated.
type ActivateDealsReturn struct {
     ValidInputs bitfield.BitField
     Weight SectorWeights
}
  1. ConfirmSectorProofsValid passes all DealIDs specified at precommit to ActivateDeals and reads the output to filter out expired deals.
  2. The SectorOnChainInfo DealIDs, DealWeight and VerifiedDealWeight fields are now assigned the non-expired deal ids, and the newly calculated deal weights.

ProveCommitAggregate

  1. ProveCommitAggregate takes in an array of ExpiredDealInfos and does internal processing to index expired deal infos by deal id.
  2. When iterating through pre commits ProveCommitAggregate checks each deal id for membership in the expiration set including the expired information to ComputeDataCommittmentInputs.
  3. ProveCommitAggregate keeps track of whether a sector has all expired deals and if so it drops the precommit from the confirmation set without erroring.
  4. All changes to ConfirmSectorProofsValid directly apply

Design Rationale

An alternate approach would be to track old piece info sufficient to calculate the data committments of sectors with expired deals on chain in the market actor. However managing this on chain map and its garbage collection is less efficient and more complicated than pushing this off to the storage provider and adding a byte to committment message params.

The ExpiredDeals type is used instead of a cbor map because of IPLD format restrictions disallowing integer keyed maps. Adding this paramter should incur only one extra byte in the ProveCommit and ProveCommitAggregate parameters of the common case where no deals are expired.

Although the state machine has knowledge about what deals are expired at the entrypoint of ProveCommit and ProveCommitAggregate in this proposal it is rediscovered during the second step, i.e. confirmSectorProofsValid by reading the return value of ActivateDeals. An alternate proposal could modify the parameters to the second stage of committment to include the expired deal set so that activation would only be called with the non-expired deal ids. This was passed over because it is more invasive in the ProveCommit case. It would require either adding the expired deal ids to the power cron queue or changing ProveCommit from a read only method to a state mutating method and modifying sector precommit infos. The later case is not bad but modifying precommit infos between precommit and final commit adds another state mutation and somewhat complicates reasoning about state.

The error semantics and bitfield return value change proposed for ActivateDeals is modeled after the existing proposal for changing PublishStorageDeals error handling.

Overwriting sector info with only the non-expired deal ids preserves the invariant that all deal ids in a sector are activated at some point during the sector's lifetime which means this proposal does not need to extend its scope to miner actor processing beyond sector committment.

Backwards Compatibility

Four miner and market actor method signatures change in this proposal: market.ComputeDataCommittment, market.ActivateDeals, miner.ProveCommitSector, and miner.ProveCommitAggregate. This means an actor major version change is needed for this proposal. The market actor methods should only be called internally by user called actor methods (except for erroneous unauthorized calls) so there are no complications changing signatures. The miner actor method parameter changes will have complications. Messages from before the upgrade will fail to unserialize to the new format. The upgrade should include a best effort stop on sector commitment messages in the hour before the upgrade epoch.

No state migration is needed.

Test Cases

Security Considerations

There are no security implications to this proposal.

Incentive Considerations

There is an incentive consideration related to this conversation. If designed incorrectly this proposal could allow arbitrary data setting without participating in the filecoin economy at the storage market layer. For this reason this proposal restricts expired deal id specification to the deal ids already validated at precommit. This prevents storage providers from specifying arbitrary unrelated expired deal ids from different storage providers that did not require market interaction from the storage provider. Note that this proposal can't prevent using different PieceCids then originally specified in PublishStorageDeals for a DealID. However doing so does not impact desired storage market interaction requirements as a storage provider would need to seal the true data and publish a deal for the overwritten data anyway

After this proposal deal clients will not have a guaranteed burn of precommit deposit by the miner if their deal id is included in precommit but not activated. This removes a stronger disincentive for storage providers to not let deals expire before committment. TODO: this needs cryptoecon analysis and sign off

Sector weights must be recalculated without expired deals to fairly reduce miner power from including deals no longer being proven.

Recalculating weights at commit time instead of precommit time will slightly increase sector power since the deadspace with no deals active between precommit and commit will be removed from the sector weight calculation. TODO: this needs cryptoecon analysis and sign off

Product Considerations

Storage provider software will need to change slightly to create correct parameters for sector committment methods. The only requirement is adding a new empty param for expired deals. Storage provider software will be able to use this new argument to provide more cost efficient error handling by explicitly listing deals as expired and committing the sector instead of letting the sector drop and burning precommit deposit. However this is not a requirement at time of upgrade.

Implementation

specs-actors PR: TODO

Copyright

Copyright and related rights waived via CC0.