Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rational PoSt #332

Merged
merged 32 commits into from
Aug 21, 2019
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
5a4517a
add base-post
dignifiedquire Jun 5, 2019
1a54e31
fix typos
dignifiedquire Jun 5, 2019
bddcf39
more typos
dignifiedquire Jun 5, 2019
57356a8
document challenge derivation
dignifiedquire Jun 6, 2019
6439626
reorg and add to the spec
dignifiedquire Jul 11, 2019
d3a5fa5
more details for post
dignifiedquire Jul 15, 2019
b1632bf
update verification call in postsubmission
dignifiedquire Jul 15, 2019
5aee5b8
address some feedback
dignifiedquire Jul 15, 2019
200933b
fix some typos
dignifiedquire Jul 15, 2019
c970d7f
more cleanup
dignifiedquire Jul 15, 2019
868ba02
first pass at commitment of faults
dignifiedquire Jul 26, 2019
6988859
fix challenge generation, now that I implemented it
dignifiedquire Jul 29, 2019
01e4150
use sortedSectors
dignifiedquire Jul 29, 2019
efd05b3
update other parts of the spec to the new construction
dignifiedquire Jul 30, 2019
d93f387
Update mining.md
dignifiedquire Jul 30, 2019
986d02e
rework challenge derivation and faults tracking
dignifiedquire Jul 31, 2019
e278a73
Define ordering of various arrays and reuse SectorSet
dignifiedquire Jul 31, 2019
cbc23f9
updates and clarifications from talking with nicola
dignifiedquire Jul 31, 2019
bafe29a
apply feedback from @laser
dignifiedquire Jul 31, 2019
11906c0
Update actors.md
dignifiedquire Aug 1, 2019
de87ac0
address more CR
dignifiedquire Aug 2, 2019
06c591a
fixup some more CR, especially from where the randomness gets pulled
dignifiedquire Aug 7, 2019
b444e03
fix typos
dignifiedquire Aug 7, 2019
732bbdd
fix: add notes about penalization and rational for early fault submis…
dignifiedquire Aug 19, 2019
d32ff13
small language clarification
dignifiedquire Aug 19, 2019
adab0e9
another wording fix
dignifiedquire Aug 19, 2019
143a3bf
update logic to account for late post submissions
dignifiedquire Aug 20, 2019
2130a85
handle super late submissions
dignifiedquire Aug 20, 2019
3b2b247
Update actors.md
dignifiedquire Aug 21, 2019
1290417
Update actors.md
dignifiedquire Aug 21, 2019
9b42555
Update proof-of-spacetime.md
dignifiedquire Aug 21, 2019
ef5022d
apply suggestions from @anorth
dignifiedquire Aug 21, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 70 additions & 109 deletions actors.md
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,12 @@ type StorageMinerActorState struct {
## when a PoSt is submitted (not as each new sector commitment is added).
provingSet &SectorSet

## Faulty sectors reported since last SubmitPost, up to the current PoSt challenge time.
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
currentFaultSet BitField

## Faults for the next round of PoSt.
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
nextFaultSet BitField

## Sectors reported during the last PoSt submission as being 'done'. The collateral
## for them is still being held until the next PoSt submission in case early sector
## removal penalization is needed.
Expand Down Expand Up @@ -497,7 +503,6 @@ type MinerInfo struct {
sectorSize BytesAmount

}
```

#### Methods

Expand All @@ -521,6 +526,9 @@ type MinerInfo struct {
| `IsLate` | 16 |
| `PaymentVerifyInclusion` | 17 |
| `PaymentVerifySector` | 18 |
| `ReportFaultySectors` | 19 |
| `RecoverFaultySectors` | 20 |
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
| `AddFaults` | 21 |

#### `Constructor`

Expand Down Expand Up @@ -590,11 +598,6 @@ func CommitSector(sectorID SectorID, commD, commR, commRStar []byte, proof SealP
Fatal("not enough collateral")
}

// ensure that the miner cannot commit more sectors than can be proved with a single PoSt
if miner.Sectors.Size() >= POST_SECTORS_COUNT {
Fatal("too many sectors")
}

// Note: There must exist a unique index in the miner's sector set for each
// sector ID. The `faults`, `recovered`, and `done` parameters of the
// SubmitPoSt method express indices into this sector set.
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -632,40 +635,22 @@ func CollateralForPower(power BytesAmount) TokenAmount {
```sh
type SubmitPost struct {
proofs PoStProof
faults [FaultSet]
recovered Bitfield
done Bitfield
doneSet Bitfield
} representation tuple
```

**Algorithm**

{{% notice todo %}}
TODO: GenerationAttackTime
{{% /notice %}}

```go
func SubmitPost(proofs PoStProof, faults []FaultSet, recovered Bitfield, done Bitfield) {
if msg.From != miner.Worker {
func SubmitPost(proofs PoStProof, doneSet Bitfield) {
if msg.From != self.Worker {
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
Fatal("not authorized to submit post for miner")
}

// ensure recovered is a subset of the combined fault sets, and that done
// does not intersect with either, and that all sets only reference sectors
// that currently exist
allFaults = AggregateBitfields(faults)
if !miner.ValidateFaultSets(faults, recovered, done) {
Fatal("fault sets invalid")
}

feesRequired := 0

if chain.Now() > self.ProvingPeriodEnd+GenerationAttackTime(self.SectorSize) {
// slashing ourselves
SlashStorageFault(self)
return
} else if chain.Now() > self.ProvingPeriodEnd {
feesRequired += ComputeLateFee(self.power, chain.Now()-self.provingPeriodEnd)
if chain.Now() > self.ProvingPeriodEnd {
feesRequired += ComputeLateFee(self.power, chain.Now() - self.provingPeriodEnd)
}

feesRequired += ComputeTemporarySectorFailureFee(self.sectorSize, recovered)
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -679,68 +664,46 @@ func SubmitPost(proofs PoStProof, faults []FaultSet, recovered Bitfield, done Bi
TransferFunds(msg.From, msg.Value-feesRequired)
}

if !CheckPostProof(miner.SectorSize, proof, faults) {
nextProvingPeriodEnd := self.ProvingPeriodEnd + ProvingPeriodDuration(self.SectorSize)

var seed
if chain.Now() < self.ProvingPeriodEnd {
// good case, submitted in time
seed = GetRandFromBlock(self.ProvingPeriodEnd - POST_CHALLENGE_TIME)
} else {
// bad case, submitted late, need to take new proving period end as reference
seed = GetRandFromBlock(nextPovingPeriodEnd - POST_CHALLENGE_TIME)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some obvious cases where this could fail if SubmitPoSt is submitted too early, or races against the proving period end arriving. I guess they're implicitly fatal, but the spec should eventually be specific.


faultSet := self.currentFaultSet

if !VerifyPoSt(self.SectorSize, self.provingSet, seed, proof, faultSet) {
dignifiedquire marked this conversation as resolved.
Show resolved Hide resolved
Fatal("proof invalid")
}

// combine all the fault set bitfields, and subtract out the recovered
// ones to get the set of sectors permanently lost
permLostSet = allFaults.Subtract(recovered)
// The next fault set becomes the current one
self.currentFaultSet = self.nextFaultSet
self.nextFaultSet = EmptySectorSet()

// burn funds for fees and collateral penalization
self.BurnFunds(CollateralForSize(self.SectorSize*permLostSet.Size()) + feesRequired)
// TODO: penalize for faults
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too dooo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, this will stay for now and a new issue for handling post faults will be created


// update sector sets and proving set
miner.Sectors.Subtract(done)
miner.Sectors.Subtract(permLostSet)
// Remove doneSet from the current sectors
self.Sectors.Subtract(doneSet)

// update miner power to the amount of data actually proved during
// the last proving period.
oldPower := miner.Power
// Update miner power to the amount of data actually proved during the last proving period.
oldPower := self.Power

miner.Power = (miner.ProvingSet.Size() - allFaults.Count()) * miner.SectorSize
StorageMarket.UpdateStorage(miner.Power - oldPower)
self.Power = (self.ProvingSet.Size() - faultSet.Count()) * self.SectorSize
StorageMarket.UpdateStorage(self.Power - oldPower)

miner.ProvingSet = miner.Sectors
self.ProvingSet = self.Sectors

// Updating proving period given a fixed schedule, independent of late submissions.
miner.ProvingPeriodEnd = miner.ProvingPeriodEnd + ProvingPeriodDuration(miner.SectorSize)
self.ProvingPeriodEnd = nextProvingPeriodEnd

// update next done set
miner.NextDoneSet = done
miner.ArbitratedDeals.Clear()
}

func ValidateFaultSets(faults []FaultSet, recovered, done BitField) bool {
var aggregate BitField
for _, fs := range faults {
aggregate = aggregate.Union(fs.BitField)
}

// all sectors marked recovered must have actually failed
if !recovered.IsSubsetOf(aggregate) {
return false
}

// the done set cannot intersect with the aggregated faults
// you can't mark a fault as 'done'
if aggregate.Intersects(done) {
return false
}

for _, bit := range aggregate.Bits() {
if !miner.HasSectorByID(bit) {
return false
}
}

for _, bit := range done.Bits() {
if !miner.HasSectorByID(bit) {
return false
}
}

return true
self.NextDoneSet = done
self.ArbitratedDeals.Clear()
}

func ProvingPeriodDuration(sectorSize uint64) Integer {
Expand Down Expand Up @@ -768,34 +731,6 @@ type SlashStorageFault struct {

**Algorithm**


## late submission and resubmission

- If After provingPeriodEnd
- post submission now requires paying pay late submission fee
- (fixed cost, dependent on the total storage size)
- [implicit] loose all power, can be explicitly slashed for
- post submission returns your power immediately
- If After `GenerationAttackTimeout` (<< 1 proving period)
- .... nothing changes atm
- If After `PoStTimeout` (< 1 proving period)
- [explicit - slashStorageFault] loose all storage collateral
- clients can arbitrate deals with the miner now
- post submission now requires paying both late fee + lost storage collateral
- the valid post submission returns your power immediately
- If After `SectorFailureTimeout` (> 1 proving period)
- [explicit - slashStorageFault] loose all sectors
- [implicit] resets proving period, as they need to resubmit all sectors after this
- there is now no way to reintroduce the sectors, unless they are resubmitted, etc
- power can not be recovered anymore

- If [miner] does not call postsubmit


Notes:
- Should post submission only return your power, after the following proving period, when after the generation attack timeout
- Is one proving period enough time for abitration of faulty deals for a client?

```go
func SlashStorageFault() {
// You can only be slashed once for missing your PoSt.
Expand Down Expand Up @@ -1154,7 +1089,7 @@ func PaymentVerifyInclusion(extra PieceInclusionVoucherData, proof InclusionProo
if !has {
Fatal("miner does not have required sector")
}

return ValidatePIP(self.SectorSize, extra.PieceSize, extra.CommP, commD, proof.Proof)
}
```
Expand Down Expand Up @@ -1186,11 +1121,37 @@ func PaymentVerifyInclusion(extra BigInt, proof Bytes) {
if len(proof) > 0 {
Fatal("unexpected proof bytes")
}

return self.HasSector(extra)
}
```

#### `AddFaults`

**Parameters**

```sh
type AddFaults struct {
faults FaultSet
} representation tuple
```

**Algorithm**

```go
func AddFaults(faults FaultSet) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually think we should simplify this all the way to just:

func AddFaults(faults FaultSet) {
  self.currentFaultSet = Merge(self.currentFaultSet, faults)
}

If you submit faults before you submit your post, they apply to the post you still havent submitted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we simplify this, it means you are not allowed to submit any faults between challenge time and post submission. Which becomes an issue if you miss a post, as you can't post new faults anymore.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would allow a miner to accept storage deals, commit and then discard sectors containing the storage, and then add faults each proving period thereafter. Since the fault set uses RLE, the cost of a the AddFault message is constant over a large number of sectors, and the cost per sector drops without bound. At some point the gas costs for faulting on all deals will drop below the value of the deals.

We're going to need some mechanism to limit the number of times a sector can be faulted or at least a mechanism for timing-out faulted sectors so deal arbitration and payment cancellation can occur.

challengeBlockHeight := self.ProvingPeriodEnd - POST_CHALLENGE_TIME

if VM.CurrentBlockHeight() < challengeBlockHeight {
// Up to the challenge time new faults can be added.
self.currentFaultSet = Merge(self.currentFaultSet, faults)
} else {
// After that they are only accounted for in the next proving period
self.nextFaultSet = Merge(self.nextFaultSet, faults)
}
}
```

### Payment Channel Actor

- **Code Cid:** `<codec:raw><mhType:identity><"paych">`
Expand Down
1 change: 1 addition & 0 deletions content/docs/proof-of-spacetime.md
1 change: 1 addition & 0 deletions content/menu/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ headless: true

* [Proof-of-Replication]({{< relref "/docs/zigzag-porep.md" >}})
* [Circuit]({{< relref "/docs/zigzag-circuit.md" >}})
* [Proof-of-Spacetime]({{< relref "/docs/proof-of-spacetime.md" >}})

[**Glossary**]({{< relref "/docs/definitions.md" >}})

Expand Down
8 changes: 7 additions & 1 deletion data-structures.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,14 @@ type Bitfield Bytes

### SectorSet

A sector set stores a mapping of sector IDs to the respective `commR`s.

```sh
type SectorSet map{SectorID:Bytes}
```

{{% notice todo %}}
Define me
Improve on this, see https://github.com/filecoin-project/specs/issues/116
{{% /notice %}}


Expand Down
Loading