more details for post

filecoin-project · Jul 15, 2019 · a560992 · a560992
1 parent 7d2562b
commit a560992
Show file tree

Hide file tree

Showing 3 changed files with 169 additions and 87 deletions.
diff --git a/actors.md b/actors.md
@@ -677,7 +677,7 @@ func SubmitPost(proofs PoStProof, faults [FaultSet], recovered Bitfield, done Bi
 		Refund(msg.Value - feesRequired)
 	}
 
-	if !CheckPostProof(miner.SectorSize, proof, faults) {
+	if !VerifyPost(self.SectorSize, self.provingSet, proof, faults) {
 		Fatal("proof invalid")
 	}
 

diff --git a/mining.md b/mining.md
@@ -44,24 +44,27 @@ TODO: sectors need to be globally unique. This can be done either by having the
 At the beginning of their proving period, miners collect the proving set (the set of all live sealed sectors on the chain at this point), and then call `ProveStorage`. This process will take the entire proving period to complete.
 
 ```go
-func ProveStorage(sectorSize BytesAmount, sectors []commR, startTime BlockHeight) (PoSTProof, []FaultSet) {
-	var proofs []Proofs
-	var seeds []Seed
-	var faults []FaultSet
-	for t := 0; t < ProvingPeriod; t += ReseedPeriod {
-		seeds = append(seeds, GetSeedFromBlock(startTime+t))
-		proof, faultset := GenPost(sectors, seeds[t], vdfParams)
-		proofs = append(proofs, proof)
-		faults = append(faults, faultset)
-	}
-	return GenPostSnark(sectorSize, sectors, seeds, proofs), faults
+func ProveStorage(sectorSize BytesAmount, sectors []commR, startTime BlockHeight) (PoStProof, FaultSet) {
+    seed := GetRandFromBlock(startTime + POST_CHALLENGE_TIME)
+    GeneratePoSt(sectors, seed)
 }
 ```
 
 Note: See ['Proof of Space Time'](proofs.md#proof-of-space-time) for more details.
 
 The proving set remains consistent during the proving period. Any sectors added in the meantime will be included in the next proving set, at the beginning of the next proving period.
 
+```go
+// Verify a given PoSt.
+func VerifyPost(sectorSize BytesAmount, sectors []commR, startTime BlockHeight, proof PoStProof, faults FaultSet) bool {
+    seed := GetRandFromBlock(startTime + POST_CHALLENGE_TIME)
+    challenges := DerivePoStChallenges(seed, faults)
+
+    // Verify Snark
+    VerifyPoStSnark(sectorSize, sectors, proof, faults, challenges)
+}
+```
+
 #### Step 3: PoSt Submission
 
 When the miner has completed their PoSt, they must submit it to the network by calling [SubmitPoSt](actors.md#submitpost). There are two different times that this *could* be done.
@@ -147,9 +150,7 @@ func VerifyBlock(blk Block) {
 	}
 
 	// 6. Verify ElectionProof
-	randomnessLookbackTipset := RandomnessLookback(blk)
-	lookbackTicket := minTicket(randomnessLookbackTipset)
-	challenge := blake2b(lookbackTicket)
+    challenge := GetRandFromBlock(blk)
 
 	if !ValidateSignature(blk.ElectionProof, pubk, challenge) {
 		Fatal("election proof was not a valid signature of the last ticket")
@@ -208,6 +209,13 @@ func (state StateTree) LookupPublicKey(a Address) PubKey {
 	}
 	Fatal("can only look up public keys for BLS controlled accounts")
 }
+
+// Get the canonical randomness from a block.
+func GetRandFromBlock(blk) []byte {
+	randomnessLookbackTipset := RandomnessLookback(blk)
+	lookbackTicket := minTicket(randomnessLookbackTipset)
+	blake2b(lookbackTicket)
+}
 ```
 
 Ticket validation is detailed as follows:
@@ -376,11 +384,11 @@ TODO: Ensure that if a miner earns a block reward while undercollateralized, the
 
 ### Notes on Block Reward Application
 
-As mentioned above, every round, a miner checks to see if they have been selected as the leader for that particular round (see [Secret Leader Election](expected-consensus.md#secret-leader-election) in the Expected Consensus spec for more detail). Thus, it is possible that multiple miners may be selected as winners in a given round, and thus, that there will be multiple blocks with the same parents that are produced at the same block height (forming a TipSet). Each of the winning miners will apply the block reward directly to their actor's state in their state tree. 
+As mentioned above, every round, a miner checks to see if they have been selected as the leader for that particular round (see [Secret Leader Election](expected-consensus.md#secret-leader-election) in the Expected Consensus spec for more detail). Thus, it is possible that multiple miners may be selected as winners in a given round, and thus, that there will be multiple blocks with the same parents that are produced at the same block height (forming a TipSet). Each of the winning miners will apply the block reward directly to their actor's state in their state tree.
 
 Other nodes will receive these blocks and form a TipSet out of the eligible blocks (those that have the same parents and are at the same block height). These nodes will then validate the TipSet. The full procedure for how to verify a TipSet can be found above in [Block Validation](#block-validation). To validate TipSet state, the validating node will, for each block in the TipSet, first apply the block reward value directly to the mining node's account and then apply the messages contained in the block.
 
-Thus, each of the miners who produced a block in the TipSet will receive a block reward. There will be no lockup. These rewards can be spent immediately. 
+Thus, each of the miners who produced a block in the TipSet will receive a block reward. There will be no lockup. These rewards can be spent immediately.
 
 Messages in Filecoin also have an associated transaction fee (based on the gas costs of executing the message). In the case where multiple winning miners included the same message in their blocks, only the first miner will be paid this transaction fee. The first miner is the miner with the lowest ticket value (sorted lexicographically). More details on message execution can be found in the [State Machine spec](state-machine.md#execution-calling-a-method-on-an-actor).
 

diff --git a/proof-of-spacetime.md b/proof-of-spacetime.md
@@ -8,9 +8,11 @@ This document descibes Rational-PoSt, the Proof of Spacetime used in Filecoin.
 ### Definitions
 
 - **PoSt Proving Period**: The time interval in which a PoSt has to be submitted.
-- **PoSt Proving Time**: The time it takes to actually run a single PoSt.
-- **PoSt Generation Start Time**: The time at which the actual work of generating the PoSt should be started. This is some delta befre the end of the `Proving Period`, and as such less then a single `Proving Period`.
+- **POST_CHALLENGE_TIME**: The time offset at which the actual work of generating the PoSt should be started. This is some delta befre the end of the `Proving Period`, and as such less then a single `Proving Period`.
 
+- start = height x
+- challenge time = height x + y
+- on post submission: verify that challenge comes from block(x + y)
 
 ### Execution Flow
 
@@ -33,94 +35,166 @@ This document descibes Rational-PoSt, the Proof of Spacetime used in Filecoin.
                                                                        └ ─ ─ ─ ─ ─ ─ ─ ─ ┘
 ```
 
-- **Setup Parameters**
-  - Same as Public parameters.
-- **Public Parameters**
-  - `challenge_count`: Number of challenges to be asked at each iteration.
-  - `sector_size`: Size of the sealed sector in bytes.
-  - `sectors_count`: Number of sectors over which the proof is performed.
-  - `challenge_bits`: Number of bits in one challenge (length of a merkle path)
-  - `seed_bits`: ?
-- **Public Inputs**
-  - `CommRs : [sectors_count]Hash` CommRs must be ordered by their timestamp on the blockchain.
-- **Private Inputs**
-  - `replicas: [sectors_count]SealedSector`: sealed sectors
-- **Proof**
-  - `post_proofs [post_periods_count]PoRepProof`
+TODO: Add post submission to the diagram.
 
-### Methods
 
-#### `Prove(ChallengeSeed, PublicParameters, PublicInputs, PrivateInputs) -> (Proof, Faults)`
 
-**TODO**: describe faults
 
-`Prove` is gets called when the `Generation Start Time` is hit (in every `Proving Period`).
+### High Level API
 
-- `(challenges, challenged_sectors) = derive_challenges(challenge_count, challenge_seed)`
-- `porep_proof = OnlinePoRep.prove(challenges, challenged_sectors, commR, replica)`
-- Output `(porep_proofs, faults)`
+#### Generation
 
-#### `Verify(PublicParameters, PublicInputs, Proof, Faults, ChallengeSeed) -> bool`
+```go
+func GeneratePoSt(sectorSize BytesAmount, sectors []commR) (PoStProof, FaultSet) {
+    // Generate the Merkle Inclusion Proofs + Faults
 
-**TODO:** Handle the passed in `Faults**
-**TODO:** Verify integration with challenges and verification
+    inclusionProofs := []
+    faults := NewFaultSet()
 
-- `(challenges, challenged_sectors) = derive_challenges(challenge_count, challenge_seed)`
-- Assert: `verify_final_challenge_derivation(challenges, partial_challenge, challenge_seed, challenge_bits)`
-- Assert: `PoSt.Verify(CommRs, challenges, post_proof)`
+    for n in 0..POST_CHALLENGES_COUNT {
+        attempt := 0
+        'inner: for {
+            challenge := DerivePoStChallenge(seed, n, faults, attempt)
+            sector := challenge % sectorSize
+            // check if we landed in a previously marked faulty sector
+            if !faults.Contains(sector) {
+                attempt += 1
+                continue
+            }
 
+            challenge_value = challenge / sectorSize
+            inclusionProof, isFault := GenerateMerkleInclusionProof(sector, challenge_value)
+            if isFault {
+                // faulty sector, generate a new challenge
+                faults.Add(sector)
+                attempt += 1
+            } else {
+                // no fault, move on to the next challenge
+                inclusionProofs[n] = inclusionProof
+                break 'inner
+            }
+        }
+    }
 
-### `GenerateChallengeSeed(t: Height) -> ChallengeSeed`
+    // Generate the snark
+    challenges := DerivePoStChallenges(sectorSize, seed, faults)
 
-Before calling `Prove`, first this executed.
+    snark_proof := GeneratePoStSnark(sectorSize, challenges, sectors, inclusionProofs)
 
-- `lookback_ticket = minTicket(RandomnessLookback(blk(t)))`
-- `challenge_seed = blake2s(lookbackTicket)`
-- Output `challenge_seed`
+    proof := PoStProof {
+        snark: snark_proof
+    }
 
-### Challenge Stream
+    return proof, faults
+}
+```
+
+#### Verification
+
+
+```go
+func VerifyPoSt(sectorSize BytesAmount, sectors []commR, seed []byte, proof PoStProof, faults FaultSet) bool {
+    challenges := DerivePoStChallenges(sectorSize, seed, faults)
+
+    // Verify snark
+    VerifyPoStSnark(sectorSize, challenges, sectors)
+}
+```
+
+
+#### Types
+
+```go
+type PoStProof struct {
+    snark []byte
+}
+```
+
+#### Challenge Derivation
+
+```go
+// Derive the full set of challenges for PoSt.
+func DerivePoStChallenges(sectorSize BytesAmount, seed []byte, faults FaultSet) [POST_CHALLENGES_COUNT][]byte {
+    challenges := []
+
+    for n in 0..POST_CHALLENGES_COUNT {
+        attempt := 0
+        'inner: for {
+            challenge := DerivePoStChallenge(seed, n, faults, attempt)
+
+            // check if we landed in a faulty sector
+            sector := challenge % sectorSize
+            if !faults.Contains(sector) {
+                // Valid challenge
+                challenges[n] = challenge
+                break 'inner
+            }
+            // invalid challenge, regenerate
+            attempt += 1
+        }
+    }
+
+    return challenges
+}
+
+// Derive a single challenge for PoSt.
+func DerivePoStChallenge(seed []byte, n Uint, attempt Uint) []byte {
+    n_bytes := WriteUintToLittleEndian(n)
+    data := concat(seed, n_bytes, WriteUintToLittleEndian(attempt))
+    challenge := blake2b(data)
+}
+```
 
-In order to reduce verification costs inside the circuit, challenge generation is split into two parts, partial challenge generation, and final challenge generation.
 
-#### `derive_challenges(challenge_count, seed)`
+### PoSt Circuit
 
-- `partial_challenge = derive_partial_challenge(seed)`
-- `while all_challenges.len() < count`
-  - `(challenges, challenged_sectors)`
-  - `all_challenges.extend(challenges)`
-  - `all_challenged_sectors.push(challenged_sectors)`
-- Output (`all_challenges[0..challenge_count]`, `all_challenged_sectors[0..challenge_count]`)
+#### Public Parameters
 
-#### `derive_partial_challenge(seed)`
+*Parameters that are embeded in the circuits or used to generate the circuit*
 
-- `partial_challenge = H(seed | 0)`
+- `POST_CHALLENGES_COUNT: UInt`: Number of challenges.
+- `POST_TREE_DEPTH: UInt`: Depth of the Merkle tree. Note, this is `(log_2(Size of original data in bytes/32 bytes per leaf))`.
+- `SECTOR_SIZE: UInt`:
+- `MAX_SECTORS_COUNT`: maximum number of sectors that can be proven with a single post
 
-#### `derive_final_challenges(partial_challenge, seed, sectors_count, challenge_bits)`
+#### Public Inputs
 
-- `mixed = partial_challenge - seed`
-- `mixed_bytes = fr_to_bytes(bits_to_fr(mixed))`
-- `for chunk in mixed_bytes.chunks(challenge_bits)`
-  - `challenge = 0`
-  - `place = 1`
-  - `for bit in chunk`
-    - `if bit`
-      - `challenge += place`
-    - `place = place << 1`
-  - `challenged_sector = ???` **FIXME**
-  - `challenges.push(challenge)`
-  - `challenged_sectors.push(challenged_sectors)`
-- Output `(challenges, challenged_sectors)`
+*Inputs that the prover uses to generate a SNARK proof and that the verifier uses to verify it*
 
-#### `verify_final_challenge_derivation(challenges, partial_challenge, seed, challenge_bits) -> bool`
+- `CommRs: [POST_CHALLENGES_COUNT]Fr`: The Merkle tree root hashe of all CommRs
+- `InclusionPaths: [POST_CHALLENGES_COUNT]Fr`: Inclusion paths for the replica leafs. (Binary packed bools)
+
+{{% notice todo %}}
+**Todo**: `CommRs` should be optimized, by combining them into a single merkle tree, with a single root `CommA`.
+Benchmark this first before commiting.
+{{% /notice %}}
+
+#### Private Inputs
+
+*Inputs that the prover uses to generate a SNARK proof, these are not needed by the verifier to verify the proof*
+
+- `InclusionProofs: [POST_CHALLENGES_COUNT][TREE_DEPTH]Fr`: Merkle tree inclusion proofs.
+- `InclusionValues: [POST_CHALLENGES_COUNT]Fr`: Value of the encoded leaves for each challenge.
+
+
+#### Circuit
+
+##### High Level
+
+In high level, we do 1 check:
+
+1. **Inclusion Proofs Checks**: Check the inclusion proofs
+
+##### Details
+
+```go
+for c in range POST_CHALLENGES_COUNT {
+  // Inclusion Proofs Checks
+  assert(MerkleTreeVerify(CommRs[c], InclusionPath[c], IncludionProof[c], InclusionValue[c]))
+}
+```
 
-- `shift_factor = bytes_to_fr(1 << challenge_bits)`
-- `packed = Fr(0)`
-- `for challenge in challenges`
-  - `fr_challenge = bytes_to_fr(challenge)`
-  - `packed = packed * shift_factor`
-  - `packed = packed + fr_challenge`
-- `fr_seed = bytes_to_fr(seed)`
-- `fr_partial = bytes_to_fr(partial_challenge)`
-- `fr_mixed = fr_mixed + packed`
+#### Verification of PoSt proof
 
-- Output `fr_partial == fr_mixed`
+- SNARK proof check: **Check** that given the SNARK proof and the public inputs, the SNARK verification outputs true
+- Challenges check: rederive the challenges, based on the seed, and check that they are equal.