Non Interactive PoRep: improving onboarding pipeline allowing for a real separation between storage and proving, without compromising security

[lucaniz]

Motivation

PoRep is currently interactive and it is composed by two steps: PreCommit and ProveCommit.
At PreCommit SP puts down a collateral (PCD) and wait 150 epochs in order to receive a challenge seed from the chain which enables the ProveCommit step. This means that

• Mining pipeline is not flexible. In order to add a sector to the network, that sector needs to be first precommited and ProveCommit needs to happen within a time window lasting. MaxSectorDuration epochs. Note that between PreCommit and ProveCommit a waiting time of 150 epochs is needed for security reasons.
• Given PreCommit needs a collateral (PCD), there is no real possible trustless separation between the entity which stores the sector and the entity which actually proves it. This means that if anyone wants to outsource the task of proving a sector, then the entity which proves and the entity which outsources the task need to agree on who puts down the collateral (trusting the other part).

A first step to mitigate PoRep bottlenecks was the introduction of Synthetic PoRep (See FIP-0059). Nevertheless, if we really want to remove the aforementioned limitations, NI-PoRep is the way to go: indeed, Non Interactive PoRep would remove the PreCommit step (and PCD), without compromising security, allowing for a simplified pipeline which can reduce costs and enable a full separation between storage and proving. Note that this is particularly relevant for Sealing as a Service (SaaS) and SupraSeal software throughput.

Protocol Specification

NI-PoRep allows to remove on-chain interaction when onboarding sector by changing the way PoRep challenges are generated.
We allow SP to locally generate challenges instead of using on-chain randomness.
On one hand, this feature of the protocol allows for drastically simplify the onboarding pipeline. On the other hand, higher security guarantee than today are needed in order to preserve network security.
We consider 128 bits of security as a robust level of security long term. It is possible to lower down security guarantees, but if we want a long term solution, 128 bits of security is the way to go.
In practice, this translates in an higher number of PoRep challenges (2253 per SDR layer instead of the current 176 per SDR layer) compared with today (12.8x).

NI-PoRep protocol can be summarized as follows:

Graph Labelling and commitments (similar to the current PC1 and PC2 computation)

1. Using ReplicaID (which contains CommD), SP computes the labels for all layers and the replica R;
2. SP compute the column commitments CommC , CommRLast and finally computes CommR=Poseidon_2(CommC, CommRLast);

Storage Provider locally generates NI_ChallengesNumber challenges and vanilla proofs;

1. Each challenge is of the form ch_i = H(prefix, commR, tag, i);
2. SP computes responses for all the NI_ChallengesNumber challenges, which result in  NI_ChallengesNumber vanilla proofs;

Storage Provider publishes the new NI_ProveCommitSector proof

1. SP takes the NI_ChallengesNumber vanilla proofs and computes the corresponding SNARK proofs for these challenges.
2. SP publishes the snark and commitment CommR (either in individual or aggregated form).
   1. Note in this step the SP will, by default, use the SnarkPack aggregation technique to prove even only one sector (see “product considerations” section for the motivation).

Chain verifies proof

1. Using CommR as a seed, the chain generates NI_ChallengesNumber challenges and this are fed into proof verification

How to get there

There are two ways we can get to NI-PoRep

• Re-using existing circuits
  • Pros: fastest way to ship NI-PoRep (no trusted setup needed, design close to be finalized)
  • Cons: no optimization possible wrt NI-PoRep limitations (see dedicated section)
• Building new ad-hoc circuits
  • Pros: we could probably mitigate some of the NI-PoRep limitations by using novel approaches
  • Cons: trusted setup needed, more engineering work, thus more time to ship

In order to have NI-PoRep as soon as possible, we opt for re-using existing circuit, and we consider that scenario in the following sections.

Impact on Filecoin

There are multiple venues where NI-PoRep would be beneficial for Filecoin. We’ll elaborate on those below

Cost Reduction: removing complexity reduces costs and enables the full potential of SupraSeal improvements

Removing interaction translates into a simplified pipeline

• No PreCommit method and message, only ProveCommit will stay (only one method and one message needed to add sectors to the network);
• No PCD (PreCommit Deposit)
  • In general, no ad-hoc deposit will be required for the PoRep step (InitialPledge will remain);
• No waiting time between PreCommit and ProveCommit

Simplification of the pipeline and more flexible ProveCommit Step would allow for maximizing throughput

• High impact in terms of SupraSeal software utilization

Trustless SaaS Enabled

Removal of PCD allows for Proving and Storing to be now decoupled

• SaaS would be possible in a trustless manner (low risk-low trust: sealer does not need to put down collateral that will need to be re-paid by buyers or buyer does not need to pre-pay this amount)
• SaaS providers can delegate proving tasks. In particular, proving can be split into specialized subtasks which get outsourced to specialized entities (labeling the graph, Snarks, …)
• Enabling HDD wholesale: it would be possible to receive brand new drives with Sector-Keys pre-generated in your name

PoRep secured Cryptographically and not Rationally

Chain cryptographic security gets increased: NI-PoRep would make misbehaving cryptographically infeasible rather than irrational.

PoRep Security now independent from Consensus

Current PoRep is interactive and needs to get randomness form the chain. Moreover, in order to be secure, 150 epochs are needed between PreCommit and ProveCommit. This is due to the fact that some Consensus attacks need to be infeasible (as putting those attacks in place would allow for faking storage).
In NI-PoRep, since randomness is derived locally, there is no link anymore between PoRep and consensus attacks. This mean that

• Consensus attacks are not a concern anymore for PoRep security
• PoRep can now work “agnostically” with any consensus protocol

Backward compatibility

• NI-PoRep would be a separate proof type with a different on-chain flow as current PoRep. Anyone can decide whether use NI-PoRep or not.
• No need for a new trusted setup

Security Considerations

Fiat-Shamir Heuristic and its effects

NI-PoRep is based on the well-known Fiat-Shamir Heuristic, which allows for having a non interactive protocol starting from any 3-message interactive protocol. However, the heuristic requires the interactive protocol to have at least 80 bits of security (instead of the current 10), and preferably 128. In order to have long term security we propose to opt for 128 bits: this means that for NI-PoRep we need  NI_ChallengesNumber128bit = 2253.

Sectors need to be anchored to the chain: how to take SealRandomness?

Interactive PoRep

The epoch in which sealing took place is identified by the corresponding chain reference, called SealRandomness which is the anchor of the sector to the chain. SealRandomness must be taken from the VRF chain, in order to ensure that no other fork can replay the Seal.

On one hand, SealRandomness needs to be taken from finalized blocks, but on the other hand it can not be taken farther back than necessary (in order to protect against long-range attacks). This means that SealRandomness needs to be verified.

Due to current on-chain interaction, randomness verification is not too complicated and works as follows.

Given:

• F– Finality (number of epochs)
• X– epoch in which sealing starts
• Z– epoch in which the sealed sector appears (in a block)
• Y– epoch announced in commitSector (should be X, but an SP could use any Y <= X)
• T– estimated time for sealing, dependent on sector size
• G = T + variance– necessary flexibility to account for network delay and sealing time variance.

When starting to prepare a seal in epoch X, the SP should draw randomness from X-F with which to compute the seal.

When verifying a seal in round Z, a verifier should ensure that the ticket used to seal the sector is found in the range of epochs [Z-F-G, Z-T-F+G].

In practice, the Filecoin protocol includes a MaxSealDuration for each sector size and proof type.

As a result, each epoch determines the oldest seal challenge epoch that will be accepted in that epoch, defined as
sealChallengeEarliest = currEpoch - ChainFinality - MaxSealDuration

NI-PoRep

If interaction is removed, SealRandomness verification becomes more difficult, but still necessary to protect the chain against the same class of attacks mentioned above.

We'll set up an interval similar to what we have today, which holds for all the sectors committed onchain together.

This means that, if PC1, PC2, C1, C2 happen locally over time resulting into different sectors sealed in different moments in time and committed on-chain at the end of the process, all sectors committed together should have a randomness which verifies against the same time window, which is set to be 30 days, as it is today. See FIP-0013

This means that we have de facto a MaxNI-SealDuration , which could be potentially set as larger than today, in order to have NI-sealChallengeEarliest = currEpoch - ChainFinality - MaxNI-SealDuration to be wide enough to fit meaningful use cases, while still preserving security (research needed here in order to find the right value, considering both security and use cases that will be covered).

Note that we could also use the same values as today as an initial step, relaxing the randomness sampling rule for the single sector, while keeping the same sealChallengeEarliest. In this case we'd have a NI-PoRep step to be completed within sealChallengeEarliest epochs overall to be valid.

Potential concerns (and mitigations) due to interaction removal

One point to take into account when dealing with NI-PoRep, compared with the status quo, is that a malicious party willing to take over the network could potentially keep accumulating sectors locally onboarding them to the network all in a sudden.

Does such an attack have immediate effects?

Even if it is true that sectors can be accumulated over time and onboarded all in a sudden, we know that power is granted to onboarded sector with some delay. Specifically, there are 900 epochs of PowerTableLookback. This means that at epoch T we refer to the ctive power at time T- 900.
As a consequence, even with NI-PoRep we have a minimum of 7.5h delay between the moment when sectors are onboarded and the moment when they actually acquire power.

In case of Interactive PoRep we have additional 75min given by the needed window of time between PreCommit and ProveCommit

What is the real difference with respect to Interactive PoRep?

The answer really depends on the threat model we want to consider. In what follows we present four different scenarios with four different adversaries, from the less to the most powerful one.
Note that when we get security against the most powerful adversary, we automatically obtain security against less powerful adversaries as well. For this reason, after presenting the different scenarios, we'll focus on Scenario 4.

Scenario 1: Adversary limited in hardware and pledge

• Interactive PoRep: the attack is infeasible
• Non Interactive PoRep: the attack is infeasible

No security risks in this case.

Scenario 2: Adversary limited in hardware but not in pledge

We consider a window of time of 30 days (i.e. MaxProveCommitDuration)

• Interactive PoRep: the attack gets noticed over time. The adversary needs to pull off the attack (onboarding sectors) over time due to hardware constraints. This means PreCommit messages need to be put on chain over time. Note that an adversary of this kind can always put in place the attack using sibling, making detection much more difficult.
• Non Interactive PoRep: the attack does not get noticed over time despite hardware constraints. The adversary needs to pull off the attack (onboarding sectors) over time due to hardware constraints but, given there are no PreCommit messages, sectors can be added all in a sudden. Detection window is 7.5h before the adversary actually gets his power.

Due to consensus security, this scenario does not represent a security issue (see "On feasibility of adversary not limited by pledge" below for further details).

Scenario 3: Adversary limited in hardware but not in pledge
As we did in Scenario 2, We consider a window of time of 30 days (i.e. MaxProveCommitDuration)

• Interactive PoRep: the attack could happen all in a sudden given the adversary is not limited in hardware, but can not actually happen since pledge limitation prevents massive onboarding.
• Non Interactive PoRep: the attack could happen all in a sudden given the adversary is not limited in hardware, but can not actually happen since pledge limitation prevents massive onboarding.

Due to consensus security, this scenario does not represent a security issue (see "On feasibility of adversary not limited by pledge" below for further details).

Adversary not limited in hardware, nor in pledge

• Interactive PoRep: the attack can happen all in a sudden. Detection window is 8.45h (150 epochs between PreCommit and ProveCommit + 900 epochs for power table updates)60 min needed for windowpost + 900 epochs for power table updates).

Due to consensus security, this scenario does not represent a security issue (see "On feasibility of adversary not limited by pledge" below for further details).

On feasibility of adversary not limited by pledge
It is true that with NI-PoRep and the associated removal of PreCommit traceability of sectors that will be added to the network is not possible anymore when adversary limited in hardware but not in pledge. Nevertheless, we are convinced that accumulating sectors and suddenly post them onchain is actually a feasible way to attack the network.

Indeed, NI-PoRep removes PreCommit and PCD, but keeps IP as it is today. This means that adding EiBs of sectors results in a massive costs both in terms of hardware and pledge. Latest analysis from CryptoNet lab show that network today is mostly secured by pledge, and this is not going to change with NI-PoRep. See [2023Q3] Consensus Security: summary doc and related analysis.

On similar attack which an Adversary with unlimited datacap and not limited by pledge could put in place today

A similar attack to the one mentioned in this section is (in theory) already possible today for an adversary who is not limited in pledge and has datacap. This adversary can takeover the network by upgrading a massive amount of CC sectors via SnapDeals. In this case the detection window is the same as NI-PoRep sectors, meaning PowerTableLookback = 900 epochs ~ 7.30h

On current feasibility of the attack given storage onboarding rate

Even if an adversary willing to takeover the network exists, current storage onboarding rate suggests that this attack is not feasible in practice today.

Latest analysis on onboarding (see here) suggests that the Filecoin network can optimistically onboard only 7.4PiB/day deal data.
Moreover, at the moment aggregation is dysfunctional, and even if it worked, it could increase the data onboarding capability only by 11%.
The real onboarding capability is probably the ~6PiB/day we observe in the network. See here for details.

This means that the amount of storage which can be onboarded into the network is currently capped by 240 PiB/month which makes impossible to takeover the network without getting noticed (given current network size).

Note that this represent a bug and we assume it will be resolved. As a consequence, we can not consider current (temporary) limited onboarding rate as a security guarantee by any mean.
On the other hand, it is true that limiting onboarding rate would mitigate the issue, but this would represent a change in Filecoin which should be acknowledged and designed ad hoc.

For all the above, we are convinced NI-PoRep does not introduce additional risks for the network.

What if we are still concerned there is a security issue?

Assume that after all we detailed above, we are still concerned there can be a security issue and we want to protect Filecoin against it.
In this case we should protect the network against the Scenario 4 adversary (given it is the most powerful one and all other scenarios would be covered once Scenario 4 is covered).

We have two issues we want to analyze:

• Early detection
• Massive Onboarding

Early Detection
If the feature we want consists in having the power to detect massive onboarding, then we need to consider the following:

• The most powerful adversary is not limited in hardware, nor in pledge. Thus, as described in Scenario 4, can onboard massive amount of sectors all in a sudden.
• Interactive PoRep sectors have a detection window of ~ 8.45h (150 + 900 epochs)
• Non Interactive PoRep sectors have a detection window of ~ 7.30h (900 epochs)

So, we need to understand how wide a detection window should be.

• If a 7.30h detection window is acceptable, then we are safe with no need of changes
• If a 8.45h detection window is acceptable, but a 7.30h window is too short, then we need to delay power acquisition for NI-PoRep sectors (and Snapped CC sectors) by 150 epochs
• If a 7.30h window is too short, then we need to delay power acquisition for Interactive and NI-PoRep sectors (and Snapped CC sectors) by the number of epochs needed in order to reach the desired duration

Massive Onboarding
Note that the adversary we are considering has unlimited hardware and pledge, thus can add massive number of sectors. In this case, what we should do is to limit the daily total onboarding by design.

Gas and Proving Overhead Implications

Gas Costs Analysis

PoRep challenges are generated outside Snark Circuits and passed as inputs at verification time. These inputs need to be generated on the spot in order for verification to happen.

With Non Interactive PoRep, with 128 bits of security the number of challenges is multiplied by either 12.8x.

We analyze the concrete impact of NI-PoRep proving overhead on gas costs for SPs. We figured out that Removing PreCommit makes NI-PoRep and Interactive PoRep gas costs comparable.

In particular,
Single Sectors (i.e. using SnarkPack for NI-PoRep only, considering a single sector [worst case for NI-PoRep])

• Considering proof verification only, NI-PoRep with 128 bits of security when aggregating 6 sectors is 15M gas units more expensive per sector (~1.9x current single proof verification costs)
• Considering total gas costs, given that NI-PoRep removes PreCommit (and associated gas costs), the difference between NI-PoRep with 128 bits of security and current PoRep goes down to < 36M gas units (123M Vs (34+53)M gas units)

Aggregated Sectors (i.e. using SnarkPack both for Interactive and NI-PoRep)

• Considering proof verification only, NI-PoRep when aggregating 6 sectors is 9M gas units more expensive per sector (~1.5x current single proof verification costs).
• Considering total gas costs, given that NI-PoRep removes PreCommit (and associated gas costs), Current sectors are 2.1x more expensive than NI-PoRep sectors with 128 bits of security (53M + 17M gas units for current sectors, 32M gas units for NI-PoRep sectors).

Note also that estimated gas cost for PreCommit ~53M gas is optimistic (i.e. it is usually higher). This means that NI-PoRep is probably performing even better than what this analysis shows, compared with current PoRep.

See full analysis here

Proving Overhead Cost Analysis

NI-PoRep allows for removing interaction at the cost of augmenting the number of challenges by a factor of 12.8, considering 128 bits of security.

The 12.8x number of challenges translates into an 12,8x Snark proving overhead. We analyzed how this snark computation overhead affects overall costs.

The conclusion is that considering PC1+PC2+C1+C2 and Storage costs (i.e. not considering maintenance costs), a 128 bits of security NI-PoRep sector is 5% more expensive overall than a Interactive PoRep sector when sector duration is 3y.

See full analysis here

[Fatman13]

No PreCommit method and message, only ProveCommit will stay (only one method and one message needed to add sectors to the network);

How would this impact the hardware spec for sealing boxes? Would current hardware setups still be relevant?

In general, no ad-hoc deposit will be required for the PoRep step (InitialPledge will remain);

Do you mean the consensus pledge of a sector will remain? AFAIU, initial pledge refers to the 20 day expected reward of a sector.

[lucaniz]

How would this impact the hardware spec for sealing boxes? Would current hardware setups still be relevant?

All the analysis is done assuming SupraSeal.
As linked in the discussion, you can find the reference configuration we considered here.

Do you mean the consensus pledge of a sector will remain? AFAIU, initial pledge refers to the 20 day expected reward of a sector.

what we are removing is PCD, which in Interactive PoRep (the current one) is needed for security reasons. All the rest stays the same.

[Fatman13]

Is NI-PoRep only for CCs rn?

[irenegia]

Is NI-PoRep only for CCs rn?

Correct

[vvkio]

Finally! I appreciate the introduction of this FIP, I've eagerly anticipated it for years!

In essence, this FIP will enable a complete separation between sealing and storage tasks. Currently, despite 'Sealing as a Service', there remains a dependency on the storage provider: the provider needs to cover the PC1 deposit and supply all sector information.

Looking at the bigger picture, sealing and storage represent two distinct operations that, ideally, shouldn't have been merged and executed by a single entity. However, due to practical considerations at the network's launch, it was impractical to segregate them.

This FIP fixes that.

In conclusion, the execution of this FIP facilitates the following advancements:

• For the first time, sealing can be executed with no dependency on the storage provider.
• It allows sealing to be conducted ahead of schedule in regions with lower energy costs.
• It paves the way for planetary or interplanetary sectors transfer, even to the moon 🚀🌕!.
• By minimizing sealing overhead, it speeds up the path to profitability and facilitates paid deals.

[suykerbuyk]

FIPS 854 is critical to further dis-aggregating the lotus stack to streamline pipeline data ingress for the Filecoin network. The further we can decompose the individual steps and states of the data pipeline, the better we can optimize the orchestration and implementation of those transformations.

[anorth]

There is a draft FIP PR at #891.

The new onboarding method proposed there follows the FIP-0076 ProveCommitSectors2 method in supplying sector activation manifests etc. This means it combines sector capacity activation with data commitments, notifications to markets etc.

The idea of cleanly separating these two actions has come up independently from a number of angles (e.g. it's covered in these discussions 1, 2 from FilDev Singapore, and was similarly affirmed at FilDev Iceland). The introduction of NI-PoRep seems like the right opportunity to introduce this.

For SAAS flows especially, it's apparent that shipping all the data over to the sealer so they can seal, then ship it all back is a very significant burden. It's much easier to seal and distribute CC sectors and then the SP can snap the data later. So we don't expect many users of this new method to ever want to supply data at the same time anyway.

If I roughly recall, some arguments in favour of supporting only onboarding CC sectors and snapping data later:

• Orchestrating the data flow for pre-sealed data sectors is complex, limiting (video 1 above)
• All sectors get a Sector Key and so can support re-snap of the data later
• The sector Key similarly supports proof-of-access to unsealed copy protocols (video 2 above)
• Data sectors add significant code complexity in the miner actor to support two different ways of committing to data
• This is the flow that SAAS will use anyway, which we expect to become increasingly cost-efficient and thus dominant
• If we support only CC+snap flow, there will be no gas differential between different flows, exploiting which adds operational complexity

The NI-PoRep onboarding method could be much simpler if it only supports CC sectors. And this is the target use case anyway. If widely adopted, we may be able to deprecate older non-interactive onboarding to realise a great simplification of both on-chain and storage pipeline code (which reduces risk and improves development velocity).

This flow depends on efficient Snap Deals, but I believe that will be addressed by FIP-0082 - Aggregate replica proofs.

cc @nicola @momack2 @jennijuju @ribasushi @snadrus

[lucaniz]

I see your point, you are proposing to basically have a unique sector type/container enabled by NI-PoRep and that snapdeals happen locally.
Intuitively i'd prefer to have the double functionality as it is always better to have a restrict usage of a wider allowed functionality rather than limit it by design.

That said, I understand that enabling the double functionality (CC and sector with deals) can be not worth the effort.

The real question to me is whether the "local" snap is something which is efficient enough for SP to perform while outsourcing the "CC part" only.
I think we should further discuss both of the aspects.

[jennijuju]

The new onboarding method proposed there follows the FIP-0076 ProveCommitSectors2 method in supplying sector activation manifests etc.

Resolved It is worth noting that ProveCommitSector3 introduced by FIP-0076 rejects sector info that includes storage market f05 deal IDs, so any sector info/activation mainefest for that method doesn't include deal IDs . We should discuss if the similar behaviour is desirable for NI Porep work flows - in other porep flows, those who would like to leverage f05 still have alternative pathways via methods like ProveCommitAggregate and ProveReplicaUpdates2, however, we dont have any in the case of NI porep. So the question is: should the protocol limit NI porep pathway to be coupled with DDO pipeline, if so, why?

[anorth]

I don't quite understand the limitation that you are pointing to. An SP will always be able to do a replica update after NI PoRep (and in the thread above I propose that this is the only way to put data in those sectors). f05 deals can be used with replica update.

[jennijuju]

in the thread above I propose that this is the only way to put data in those sectors

my q here is decoupled from your proposal above and is based on the world where SPs can directly onboard sectors with data.

[jennijuju]

In which with existing PoRep, SPs can onboard sectors with f05 deals and their data directly using ProveCommitSectors (until its dropped, if it ever gets dropped) or ProveCommitAggregate, however, it doesn't seem to be the case with NI PoRep.

[anorth]

An SP can still activate a sector with f05 deals using the FIP-0076 methods by providing the deal ID in the activation manifest. The FIP draft copied that method signature, so allows the same specification. I'm assuming the same behaviour is supported, since otherwise it would have to explain why it used the same parameters but ignored some of them.

It is worth noting that ProveCommitSector3 introduced by FIP-0076 rejects sector info that includes storage market f05 deal IDs, so any sector info/activation mainefest for that method doesn't include deal IDs

I'm not that sure I'm interpreting this correctly, but I don't think it's correct.

[jennijuju]

Chatted in sync and I was mistaken - the PSC3 only rejects any sectors with deal id in the pre commited sector info, however will accept activation manifest with deal id just fine like anorth mentioned. And with NI PoRep it fits the later and won’t have any precommit info, it will just work fine

[frommww]

Here is a statement from the Decentralized Storage Alliance (DSA) in support of this FIP (FIP-854).

Statement on Improving Data Storage on the Filecoin Network via Non-Interactive Proof of Replication (NI-PoRep)

The members of Decentralized Storage Alliance (DSA) recognize the importance of simplifying and streamlining the input of data into the Filecoin Network and hereby support the introduction of Non-Interactive Proof-of-Replication (NI-PoRep) as part of the Filecoin Protocol.

Specifically, the members support FIP-854 as this improvement proposal outlines a method of arriving at Non-Interactive Proof of Replication via the use of existing network circuits, thereby reducing the engineering effort and network changes needed to implement this capability.

It is our understanding and desire that this change to the Proof of Replication process will serve to increase security by increasing the use of cryptographic measures in the sealing process, reduce the cost of sealing operations, separate Proof of Replication security from consensus thereby supporting greater trustless interaction, and enable novel network capabilities including but not limited to services such as Sealing as a Service.

Furthermore, the DSA recommends the maximum time-window between the sector initiation process (SealRandomnessEpoch stage) and the posting of storage commit and sector activation (ProveCommit stage) be of sufficient duration to accrue the benefits of prepping and pre-sealing storage sectors independent and distinct from sealing operations while still supporting the underpinning of the PoRep theorem that the timestamps between sector anchors shall be of a reasonable duration.

The DSA hereby supports a time-window of no less than 30 days as specified in the current proposal but strongly endorses extending this window to a maximum of 180 days to allow for storage providers to provision and ship pre-configured drives for maximum efficiency. The extension of the window is under the provision that no material concerns are uncovered during development and with proper review that compromise the mathematical proofs and security guarantees contained in the existing proposal.

For these reasons and as outlined in further detail below, the DSA supports the advancement of FIP-854 into development with sufficient prioritization as to see its success and timely release within the Filecoin Network so as to greatly simplify the sealing process and accelerate the adoption of decentralized storage for mission-critical and commercial-grade storage loads.

Full statement can be found here:
DSA Statement - Non Interactive Proof of Replication (Ni-PoRep).pdf

[rjan90]

Hey 👋

We’re currently exploring/writing up a project timeline for the reamining pieces of NI-PoRep work. And we want to gather feedback if you or your team foresee benefits from having NI-PoRep included in the network version 23 upgrade that is planned to happen on June 20th - 2024?

Your support and opinions are crucial in advocating for the inclusion of NI-PoPep in a given network upgrade.

[mtrisic]

Hey @rjan90 , I'm reaching out to express support for including NI-PoRep in the upcoming network version 23 upgrade. Eager to see this FIP launched, we view it as a valuable enhancement to the Filecoin protocol, streamlining the sealing and storage process. It simplifies offloading sealing tasks to compute markets and reduces complexity for storage providers.

Introducing NI-PoRep promises significant improvements for 'sealing as a service' pipelines, lowering sealing costs, fostering network growth, and positively impacting Filecoin's adoption and economy 🚀 🚀 🚀

[luckyparadise]

I just wanted to include here additional spec details for this FIP that is currently opened as a PR in the FIPs repo. HERE. This new PR adds fields for specifying sealer actor id and sealer actor specified sector number to the NI porep method. This still forms part of the FIP draft once approved and merged by FIP Editors.

[rvagg]

There's a discrepancy that I think needs to be discussed and resolved in some way, ideally with clarification in 0090, arising out of that last pull request @luckyparadise mentioned above, #1000.

• SealingNumber and SealerID aren't mentioned in the FIP text, the only guidance is in the comments but I don't think it's enough
• The initial implementation in builtin-actors @ Non Interactive Proof of Replication builtin-actors#1537 includes both of these new properties, but SealerID is nullable / optional. But it doesn't have any handling of that parameter in the code, I suspect it's not clear what should be done with it.
• The FIP proposing to make use of SealerID @ FIPXXXX: Introducing sealerID #993 has language that suggests it should be optional because it has handling for the case where it's "empty".

It would be good if we could get clarity into 0090 on:

1. Whether one or both of these parameters can be null, ideally such that the follow-up FIP doesn't have to alter the parameters at all.
2. The precise validation behaviour of both of these parameters for implementation of 0090.

I'd be happy to do a PR to 0090 with some language but it's unclear from the various activities outlined above what the expected behaviour and format of parameters should be.

@ZenGround0 perhaps this is one for you?

[lucaniz]

It would be good if we could get clarity into 0090 on:

1. Whether one or both of these parameters can be null, ideally such that the follow-up FIP doesn't have to alter the parameters at all.

Yes, the idea is that until SealerID FIP is not introduced, then those values can not be "changed". This was our "meaning" of the term "empty". I guess whta you propose (set them to null) is exactly what we were thinking. @ZenGround0 can you confirm?

2. The precise validation behaviour of both of these parameters for implementation of 0090.

As 0090 does not introduce any "sealer" different from the SP, then those two values should be set to null so that we can "only
" have MInerID and SectorNumber is the one of the SP.
Again, @ZenGround0 please confirm/explain it better than I did if needed.

cc @irenegia

[anorth]

Requiring the values to be specified and equal to the miner's id / sector number would make the actor code marginally simpler (especially after sealer IDs are used and the must-be-equal check is removed)

[ZenGround0]

I thought the comments I added to the fields in PR (https://github.com/filecoin-project/FIPs/pull/1000/files#diff-4a7ea163a5dbc3e7b6b38faab35595205e5865377050222e42fbc9679b6556d8R102-R103) made it clear these should not be nullable but set explicitly. And for simplicity in fip 0090 it must be the case that SealerID == ActorID and SealingNumber == SectorNumber. But it looks like I wasn't direct enough.

[rvagg]

Given that the size of the NI-PoRep proofs is quite large, I'd like to raise the possibility of removing the ability to submit batch proofs and always require aggregates, in all cases, to simplify this FIP even more.

A single 32 or 64G sector proof submitted via the new method is going to be 23,124 bytes. If you batched 90 of these then you'd push past a 2MiB message (I don't know what hard limit on message sizes we have, probably dictated by bitswap, but it'll be somewhere under 2MiB I believe). I'm unsure what the aggregate size will be and how it'll multiply as you add more to your aggregate (this would be good information to have in the FIP actually), but I assume it'll scale a lot slower as you add more sectors in.

So, could we get rid of batch, simply the messages, simplify the actors implementation, simplify the miner software implementation, simplify the gas modelling that users have to apply, and simplify all our testing requirements?

  struct ProveCommitSectorsNIParams {
     // Information about sealing of each sector.
     Sectors: []SectorNIActivationInfo
     // Proof type for each seal (must be an NI-PoRep variant)
     SealProofType: RegisteredSealProof,
     // Proofs for each sector, parallel to activation manifests.
     // Exactly one of sector_proofs or aggregate_proof must be non-empty.
     SectorProofs: [][]byte,
     // Aggregate proof for all sectors.
     // Exactly one of sector_proofs or aggregate_proof must be non-empty.
     AggregateProof: []byte,
     // Proof type for aggregation, if aggregated
     AggregateProofType: Option<RegisteredAggregateProof>
     // Whether to abort if any sector activation fails.
     RequireActivationSuccess: bool,
   }

becomes

  struct ProveCommitSectorsNIParams {
     // Information about sealing of each sector.
     Sectors: []SectorNIActivationInfo
     // Aggregate proof for all sectors.
     AggregateProof: []byte,
     // Proof type for aggregation
     AggregateProofType: RegisteredAggregateProof
     // Whether to abort if any sector activation fails.
     RequireActivationSuccess: bool,
   }

[jennijuju]

Current NI-PoRep spec is already sealerID ready

[anorth]

#993 is not close to ready, and is conceptually quite distinct.

[irenegia]

#993 is not close to ready,

design research phase is complete there, some specs details likely missing in the current version, but I am confident we can add those!
Do you see something else missing?

and is conceptually quite distinct.

I don't agree on this. The concept of sealerID is what is currently missing in ni-porep to get the complete separation between the sealing/onboarding operations and storage (and create different and independent roles in the network). With ni-porep only, some interaction between the sealer and the SP is needed, with niporep+sealerid we remove it

[lucaniz]

To give a concrete example, pre-sealed sector market would need SealerID to be enabled

[anorth]

I understand the proof-related parts are complete. What's missing is all the specification for the new built-in act actor, the methods it supports, its state schema, and the preconditions and algorithm for its methods, etc. As with many changes to actors, we should implement it before we have too much confidence we understand it completely enough to call a spec final.

[ipfsforcezuofu]

Removing the batch option would be beneficial for process optimization.

[rvagg]

Another discussion topic for possible amendment or inclusion in 0092: FIP-0090 currently specifies sealChallengeEarliest as 30 days; so the earliest that chain randomness can be sampled for the seal is 30 days prior to onboarding. The current buitin-actors WIP implementation sets it to 365 days plus finality.

During a review, @irenegia and @lucaniz suggested that 180 days might be a more appropriate compromise to support the SaaS case, but still caters to the security concerns related to replays already outlined in 0090.

[jennijuju]

I am glad to see that FIP-0092 is proposing changes to simplify the protocol even more & ease SP operations. That said, I am a little 😞 that batch balancer is still being introduced to this new onboarding method, and seems like the reason it is added is b/c "we have been doing it elsewhere, so why not here too" instead of a clear benefit to the network or the SPs - if there is one, I would love to learn.

I am also a bit skeptical of the parameters chosen as it is based on data from FIP0024, which I think may be outdated & needs a re-evaluation for post-fvm + current network condition.