Skip to content

Latest commit

 

History

History
396 lines (304 loc) · 24.9 KB

frc-0068.md

File metadata and controls

396 lines (304 loc) · 24.9 KB
Raw
fip title author discussions-to status type created
0068
Deal making between SPs and FVM contracts
@aashidham, @raulk, @skottie86, @jennijuju, @nonsense, @shrenujbansal
Draft
FRC
2023-04-25

FRC-0068: Deal making between SPs and FVM contracts

Simple Summary

Filecoin deal-making flows previously required manual off-chain interactions between a Filecoin Storage Provider and a deal client. This FRC proposes a robust standard for an FEVM smart contract to act abstractly as a deal client, thus enabling use cases for automated deal-making.

Abstract

A set of solidity interface standards are introduced. These interfaces support both proposing programmatic deals to a Filecoin storage provider and verifying deals as a client. Instead of the client proposing off-chain deal parameters that are accepted by the storage provider and must then be verified by the client, a contract can implement either the WildcardDealProposer or OneToOneDealProposer interface (which both inherit from BaseDealProposer). This results in the emission of a *DealProposalCreate event.

These events are visible to storage providers, who can then call getDealProposal on the event emitter and get details to begin proving the deal. Instead of requiring the off-chain client signature as input to the PostStorageDeal flow, it will now utilize the BaseDealProposer interface and execute the handle_filecoin_method method to verify the client’s deal.

Motivation

Providing a smart contract interface for deal clients in the Filecoin network brings several benefits to the FVM ecosystem:

  1. Enhanced automation: Smart contracts enable the automation of deal-making, dispute resolution, and payments. This could streamline the deal-making process for clients and storage providers, making it more efficient and cost-effective.
  2. Agreement customization: Smart contracts allow for the creation of customizable agreements between clients and storage providers. This means that the terms and conditions of a deal could be tailored to the specific requirements of both parties, creating more flexible and adaptable solutions. This unlocks DAO treasury mechanics, perpetual storage, and replica processes.
  3. Improved trust: The use of smart contracts can increase trust between clients and storage providers since the execution of the contract is controlled by code rather than relying on manual intervention. This reduces the possibility of manipulation or misinterpretation of the agreement.
  4. Composability: A standard interface for programmatic deal making (proposals, verification, payments) allows both clients, storage providers, and developers to build together efficiency.

Specification

A valid client contract implementation would minimally support either the OneToOneDealProposer or WildcardDealProposer interface. Both of these inherit the BaseDealProposer interface.

In particular, the OneToOneDealProposer interface is for client contracts that have specified a single SP for every deal proposal. Developers implementing this interface will likely have to reach out and coordinate with SPs specified for a particular deal. SPs with address x can then listen to OneToOneDealProposalCreate events with the provider field set to x. This flow is simple since it only deals with one client contract and one SP per deal proposal.

On the other hand, the WildcardDealProposer interface is for client contracts that have not specified any SPs for a deal proposal. SPs can subscribe to a particular client contract, and will be selected by the contract in order to receive a certain deal proposal. If a PSD message is not released by the elected SP for the deal, the contract may elect another subscribed SP to take the deal proposal on. This process is more universal and does not require offchain coordination between an SP and a client contract.

Note: we may define other modalities that also inherit from BaseDealProposer that are not currently represented by OneToOneDealProposer or WildcardDealProposer. This is meant to be a modular standard that can be expanded upon later, with different approaches for SPs to interact with client contracts.

interface BaseDealProposer {
    /**
     * ClientContractDeployed
     *
     * This event is emitted by the client contract when the contract
     * is successfully deployed. Deployed once per lifetime of the contract.
     *
     */
    event ClientContractDeployed();

    /**
     * handle_filecoin_method
     *
     * This is the entry point for all built-in actor calls. This
     * must be properly dispatched to provide a client signature verification
     * of a proven deal from a storage provider, notify, or receive data cap.
     *
     * @param method the method ID the built-in actor is attempting to call
     * @param _unused ignored paramter
     * @param params the calldata parameters associated with the method.
     * @return exit code [as defined here](https://github.com/filecoin-project/ref-fvm/blob/c7b80f021edfe2147ca0a3868408c5fd1a138432/shared/src/error/mod.rs#L90)
     * @return codec
     * @return memory the data itself
     */
		function handle_filecoin_method(
				uint64 method,
				uint64 _unused,
				bytes memory params)
		external returns (uint32 exit, uint64 codec, bytes memory);
}

interface OneToOneDealProposer is BaseDealProposer {

    /**
     * getDealProposal
     *
     * Given a unique identifier for an available deal proposal, return
     * the deal definition payload (with the provider field defined)
     *
     * @param id the unique identifer for this deal proposal
     * @return the deal definition for that deal proposal
     */
    function getDealProposal(bytes32 id) external view returns (DealDefinition memory);

    /**
     * OneToOneDealProposalCreate
     *
     * This event is emitted when a new deal is available to
     * storage providers from this client. Client needs to specify provider.
     *
     * @param id the miner ID of the provider (required)
     * @param id the unique identifer for this deal proposal (required)
     * @param size the data size of the deal itself (optional, can be left as 0 if not available)
     * @param verified true if the deal is verified, false otherwise (optional, can be left as false if unknown)
     * @param price the nanoFIL the client is offering for the deal per GiB per epoch of storage (optional, can be left as 0 if no price offered)
     */
    event OneToOneDealProposalCreate(
        bytes32 indexed provider, 
        bytes32 indexed id,
        uint64 size,
        bool indexed verified,
        uint256 price);

}

interface WildcardDealProposer is BaseDealProposer {

    /**
     * getDealProposal
     *
     * Given a unique identifier for an available deal proposal, return
     * the deal definition payload (without the provider field)
     *
     * @param id the unique identifer for this deal proposal
     * @return the deal definition for that deal proposal
     */
    function getDealProposal(bytes32 id) external view returns (DealDefinitionNoProvider memory);

    /**
     * WildcardDealProposalCreate
     *
     * This event is emitted when a new deal is available to
     * storage providers from this client. Client does not need to specify provider.
     *
     * @param id the unique identifer for this deal proposal (required)
     * @param size the data size of the deal itself (optional, can be left as 0 if not available)
     * @param verified true if the deal is verified, false otherwise (optional, can be left as false if unknown)
     * @param price the nanoFIL the client is offering for the deal per GiB per epoch of storage (optional, can be left as 0 if no price offered)
     */
    event WildcardDealProposalCreate(
        bytes32 indexed id,
        uint64 size,
        bool indexed verified,
        uint256 price);

    /**
     * getSPOrdering
     *
     * Given a unique identifier for an available deal proposal, return
     * the ordering of SPs that are subscribed to receive this deal proposal
     * from the contract.
     *
     * @return an array of addresses, where the first entry has the highest rank
     */
    function getSPOrdering(bytes32 id) external view returns (address[] memory);

    /**
     * subscribe
     *
     * Subscribe the caller to all subsequent deals proposed by the contract.
     * This means the caller is electing to add themselves
		 * as a candidate to all future deals emmitted by the contract.
		 * Meant to be called by SPs, not simple ethereum accounts.
		 *
     */
    function subscribe() external;

    /**
     * unsubscribe
     *
     * Unsubscribe the caller to all subsequent deals proposed by the contract.
		 * Meant to be called by SPs, not simple ethereum accounts.
		 *
     */
    function unsubscribe() external;

}

*DealProposalCreate (including WildcardDealProposalCreate and OneToOneDealProposalCreate)

  • MUST emit a *DealProposalCreate event when a new deal definition is available for SPs to analyze and propose storage deals for.

This contract client:

  • SHOULD not emit a *DealProposalCreate event with the same proposal ID more than once.
  • MUST provide a proposal ID that is unique to all deals that are proposed for that client.

ClientContractDeployed

This contract client:

  • MUST emit a ClientContractDeployed event exactly once — when the contract is first minted on the blockchain

getDealProposal

(By DealDefinition* we mean DealDefinition or DealDefinitionNoProvider depending on which interface is inherited.)

This contract client:

  • MUST provide a DealDefinition* struct response for a valid caller and proposal ID
  • SHOULD return an empty DealDefinition* struct for an invalid proposal ID
    • The proposal ID may never have been created
  • SHOULD return an empty DealDefinition* struct for an invalid caller
    • The caller could be invalid because it is not at the head of the SP queue (for a WildcardDealProposer client contract)
    • The caller could be invalid because it never subscribed to the contract (for a WildcardDealProposer client contract)
    • The caller could be invalid because it was not specified in the OneToOneDealProposalCreate event signature

getSPOrdering

This contract client:

  • MUST return an ordered list of all subscribed SPs for every valid proposal ID. This ordering must not change depending on the caller.
  • MUST update the ordered list of all subscribed SPs when the top of the queue for a proposal ID does not deliver a PublishStorageDeals message within 150 epochs of the emission of the WildcardDealProposalCreate event
    • In this situation, the chosen SP for the deal proposal has failed, and the queue must be updated to the next available subscribed SP for that deal proposal
  • MUST return an empty list if the proposal ID does not exist or is invalid.

handle_filecoin_method

This contract client:

  • MUST implement a proper routing for method 2643134072, which is the method number for authenticating a PostStorageDeal in place of using an offline client signature.
    • MUST honor the responses from getDealRegistration as it relates to the storage provider posting the storage deal, and the current epoch.
  • MUST implement a proper routing for method 4186741094, which is the method number of notifying the built-in market of the deal. This will ensure that the client understands that the deal has been successfully registered on-chain.
  • MAY implement a proper routing for method 3726118371, which is the callback for a contract receiving a data-cap allocation. The contract client does not have to offer or support data-cap allocation or FIL+ deals.

The DealDefinition* structs are defined as:

struct DealDefinitionNoProvider {
    bytes piece_cid;
    uint64 piece_size;
    bool verified_deal;
    string label;
    int64 start_epoch;
    int64 end_epoch;
    uint256 storage_price_per_epoch;
    uint256 provider_collateral;
    uint256 client_collateral;
    uint64 extra_params_version;
    ExtraParamsV1 extra_params;
}

struct DealDefinition {
    bytes piece_cid;
    bytes provider;
    uint64 piece_size;
    bool verified_deal;
    string label;
    int64 start_epoch;
    int64 end_epoch;
    uint256 storage_price_per_epoch;
    uint256 provider_collateral;
    uint256 client_collateral;
    uint64 extra_params_version;
    ExtraParamsV1 extra_params;
}

struct ExtraParamsV1 {
    string location_ref;
    uint64 car_size;
    bool skip_ipni_announce;
    bool remove_unsealed_copy;
}

The fields of DealDefinition* structs are the following:

Name of field Definition Example Notes
piece_cid Hash that represents the root node of the IPLD DAG (further described https://spec.filecoin.io/systems/filecoin_files/piece/) “baga6ea4seaqesm5ghdwocotmdavlrrzssfl33xho6xtrr5grwyi5gj3vtairaoq”
piece_size Size in bytes of the IPLD DAG (further described https://spec.filecoin.io/systems/filecoin_files/piece/) 262144
verified_deal Whether or not the deal is verified. false
label Human readable string that describes the deal. “baga6ea4seaqesm5ghdwocotmdavlrrzssfl33xho6xtrr5grwyi5gj3vtairaoq” Usually labelled with the Payload CID of the data.
start_epoch When the deal should start, described as a Filecoin epoch / block height. 2777713
end_epoch When the deal should end, described as a Filecoin epoch / block height. 3929712
storage_price_per_epoch The amount of attoFIL per GiB per epoch offered for this deal 0 Set to 0 for a unpaid deal
provider_collateral The amount of collateral set on the provider Must be within bounds (network parameters) — current bounds are returned by StateDealProviderCollateralBounds (min and max collateral a storage provider can issue). API takes the deal size and verified status as parameters.
client_collateral The amount of collateral set on the client 0
extra_params_version Version of the extra params payload used 1 Set to 1
location_ref Location (as an http hyperlink) of the car file. https://example.com/file.car
car_size Size in bytes of the car file. (further described https://spec.filecoin.io/systems/filecoin_files/piece/) 236445
skip_ipni_announce Boolean representing whether or not the deal should be announced to IPNI indexers false Set to false
remove_unsealed_copy Boolean representing whether or not the provider should remove the unsealed copy of the data. If not removed, the client has the ability to immediately retrieve its data. If removed, the provider first has to unseal the data before offering it for retrieval. false Set to false

Deal Flow I (Implementing WildcardDealProposer)

Wildcard2

The flow for this interface would be as follows, between client contract A and SP B. It requires the implementation of WildcardDealProposer , which itself inherits from BaseDealProposer.

  1. A gets deployed onchain and emits a ClientContractDeployed event.
    1. B sees the ClientContractDeployed event and calls subscribe() on A.
    2. B is now on the SP queue for all future deal proposals emitted by A.
  2. A emits a WildcardDealProposalCreate message for a deal proposal, and orders all subscribed SPs (including B) in a queue.
    1. The algorithm driving this ordering is entirely up to A. It can be random, deterministic, based on SP power, or something else entirely.
  3. B sees WildcardDealProposalCreate and calls getSPOrdering to determine B’s rank in the queue.
    1. If B is top of the queue, B calls getDealProposal to receive the deal proposal payload, as specified by the DealDefinition* struct.
    2. If B is not at the top of the queue, it takes no action.
  4. If B is top of the queue, B uses the deal definition to complete the deal, calling PostStorageDeal.
    1. This will call handle_filecoin_method for 2643134072, which should accept the deal as long as the provider is registered against it and the epoch is before or equal to the expiry epoch.
  5. If the storage deal aligning with the deal proposal is not published within 150 epochs, steps 3 and 4 are repeated with another SP that is next in the queue.
  6. The deal is posted to the chain.

Deal Flow II (Implementing OneToOneDealProposer)

OneToOne

The flow for this interface would be as follows, between client contract A and an SP B. It requires the implementation of OneToOneDealProposer , which itself inherits from BaseDealProposer.

NOTE: This flow will benefit from prior coordination off-chain. It would be beneficial for A to communicate with B off-chain ahead of time, so that B is aware that deal proposals that A will send to B. Unlike the WildcardDealProposer approach, A must determine (ahead of time) a single SP provider for each deal proposal that it is sent to the event log ahead of time. This will populate the provider field inside the OneToOneDealProposalCreate event.

The flow proceeds as follows:

  1. A gets deployed onchain and emits a ClientContractDeployed event.
  2. A emits a OneToOneDealProposalCreate message for a deal proposal, which specifies a single SP B inside the event signature.
  3. B calls getDealProposal to receive the deal proposal payload, as specified by the DealDefinition* struct.
  4. B uses the deal definition to complete the deal, calling PostStorageDeal.
    1. This will call handle_filecoin_method for 2643134072, which should accept the deal as long as the provider is registered against it and the epoch is before or equal to the expiry epoch.
  5. The deal is posted to the chain.

Design Rationale

The following design decisions have been made:

  • No Formal Deal Proposal Flow: The standard defines a DealDefinition* set of structs which standardizes the format for how storage providers receive deal parameters from contract clients, but does not define a contract method for generating this proposal. This enables the contract client to define, build, register, or otherwise accept deals in any manner they so choose. This flexibility ensures that we do not exclude use cases. In other words, this interface is scoped entirely to the interaction between the storage provider and the client contract.
  • Event-based Notifications: Using the event logs from transactions is a decentralized and trustless way of notifying network participants that deals are ready and available. Because every storage provider runs a node, event processing is a natural extension of their operations. The motivation is to enable SPs to listen and consume these events easily.
  • Account Abstraction Authorization: Instead of requiring offline signatures and trusted interactions between the storage provider and the client, the handle_filecoin_method integration for authenticating PostStorageDeal methods is a critical aspect of being able to programmatically accept deals.
  • Bandwidth and Cost Considerations: We want to ensure that storage providers do not have to needlessly try to process deals that are either already in progress. Some form of ordering, preference, and deal availability is needed to prevent wasted effort, front-running, or race conditions.
  • Permissionless: The interface does not account for regulations, client concerns, or client / storage provider relationships. Things like allow listing, deny listing, or storage provider selection are outside the scope of this standard. However, this standard provides the interface to encapsulate and enable permissioning.
  • Easy Entrance to Marketplace: We want to reduce the surface area and requirements to connect a client and a storage provider to the minimum necessary. This means removing any sort of previous relationship requirements for a storage provider to offer completing a deal.
  • Less Polling: Providing an expiration epoch for registration clearly communicates at a protocol level when a client expects a deal to complete, and when competing storage providers can come back to register should they have missed the previous opportunity.

Backwards Compatibility

Not relevant for this FRC.

Test Cases

For WildcardDealProposer:

  • For an SP A that subscribed to the first deal proposal, and then unsubscribed:
    • Was A part of the initial deal proposal SP queue in the client contract?
    • Was A part of subsequent deal proposal SP queues in the client contract?
  • For an SP that failed at submitting a PublishStorageDeals method for a deal proposal:
    • Did the client contract update the SP ordering after 150 epochs?
    • Was the next SP in the queue able to attempt to make the deal?

For OneToOneDealProposer:

  • What if a client contract A chose an SP B that was not listening to A’s events?
    • Would there be a time out after a certain number of epochs?
  • What if a client contract A chose an SP B, but the PSD message for the deal was submitted by SP C?
    • How could we ensure that C doesn’t get access to deal proposals it does not have permissions to access?

TODO: more engineering test cases

Security Considerations

There are a couple of security considerations that have been considered by this standard, and must be considered by both client implementations and storage provider configurations to remain robust.

For the ContractClient implementations:

  • Collateral, Datacap, and proposing deals needs to be secured against unauthorized actors from managing these assets. Additional interfaces, methods, or controls would be needed for on-going deal making and maintenance.
  • Accepting deal registrations should be done in a way that prevents denial of service attacks on the client. For instance, a client needs to sufficiently trust storage providers to complete the deal once registered. Sybil attacks on wide-open registrations could enable malicious storage providers to continually lock up deal registration. Integrating logic to reduce this surface area of attack is encouraged.
  • Execution of handle_filecoin_method is critical. Implementations must assume that malicious actors are entering this code path.

For the Storage Provider:

  • Must sufficiently trust the contract client to honor deal registrations. A malicious contract client could spawn a bunch of proposals, enable proper registrations, and then fail PSD authorization via handle_filecoin_method to deny deal completion. This would waste storage provider resources, and with full unmonitored automation could prove to be a denial of service vector against specific storage providers.
  • Because of this, the storage provider should consider listening to certain clients or marketplaces, but not all by default.

Incentive Considerations

This standard provides some acceleration of existing incentives, but does not add any new ones:

  • Storage Providers are incentivized to listen to contract clients as a no-touch way to accept deals. This includes the potential upside of sourcing FIL+ deals without having to discover or engage customers
  • Programmatic deal making provides an additional incentive for SPs to run deal-making software programs like Boost, which will support this standard out-of-the-box and provide additional configuration.

Product Considerations

This FRC is built to enable two key product goals:

  1. Allow FVM devs to build dapps and write smart contracts on FVM that enable them to create storage deals with SPs.
  2. Enable SPs to participate efficiently with smart contracts on FVM with minimal time, bandwidth and gas expense.

This opens up a large world of use cases in the FVM landscape: DataDAOs that incentivize the storage of data, perpetual storage, storage automation (though both repair and replica workers), and more. This standard aims to enable user actors to stay composable and extensible.

Note, however that the FRC is ONLY meant to standardize communication between participating SPs and FVM smart contracts. It is not meant to standardize deal proposal creation, SP ordering or any other aspect in the lifecycle of deal making.

While the aforementioned will be present in an opinionated (and complete) implementation available for developers, it will not be specified in this FRC. As such, it can be readily modified based on storage use cases on FVM.

Some examples:

  • The client has complete freedom in how the deal proposal is implemented and stored. The deal proposal could be stored in state. It could also be generated on the spot, stored in a database, or stored through some other method altogether.
  • The client may need extra interaction to claim a deal. This might mean making an off-chain call or some other private implementation. The FRC grants the client the flexibility of doing so as a separate set of methods specific to their use case.

Implementation

Copyright

Copyright and related rights waived via CC0.