Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: filecoin api services events and tests #974

Merged
merged 21 commits into from
Oct 23, 2023
Merged

refactor: filecoin api services events and tests #974

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
4562f13
feat: wip filecoin-api implementation
Oct 5, 2023
8142de1
feat: wip events
Oct 10, 2023
127b110
refactor: filecoin api 2
vasco-santos Oct 12, 2023
7cacbbe
refactor: aux property
vasco-santos Oct 23, 2023
20a7a38
fix: receipt return types
vasco-santos Oct 23, 2023
3d4a24e
fix: invocation service config in tests
vasco-santos Oct 23, 2023
a55a18a
refactor: date to iso string
vasco-santos Oct 23, 2023
500f301
refactor: remove task cid
vasco-santos Oct 23, 2023
46a87f9
refactor: simplify errors
vasco-santos Oct 23, 2023
57c4e05
refactor: add type for deal info reducer
vasco-santos Oct 23, 2023
fab21b9
refactor: add type for deal info reducer
vasco-santos Oct 23, 2023
712c9d2
refactor: storefront config renamed to options
vasco-santos Oct 23, 2023
6d424f1
refactor: rename store error to record not found
vasco-santos Oct 23, 2023
d3c9f54
refactor: make has fail when not found like get
vasco-santos Oct 23, 2023
206be46
fix: client test
vasco-santos Oct 23, 2023
8e9df57
refactor: test interfaces in tests
vasco-santos Oct 23, 2023
084c969
refactor: test interfaces in tests
vasco-santos Oct 23, 2023
bb056f6
chore: remove todo
vasco-santos Oct 23, 2023
ae02947
fix: imports of test interface
vasco-santos Oct 23, 2023
d8cde3c
fix: remove unused imports
vasco-santos Oct 23, 2023
1fc9313
fix: imports
vasco-santos Oct 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/capabilities/src/filecoin/dealer.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ export const aggregateOffer = capability({
* CID of the DAG-CBOR encoded block with offer details.
* Service will queue given offer to be validated and handled.
*/
pieces: Schema.link(),
pieces: Schema.link({ version: 1 }),
}),
derives: (claim, from) => {
return (
Expand Down
4 changes: 2 additions & 2 deletions packages/capabilities/src/filecoin/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/**
* Filecoin Capabilities
*
*
* These capabilities are the entrypoint to the filecoin pipeline and are
* aliases for the filecoin storefront capabilities.
*
Expand All @@ -15,5 +15,5 @@
export {
filecoinOffer as offer,
filecoinSubmit as submit,
filecoinAccept as accept
filecoinAccept as accept,
} from './storefront.js'
16 changes: 9 additions & 7 deletions packages/capabilities/src/filecoin/lib.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,12 @@ const FR32_SHA2_256_TRUNC254_PADDED_BINARY_TREE = /** @type {const} */ (0x1011)
*/
const RAW_CODE = /** @type {const} */ (0x55)

export const PieceLink = /** @type {import('../types').PieceLinkSchema} */ (Schema.link({
code: RAW_CODE,
version: 1,
multihash: {
code: FR32_SHA2_256_TRUNC254_PADDED_BINARY_TREE,
},
}))
export const PieceLink = /** @type {import('../types').PieceLinkSchema} */ (
Schema.link({
code: RAW_CODE,
version: 1,
multihash: {
code: FR32_SHA2_256_TRUNC254_PADDED_BINARY_TREE,
},
})
)
51 changes: 39 additions & 12 deletions packages/capabilities/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,19 @@
import type { TupleToUnion } from 'type-fest'
import * as Ucanto from '@ucanto/interface'
import type { Schema } from '@ucanto/core'
import { InferInvokedCapability, Unit, DID, DIDKey, Link } from '@ucanto/interface'
import { Phantom, PieceLink, ProofData, uint64 } from '@web3-storage/data-segment'
import {
InferInvokedCapability,
Unit,
DID,
DIDKey,
Link,
} from '@ucanto/interface'
import {
Phantom,
PieceLink,
ProofData,
uint64,
} from '@web3-storage/data-segment'
import { space, info } from './space.js'
import * as provider from './provider.js'
import { top } from './top.js'
Expand Down Expand Up @@ -161,14 +172,20 @@ export type Space = InferInvokedCapability<typeof space>
export type SpaceInfo = InferInvokedCapability<typeof info>

// filecoin
export interface DealMetadata {
dataType: uint64
dataSource: SingletonMarketSource
}
/** @see https://github.com/filecoin-project/go-data-segment/blob/e3257b64fa2c84e0df95df35de409cfed7a38438/datasegment/verifier.go#L8-L14 */
export interface DataAggregationProof {
/**
* Proof the piece is included in the aggregate.
*/
inclusion: InclusionProof
auxDataType: uint64
auxDataSource: SingletonMarketSource
vasco-santos marked this conversation as resolved.
Show resolved Hide resolved
/**
* Filecoin deal metadata.
*/
aux: DealMetadata
}
/** @see https://github.com/filecoin-project/go-data-segment/blob/e3257b64fa2c84e0df95df35de409cfed7a38438/datasegment/inclusion.go#L30-L39 */
export interface InclusionProof {
Expand Down Expand Up @@ -213,15 +230,25 @@ export interface FilecoinSubmitSuccess {

export type FilecoinSubmitFailure = InvalidPieceCID | Ucanto.Failure

export type FilecoinAcceptSuccess = DataAggregationProof
export interface FilecoinAcceptSuccess extends DataAggregationProof {
aggregate: PieceLink
piece: PieceLink
}

export type FilecoinAcceptFailure = InvalidContentPiece | Ucanto.Failure
export type FilecoinAcceptFailure =
| InvalidContentPiece
| ProofNotFound
| Ucanto.Failure
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I seem to recall gozala saying we shouldn't add generic Failure to these unions...(there are more instances of this).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well this looks to be everywhere... Even in the recent capabilities like RateLimit, Customer, ...
If I remove it, there are a lot of typescript failures, so I will keep it and we can consider doing this removal across the board


export interface InvalidContentPiece extends Ucanto.Failure {
name: 'InvalidContentPiece'
content: PieceLink
}

export interface ProofNotFound extends Ucanto.Failure {
name: 'ProofNotFound'
}

// filecoin aggregator
export interface PieceOfferSuccess {
/**
Expand Down Expand Up @@ -256,7 +283,9 @@ export interface AggregateOfferSuccess {
}
export type AggregateOfferFailure = Ucanto.Failure

export type AggregateAcceptSuccess = DataAggregationProof
export interface AggregateAcceptSuccess extends DealMetadata {
aggregate: PieceLink
}
export type AggregateAcceptFailure = InvalidPiece | Ucanto.Failure

export interface InvalidPiece extends Ucanto.Failure {
Expand All @@ -269,7 +298,7 @@ export interface InvalidPiece extends Ucanto.Failure {
}

export interface InvalidPieceCID extends Ucanto.Failure {
name: 'InvalidPieceCID',
name: 'InvalidPieceCID'
piece: PieceLink
}

Expand All @@ -283,7 +312,7 @@ export interface DealDetails {
// TODO: start/end epoch? etc.
}

export type FilecoinAddress = `f${string}`
export type FilecoinAddress = string

export type DealInfoFailure = DealNotFound | Ucanto.Failure

Expand Down Expand Up @@ -343,9 +372,7 @@ export type AggregateOffer = InferInvokedCapability<
export type AggregateAccept = InferInvokedCapability<
typeof DealerCaps.aggregateAccept
>
export type DealInfo = InferInvokedCapability<
typeof DealTrackerCaps.dealInfo
>
export type DealInfo = InferInvokedCapability<typeof DealTrackerCaps.dealInfo>
// Top
export type Top = InferInvokedCapability<typeof top>

Expand Down
16 changes: 9 additions & 7 deletions packages/filecoin-api/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
"dealer": [
"dist/src/dealer.d.ts"
],
"chain-tracker": [
"dist/src/chain-tracker.d.ts"
"deal-tracker": [
"dist/src/deal-tracker.d.ts"
],
"errors": [
"dist/src/errors.d.ts"
Expand Down Expand Up @@ -54,9 +54,9 @@
"types": "./dist/src/dealer.d.ts",
"import": "./src/dealer.js"
},
"./chain-tracker": {
"types": "./dist/src/chain-tracker.d.ts",
"import": "./src/chain-tracker.js"
"./deal-tracker": {
"types": "./dist/src/deal-tracker.d.ts",
"import": "./src/deal-tracker.js"
},
"./storefront": {
"types": "./dist/src/storefront.d.ts",
Expand Down Expand Up @@ -86,7 +86,7 @@
"@ucanto/server": "^8.0.0",
"@ucanto/transport": "^8.0.0",
"@web3-storage/capabilities": "workspace:^",
"@web3-storage/data-segment": "^3.0.1"
"@web3-storage/data-segment": "^3.2.0"
},
"devDependencies": {
"@ipld/car": "^5.1.1",
Expand All @@ -108,7 +108,9 @@
"project": "./tsconfig.json"
},
"rules": {
"unicorn/expiring-todo-comments": "off"
"unicorn/expiring-todo-comments": "off",
"unicorn/prefer-number-properties": "off",
"jsdoc/check-indentation": "off"
},
"env": {
"mocha": true
Expand Down
118 changes: 0 additions & 118 deletions packages/filecoin-api/src/aggregator.js
View file
Open in desktop

This file was deleted.

1 change: 1 addition & 0 deletions packages/filecoin-api/src/aggregator/api.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
export {}
Loading
Loading