Implementation of the Content Claims Protocol.
Read the spec.
These are the types of claim that we're interested in from the spec:
Claims that a CID is available at a URL.
Capability: assert/location
Input:
{
content: CID /* CAR CID */,
location: ['https://r2.cf/bag...car', 's3://bucket/bag...car'],
range?: { offset: number, length?: number } /* Optional: Byte Range in URL */
}Claims that a CID includes the contents claimed in another CID.
Capability: assert/inclusion
Input:
{
content: CID /* CAR CID */,
includes: CID /* CARv2 Index CID */,
proof?: CID /* Optional: zero-knowledge proof */
}Claims that a content graph can be found in blob(s) that are identified and indexed in the given index CID.
Capability: assert/index
Input:
{
content: CID /* CAR CID */,
/**
* Link to a Content Archive that contains the index.
* e.g. `index/sharded/[email protected]`:
* @see https://github.com/storacha/specs/blob/main/w3-index.md
*/
index: CID
}Claims that a CID's graph can be read from the blocks found in parts.
Capability: assert/partition
Input:
{
content: CID /* Content Root CID */,
blocks?: CID /* CIDs CID */,
parts: [
CID /* CAR CID */,
CID /* CAR CID */,
...
]
}Claims that the same data is referred to by another CID and/or multihash. For example a CAR CID and it's CommP Piece CID.
Capability: assert/equals
Input:
{
content: CID /* CID */,
equals: CID /* CID */
}Claims that a CID links to other CIDs. Like a partition claim crossed with an inclusion claim, a relation claim asserts that a block of content links to other blocks, and that the block and it's links may be found in the specified parts. Furthermore, for each part you can optionally specify an inline inclusion claim (specifying what is included in the part) and for each inclusion an optional inline partition claim (specifying parts in which the inclusion CID may be found).
Capability: assert/relation
Input:
{
content: CID /* Block CID */,
children: [
CID /* Linked block CID */,
CID /* Linked block CID */,
...
],
parts: [
{
content: CID /* CAR CID */,
includes?: {
content: CID /* CARv2 Index CID */,
parts?: [
CID /* CAR CID */,
...
]
}
},
...
]
}Client libraries make reading and writing claims easier.
The production deployment is at https://claims.web3.storage.
Fetch a CAR full of content claims for the base58 encoded (Multibase base58btc) content hash in the URL path.
Query parameters:
?walk=- a CSV list of properties in claims to walk in order to return additional claims about the related CIDs. Any property that is a CID can be walked. e.g.?walk=parts,includes.
As above, except passing a CID instead of multihash.
There is a command line interface for invoking the HTTP API in ./packages/cli.
It's published to npm, so if you have npm installed, you should be able to:
- run the cli with
npx @web3-storage/content-claims-cli # args - install the cli globally with
npm install -g @web3-storage/content-claims-cli
- then invoke it like this to see CAR bytes on stdout
claim read bafybeifftytx763g6u5gvisyrng5de4wmxhwlc4sjxfhohmab6yjprcmbi --walk=parts,includes
- then invoke it like this to see CAR bytes on stdout
For more CLI Usage instructions, see the content-claims-cli README
The repo contains the infra deployment code and the service implementation.
├── packages - content-claims core, CLI and AWS infra
└── stacks - sst and AWS CDK code to deploy all the things
To work on this codebase you need:
- Node.js >= v18 (prod env is node v18)
- An AWS account with the AWS CLI configured locally
- Copy
.env.tplto.envand fill in the blanks - Install the deps with
npm i
Deploy dev services to your AWS account and start dev console. You may need to provide a --profile to pick the aws profile to deploy with.
npm startSee: https://docs.sst.dev for more info on how things get deployed.
The following should be set in the env when deploying. Copy .env.tpl to .env to set in dev.
# Private key for the service.
PRIVATE_KEY=MgCblCY...
# Name of DynamoDB table for indexing claims.
CLAIM_TABLE=claim
# Region of the DynamoDB to query.
CLAIM_TABLE_REGION=us-west-2
# Name of the S3 bucket for storing signed claims.
CLAIM_BUCKET=claim
# Region of the claim bucket.
CLAIM_BUCKET_REGION=us-west-2
# (optional) Elastic IPFS DynamoDB block index table.
BLOCK_INDEX_TABLE=blocks-cars-position
# (optional) Elastic IPFS DynamoDB region.
BLOCK_INDEX_REGION=us-west-2
# (optional) Service DID, if using DID with method other than `key`.
SERVICE_DID=did:web:claims.web3.storage
# (optional) Sentry key for error reporting.
SENTRY_DSN=The multibase encoded ED25519 keypair used as the signing key for content-claims.
Generated by @ucanto/principal EdSigner via ucan-key
Example: MgCZG7EvaA...1pX9as=
Name of DynamoDB table for indexing claims.
Region of the DynamoDB to query.
Name of the S3 bucket for storing claims.
Region of the S3 bucket.
The Elastic IPFS DynamoDB block index table. If set and no other claim is found for a CID, the content claims API will query this table and materialize claims for any index data found. This is expected to be a table with the following structure:
interface BlockIndexTable {
blockmultihash: string // base58btc encoded block multihash (partition key)
carpath: string // bucket key, format: `REGION/BUCKET_NAME/KEY.car`
offset: number // byte offset within CAR file
length: number
}Note: this table requires read-only access.
Region of the DynamoDB that houses the block index table (set if different from CLAIM_TABLE_REGION).
The DID of the service, if using DID with method other than key.
The URL of the deployed service.
Data source name for Sentry application monitoring service.
Set production secrets in AWS SSM via sst secrets. The region must be set to the one you deploy that stage to:
# set `PRIVATE_KEY` for prod
$ npx sst secrets set --region us-west-2 --stage prod PRIVATE_KEY "MgCblCY...="To set a fallback value for staging or an ephemeral PR build use sst secrets set --fallback
# set `PRIVATE_KEY` for any stage in us-east-2
$ npx sst secrets set --region us-east-2 --fallback PRIVATE_KEY "MgCZG7...="note The fallback value can only be inherited by stages deployed in the same AWS account and region.
Confirm the secret value using sst secrets list
$ npx sst secrets list --region us-east-2
PRIVATE_KEY MgCZG7...= (fallback)
$ npx sst secrets list --region us-west-2 --stage prod
PRIVATE_KEY M...=interface ClaimTable {
/** CID of the UCAN invocation task we received this claim in. */
claim: string // Note: sort key
/** The subject of the claim (base58 encoded multihash). */
content: string // Note: partition key
/** UCAN expiration */
expiration: number
}Signed claim data is stored in an S3 bucket with the following key format:
<CLAIM_CID>/<CLAIM_CID>.car
Note: CID is base32 encoded.
Feel free to join in. All welcome. Please open an issue!
Dual-licensed under MIT + Apache 2.0