Skip to content
This repository has been archived by the owner on Nov 22, 2024. It is now read-only.

Content formatting updates #34

Merged
merged 9 commits into from
Oct 9, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 118 additions & 69 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,22 @@
# SCITT API interoperability client
# SCITT API Interoperability Client

This repository contains the source code for the SCITT API interoperability client and sample emulator.

It is meant to allow experimenting with [SCITT](https://datatracker.ietf.org/wg/scitt/about/) APIs and formats and proving interoperability of implementations.
It is meant to allow experimenting with [SCITT](https://datatracker.ietf.org/wg/scitt/about/) APIs and formats and proving interoperability of implementations.

Note the SCITT standards are not yet fully published and are subject to change. This repository aims to keep up with changes to the WG output as faithfully as possible but in the event of inconsistencies between this and the IETF WG documents, the IETF documents are primary.
Note the SCITT standards are not yet fully published and are subject to change.
This repository aims to keep up with changes to the WG output as faithfully as possible but in the event of inconsistencies between this and the IETF WG documents, the IETF documents are primary.

## Prerequisites

The emulator assumes a Linux environment with Python 3.8 or higher.
The emulator assumes a Linux environment with Python 3.10 or higher.
On Ubuntu, run the following to install Python:

```sh
sudo apt install python3.8 python3.8-venv
sudo apt install python3.10-venv
```

### Optional Dependencies

If you want to use conda, first install it:

Expand All @@ -27,111 +29,148 @@ conda env create -f environment.yml
conda activate scitt
```

## Clone the emulator
## Clone the Emulator

Clone the scitt-api-emulator repository and change into the scitt-api-emulator folder:
1. Clone the scitt-api-emulator repository and change into the scitt-api-emulator folder:

```sh
git clone https://github.com/scitt-community/scitt-api-emulator.git
```
or for ssh:
```sh
git clone https://github.com/scitt-community/scitt-api-emulator.git
```

```sh
git clone [email protected]:scitt-community/scitt-api-emulator.git
```
then:
1. Move into the emulator director to utilize the local commands

```sh
cd scitt-api-emulator
```
```sh
cd scitt-api-emulator
```

## Start the proxy server
## Start the Proxy Server

The proxy server supports 2 options currently:
- 'CCF' uses the emulator server to create and verify receipts using the CCF tree algorithm
- 'RKVST' uses the RKVST production SaaS server to create and verify receipts using native Merkle trees

Note the emulated is for experimentation only and not recommended for production use.
- 'CCF' uses the emulator server to create and verify receipts using the CCF tree algorithm
- 'RKVST' uses the RKVST production SaaS server to create and verify receipts using native Merkle trees

### Start a fake emulated SCITT service
**Note:** _the emulator is for experimentation only and not recommended for production use._

```sh
./scitt-emulator.sh server --workspace workspace/ --tree-alg CCF
```
### Start a Fake Emulated SCITT Service

### Start an RKVST SCITT proxy service
1. Start the service, under the `/workspace` directory, using CCF

```sh
./scitt-emulator.sh server --workspace workspace/ --tree-alg RKVST
```
```sh
./scitt-emulator.sh server --workspace workspace/ --tree-alg CCF
```

1. The server is running at http://localhost:8000/ and uses the `/workspace` folder to store the service parameters and service state
**Note:** _The default port is `8000` but can be changed with the `--port` argument._
SteveLasker marked this conversation as resolved.
Show resolved Hide resolved
1. Start another shell to run the test scripts, leaving the above shell for diagnostic output
1. Skip to [Create Claims](#create-claims)

### Start an RKVST SCITT Proxy Service

1. Start the service, under the `/workspace` directory, using RKVST
The default port is `8000` but can be changed with the `--port` argument.

The default port is 8000 but can be changed with the `--port` argument.
```sh
./scitt-emulator.sh server --workspace workspace/ --tree-alg RKVST
```

Now the server is running at http://localhost:8000/ and uses `workspace/` to store the service parameters and service state.
### Executing Commands

The service has the following REST API:

- `POST /entries` - submit a COSE_Sign1 claim to the emulator and return an entry id
- `POST /entries` submit a COSE_Sign1 claim as HTTP body, with a JSON response containing `"entry_id"`
- `GET /entries/<entry_id>` - retrieve the COSE_Sign1 claim for the corresponding entry id
- `GET /entries/<entry_id>/receipt` - retrieve the SCITT receipt for corresponding entry id
- `GET /entries/<entry_id>/receipt` to retrieve the SCITT receipt.

The following steps should be done from a different terminal, leaving the service running in the background.
**Note:** The `submit-claim` and `retrieve-claim` commands use the default service URL `http://127.0.0.1:8000` which can be changed with the `--url` argument.
They can be used with the built-in server or an external service implementation.

### Create claims
### Create Signed Claims

```sh
./scitt-emulator.sh client create-claim --issuer did:web:example.com --content-type application/json --payload '{"sun": "yellow"}' --out claim.cose
```
1. Create a signed `json` claim with the payload: `{"sun": "yellow"}`, saving the formatted output to `claim.cose`

Note: The emulator does not verify claim signatures and generates an ad-hoc key pair to sign the claim.
```sh
./scitt-emulator.sh client create-claim \
--issuer did:web:example.com \
--content-type application/json \
--payload '{"sun": "yellow"}' \
--out claim.cose
```

### Submit claims and retrieve receipts
_**Note:** The emulator generates an ad-hoc key pair to sign the claim and does not verify claim signatures upon submission._

```sh
./scitt-emulator.sh client submit-claim --claim claim.cose --out claim.receipt.cbor
```
2. View the signed claim by uploading `claim.cose` to one of the [CBOR or COSE Debugging Tools](#cose-and-cbor-debugging)

### Submit Claims and Retrieve Receipts

1. Submit the Signed Claim

```sh
./scitt-emulator.sh client submit-claim \
--claim claim.cose \
--out claim.receipt.cbor
```

The `submit-claim` command uses the default service URL `http://127.0.0.1:8000` which can be changed with the `--url` argument. It can be used with the built-in server or an external service implementation.
1. View the response, noting the `Entry ID` value

This command sends the following two requests:
```output
Claim Registered:
json: {'entryId': '1'}
Entry ID: 1
Receipt: ./claim.receipt.cbor
```

1. `POST /entries` with the claim file as HTTP body. The response is JSON containing `"entry_id"`.
2. `GET /entries/<entry_id>/receipt` to retrieve the SCITT receipt.
1. Save the entryId to an environment variable

### Retrieve claims
```sh
ENTRY_ID=<entryId>
```

### Retrieve Claims

1. Retrieve the claim, based on the ENTRY_ID set from the `submit-claim` command above

```sh
./scitt-emulator.sh client retrieve-claim --entry-id 123 --out claim.cose
./scitt-emulator.sh client retrieve-claim \
--entry-id $ENTRY_ID \
--out claim.cose
```

The `retrieve-claim` command uses the default service URL `http://127.0.0.1:8000` which can be changed with the `--url` argument. It can be used with the built-in server or an external service implementation.

This command sends the following request:

- `GET /entries/<entry_id>` to retrieve the claim.

### Retrieve receipts
### Retrieve Receipts

```sh
./scitt-emulator.sh client retrieve-receipt --entry-id 123 --out receipt.cbor
```
1. Replace the `<entryId>` with the value from the `submit-claim` command above

```sh
./scitt-emulator.sh client retrieve-receipt \
--entry-id $ENTRY_ID \
--out receipt.cbor
```

The `retrieve-receipt` command uses the default service URL `http://127.0.0.1:8000` which can be changed with the `--url` argument. It can be used with the built-in server or an external service implementation.
The `retrieve-receipt` command uses the default service URL `http://127.0.0.1:8000` which can be changed with the `--url` argument.
It can be used with the built-in server or an external service implementation.

This command sends the following request:

- `GET /entries/<entry_id>/receipt` to retrieve the receipt.

### Validate receipts
### Validate Receipts

```sh
./scitt-emulator.sh client verify-receipt --claim claim.cose --receipt claim.receipt.cbor --service-parameters workspace/service_parameters.json
./scitt-emulator.sh client verify-receipt \
--claim claim.cose \
--receipt claim.receipt.cbor \
--service-parameters workspace/service_parameters.json
```

The `verify-receipt` command verifies a SCITT receipt given a SCITT claim and a service parameters file. This command can be used to verify receipts generated by other implementations.
The `verify-receipt` command verifies a SCITT receipt given a SCITT claim and a service parameters file.
This command can be used to verify receipts generated by other implementations.

The `service_parameters.json` file gets created when starting a service using `./scitt-emulator.sh server`. The format of this file is not standardized and is currently:
The `workspace/service_parameters.json` file gets created when starting a service using `./scitt-emulator.sh server`.
The format of this file is not standardized and is currently:

```json
{
Expand All @@ -145,30 +184,40 @@ The `service_parameters.json` file gets created when starting a service using `.

`"signatureAlgorithm"` and `"serviceCertificate"` are additional parameters specific to the [`CCF` tree algorithm](https://ietf-scitt.github.io/draft-birkholz-scitt-receipts/draft-birkholz-scitt-receipts.html#name-additional-parameters).

### COSE and CBOR debugging
To view the file:

```sh
cat workspace/service_parameters.json | jq
```

### COSE and CBOR Debugging

The following websites can be used to inspect COSE and CBOR files:

- https://gluecose.github.io/cose-viewer/
- https://cbor.me/
- [gluecose.github.io/cose-viewer](https://gluecose.github.io/cose-viewer/)
- [cbor.me](https://cbor.me/)

## Code structure
## Code Structure

`scitt_emulator/scitt.py` contains the core SCITT algorithms that are agnostic of a specific tree algorithm.

`scitt_emulator/ccf.py` is the implementation of the [CCF tree algorithm](https://ietf-scitt.github.io/draft-birkholz-scitt-receipts/draft-birkholz-scitt-receipts.html#name-ccf-tree-algorithm). For each claim, a receipt is generated using a fake but valid Merkle tree that is independent of other submitted claims. A real CCF service would maintain a single Merkle tree covering all submitted claims and auxiliarly entries.
`scitt_emulator/ccf.py` is the implementation of the [CCF tree algorithm](https://ietf-scitt.github.io/draft-birkholz-scitt-receipts/draft-birkholz-scitt-receipts.html#name-ccf-tree-algorithm).
For each claim, a receipt is generated using a fake but valid Merkle tree that is independent of other submitted claims.
A real CCF service would maintain a single Merkle tree covering all submitted claims and auxiliary entries.

`scitt_emulator/rkvst.py` is a simple REST proxy that takes SCITT standard API calls and routes them through to the [RKVST production SaaS service](https://app.rkvst.io). Each claim is stored in a Merkle tree underpinning a Quorum blockchain and receipts contain valid, verifiable inclusion proofs for the claim in that Merkle proof. [More docs on receipts here](https://docs.rkvst.com/platform/overview/scitt-receipts/).
`scitt_emulator/rkvst.py` is a simple REST proxy that takes SCITT standard API calls and routes them through to the [RKVST production SaaS service](https://app.rkvst.io).
Each claim is stored in a Merkle tree underpinning a Quorum blockchain and receipts contain valid, verifiable inclusion proofs for the claim in that Merkle proof.
[More docs on receipts here](https://docs.rkvst.com/platform/overview/scitt-receipts/).
SteveLasker marked this conversation as resolved.
Show resolved Hide resolved

`scitt_emulator/server.py` is a simple Flask server that acts as a SCITT transparency service.

`scitt_emulator/client.py` is a CLI that supports creating claims, submitting claims to and retrieving receipts from the server, and verifying receipts.

In order to add a new tree algorithm, a file like `scitt_emulator/ccf.py` must be created and the containing class be added in `scitt_emulator/tree_algs.py`.

## Run tests
## Run Tests

```
```bash
./run-tests.sh
```

Expand Down
10 changes: 5 additions & 5 deletions docs/registration_policies.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,21 +140,21 @@ echo ${CLAIM_PATH}
Example running allowlist check and enforcement.

```console
$ npm install -g nodemon
$ nodemon -e .cose --exec 'find workspace/storage/operations -name \*.cose -exec nohup sh -xe policy_engine.sh $(cat workspace/service_parameters.json | jq -r .insertPolicy) {} \;'
npm install -g nodemon
nodemon -e .cose --exec 'find workspace/storage/operations -name \*.cose -exec nohup sh -xe policy_engine.sh $(cat workspace/service_parameters.json | jq -r .insertPolicy) {} \;'
```

Also ensure you restart the server with the new config we edited.

```console
$ scitt-emulator server --workspace workspace/ --tree-alg CCF --use-lro
scitt-emulator server --workspace workspace/ --tree-alg CCF --use-lro
```

Create claim from allowed issuer (`.org`) and from non-allowed (`.com`).

```console
$ scitt-emulator client create-claim --issuer did:web:example.com --content-type application/json --payload '{"sun": "yellow"}' --out claim.cose
Claim written to claim.cose
A COSE signed Claim was written to: claim.cose
SteveLasker marked this conversation as resolved.
Show resolved Hide resolved
$ scitt-emulator client submit-claim --claim claim.cose --out claim.receipt.cbor
Traceback (most recent call last):
File "/home/alice/.local/bin/scitt-emulator", line 33, in <module>
Expand All @@ -176,7 +176,7 @@ On instance['issuer']:
'did:web:example.com'

$ scitt-emulator client create-claim --issuer did:web:example.org --content-type application/json --payload '{"sun": "yellow"}' --out claim.cose
Claim written to claim.cose
A COSE signed Claim was written to: claim.cose
$ scitt-emulator client submit-claim --claim claim.cose --out claim.receipt.cbor
Claim registered with entry ID 1
Receipt written to claim.receipt.cbor
Expand Down
5 changes: 2 additions & 3 deletions postman/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
Testing the SCITT Emulator with Postman.
# Testing the SCITT Emulator with Postman

- [Install Postman](https://www.postman.com/downloads/)
- [Install Newman](https://support.postman.com/hc/en-us/articles/115003703325-How-to-install-Newman)

We will need to install some extra newman commands as well:

```
```sh
npm install -g newman newman-reporter-htmlextra
```

Expand All @@ -14,4 +14,3 @@ Run this command from this directory:
```sh
./run-sanity.sh
```

11 changes: 8 additions & 3 deletions scitt_emulator/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,9 @@ def submit_claim(
# Submit claim
response = client.post(f"{url}/entries", content=claim, headers={
"Content-Type": "application/cose"})

post_response=response.json()

if response.status_code == 201:
entry = response.json()
entry_id = entry["entryId"]
Expand All @@ -115,13 +118,15 @@ def submit_claim(
response = client.get(f"{url}/entries/{entry_id}/receipt", timeout=15)
receipt = response.content

print(f"Claim registered with entry ID {entry_id}")
print("Claim Registered:")
print(f" json: {post_response}")
print(f" Entry ID: {entry_id}")

# Save receipt to file
with open(receipt_path, "wb") as f:
f.write(receipt)

print(f"Receipt written to {receipt_path}")
print(f" Receipt: ./{receipt_path}")

# Save entry ID to file
if entry_id_path:
Expand All @@ -138,7 +143,7 @@ def retrieve_claim(url: str, entry_id: Path, claim_path: Path, client: HttpClien
with open(claim_path, "wb") as f:
f.write(claim)

print(f"Claim written to {claim_path}")
print(f"A COSE signed Claim was written to: {claim_path}")


def retrieve_receipt(url: str, entry_id: Path, receipt_path: Path, client: HttpClient):
Expand Down
Loading
Loading