Test suite generation

[stevenvegt]

This is a proposal for decentralized-identity/trustdidweb#87.

The format is inspired by the JSON-LD test suite: https://w3c.github.io/json-ld-api/tests/.

The tests are described in a manifest with entries.

The datatype of an entry looks like this (in Go):

type TestEntry struct {
	Id          string           `json:"id"`
	Type        []string         `json:"type"`
	Purpose     string           `json:"purpose"`
	Input       string           `json:"input,omitempty"`
	Expect      string           `json:"expect,omitempty"`
	SigningKey  jwk.Key          `json:"signingKey,omitempty"`
	Params      tdw.LogParams    `json:"params,omitempty"`
	DIDDocument tdw.DIDDocument  `json:"didDocument,omitempty"`
	Options     TestEntryOptions `json:"options,omitempty"`
}

There are 3 types of tests: CreationTest, UpdateTest and VerificationTest. Depending on the test type, a combination of fields of the entry is used.
There are 2 file references, input and expected which contain filenames to DID Logs. Their usage depends on the test type. For example, CreationTest only uses expected but the UpdateTest also uses the input.

A test can be either a PositiveEvaluationTest or a NegativeEvaluationTest. Inspired by the JSON-LD test vocabulary

Example of an entry:

  {
    "id": "tc001",
    "type": [
      "CreationTest",
      "PositiveEvaluationTest"
    ],
    "purpose": "Create a new log",
    "expect": "testdata/tc001-expect.json",
    "signingKey": {
      "crv": "Ed25519",
      "d": "J3l2VTBrN9GckzX8wt0p_FjJ8TDFdKWWNFM-MuHZWVE",
      "kty": "OKP",
      "x": "Bm-ZiwCZgDgv8K7xf3eGR77Z3vqItWhd0Np5PftRYvM"
    },
    "didDocument": {
      "@context": [
        "https://www.w3.org/ns/did/v1"
      ],
      "id": "did:tdw:{SCID}:example.com"
    },
    "options": {
      "signingTime": "2024-09-06T15:53:15+02:00"
    }
  }

Tests can be generated by executing $ go run cmd/*.go generate --tests from the projects base directory.

Tests can be performed by executing $ go test -v run TestSuite ./...

[andrewwhitehead]

I've added a draft implementation here: decentralized-identity/trustdidweb-py#11

It would be useful for the verification tests if the metadata included the DID to be verified, as the resolver expects to know that in advance (and check it). Maybe as options["did"]?

[stevenvegt]

I want to update the main branch to support v0.4 and update the testsuite in one go. So I'm going to merge this PR and work on it together.