Support DSSE in Metadata API

[lukpueh]

Notes to reviewer:

• based on: #2165 and sslib#487
  (tests should pass locally, when installing tuf and securesystemslib from these branches)
• relevant diff: lukpueh/tuf@no-key...dsse
• leaving as draft until above PRs are merged and, in the case of securesystemslib, released
• the design is smart (h/t @PradyumnaKrishna), but adds quite some complexity to metadata API
• this needs better tests and documentation, but I wanted to get a first round of feedback before spending more time on it
• the commit messages have many details

Description of the changes being introduced by the pull request:

Adds a TUF-specific DSSE (Envelope) implementation and defines an abstract interface (BaseMetadata) for common Envelope and Metadata operations. BaseMetadata can be used independently of the underlying metadata type to read or write from and to file or bytes, and to create or verify signatures.

caveat: Optional arguments may be different for common Envelope and Metadata methods. This is because they require de/serialization operations at different stages. Envelope must deserialize, when accessing the payload (get_signed), but Metadata not. Metadata, OTOH, must serialize to canonical json when signing or verifying (sign and verify_delegate), but Envelope not.

This PR also adopts the new securesystemslib de/serialization infrastructure, which, most notably,

• separates basic JSON bytes deserialization from custom object instantiation,
• adds a SerializationMixin, which provides convenience methods to read or write from and to file or bytes

caveat: users of the serialization mixin must implement methods that return default de/serializers that can be used by the mixin.

[lukpueh]

I created these aliases to not break the Metadata API.

[lukpueh]

@jku raised a valid concern about whether there is any merit in a common base class for Metadata and Envelope in the python-tuf metadata API.

Coming from the in-toto dsse integration work I had the following workflows in mind:

# metadata consumer
C1. load and verify metadata envelope
C2. extract payload and use it

# metadata producer
P1. create payload and wrap in metadata envelope
P2. sign and dump

A common base class is helpful for applications to reduce case handling for common operations (C1, P2). Users of the python-tuf metadata API, however, can be explicit about which type of metadata envelope they want to load, verify, sign, dump.

Also see inline comments for issues with common operations having different requirements...

[lukpueh]

The deserializer argument in get_signed should be configurable, e.g. via optional delegator_deserializer argument in verify_delegate.

Or maybe verify_delegate just should not be a method on the envelope, but rather on the already deserialized payload. At least for DSSE it does not seem like the right place to deserialize the payload here.

[lukpueh]

Using the current implementation of JSONDeserializer as default deserializer for any BaseMetadata (i.e. Metadata and Envelope) can lead to unexpected behavior, because it can deserialize both traditional metadata and dsse:

python-tuf/tuf/api/serialization/json.py

Lines 51 to 55 in 2bc97c4

	if "payload" in json_dict:
	return Envelope.from_dict(json_dict)
	if "signed" in json_dict:
	return Metadata.from_dict(json_dict)

This is the intended behavior for BaseMetadata, but a bug on either Metadata or Envelope:

#  Each `from_file` call below uses the default `JSONDeserializer` returned
#  from `_default_deserializer()`, and thus can return both `Metadata` or `Envelope`
BaseMetadata.from_file("path/to/any/metadata")  # returning `Metadata` or `Envelope` is expected
Metadata.from_file("path/to/any/metadata")  # should only return `Metadata`
Envelope.from_file("path/to/any/metadata")  # should only return `Envelope` 

[lukpueh]

Closing in favour of #2385