Add metadata container classes for signed metadata

[lukpueh]

Description of issue or feature request:
Both TUF and in-toto use a class based model for their metadata, with a similar signature wrapper class. The signature container (i.e. Metadata) and the base class for the signed contents (i.e. Signed) could be hosted in securesystemslib to re-use code and to better draw the line between in-toto/tuf concepts and pure cryptographic signing.

Current behavior:
No class-based metadata model.

Expected behavior:
Add class-based metadata model.

NOTE: I strongly suggest to adopt the classes that where recently added to TUF (theupdateframework/python-tuf#1112), which were based on the the ones from in-toto but don't use the 3rd-party attrs package and slightly streamline the design (most notably see recursive serialisation functions from_dictand to_dict).

Caveat:

A from_dict factory method on a generic Metadata class needs to know about the specific contained metadata, i.e. the subclass of Signed (e.g. tuf.Targets, tuf.Snapshot, in_toto.Link, in_toto.Layout, etc..), which will not be implemented in securesystemslib.

Possible solutions:

• Provide a way to register a signed._type-to-subclass-of-Signed mapping on the Metadata class
• Sub-class Metadata in tuf and in-toto and implement the factory there, e.g. class tuf.Metadata(securesystemslib.Metadata)

[lukpueh]

In regards to the implementation of de/serialization from and to different wireline formats, as well as canonicalization, what do others prefer? Implement these as different (1) methods on a generic class or use (2) subclassing? E.g.:

(1)Metadata.to_json(self), Metadata.to_asn1(self), Signed.to_canonical_json_bytes(self), etc...
or
(2) JSONMetadata(Metadata), ASN1Metadata(Metadata), OLPCCanonicalJsonSigned(Signed), etc...

I have a slight preference for (1) which is why I implemented it that way in tuf#1112. It seems a bit less engineered and also more explicit in it's usage. (2) OTOH might be more flexible and also feels purer in an OOP-sense.

Thoughts?

[joshuagl]

I have a slight preference for (2), but only because I think it makes it easier for adopters to implement the wireline format they care about in their own project and plug it in to securesystemslib and therefore tuf/in-toto. I'm imagining an interface, much like the StorageBackendInterface

[MVrachev]

We have implemented in TUF all of the classes planned for the metadata complex attribute and roles and I think we can reevaluate
the idea of that issue.

When we compare tuf Metadata class and In-toto Metablock we can see that the two classes diverge in multiple ways:

• Metablock uses ValidationMixin as its way of class attribute validation. In comparison, for Metadata we are in a process of identifying the best solution for validation. It's likely that for Metadata in TUF we will use a different approach.
  See ADR0007: How we are going to validate the new API codebase theupdateframework/python-tuf#1301 for context in the discussion and currently I am in the process of documenting information for each of the metadata attributes asking those 7 questions: Prototype descriptors and validation functions theupdateframework/python-tuf#1366 (comment).
• Metablock API functionality and naming is different;
  Metablock.load() should be equal to Metadata.from_dict() and Metablock.dump() should be equal to Metadata.to_file(), but in both of those cases the compared methods have a different functionality.
• Metablock has one additional argument self.compact_json() which seems to be used only in the __repr__ method

Then, the other option is to compare Signed and Signable. Then we see that those two diverge too:

• again in In-toto they have decided to use ValidationMixin compared with the situation in TUF where we have not decided.
• the only possible common attribute between the two classes is _type. Even expires is not common, because it's defined for Layout, but it's not for Link.

To conclude: I am not sure it will be easy or even that it makes sense to relocate Metadata or Signed in securesystemslib and use it in TUF and In-toto.
The only candidate that can be relocated from TUF to securesystemslib is the newly added Key class, but then the differences in how we validate our attributes should be resolved.

[joshuagl]

Metablock and Metadata are alternative implementations of the same shared concept (a container with signed data and signatures). Switching in-toto to use Metadata would require some effort, but may be worthwhile. If in-toto and tuf can share the same code for creating, validating and serialising Meta[data|block] via securesystemslib both projects will benefit from shared bug fixing and less code to maintain independently.

[MVrachev]

I agree about the positives, but let's see what the in-toto folks think about this @adityasaky, @SantiagoTorres?

[joshuagl]

I suspect in-toto's interest may depend on the progress of ITE-6 and how long they intend to retain support for the pre ITE-6 link format. Would certainly be good to hear from in-toto maintainers.

[adityasaky]

If I'm not mistaken, the Metablock / Metadata parts are more ITE-5 than ITE-6. I think I'm in favour of using sslib for that, because any upcoming Python support for signing-spec could also naturally be a part of sslib then.

I think the signed / signable formats are very project specific though? So worth keeping them in the respective implementations?

[SantiagoTorres]

I think it'd be wortwhile to maybe move this class to sslib... This would in fact reflect the intention of ITE-5, which is to dissociate the signature wrapper from the spec, and allow for implementers to use their own (while at the same time blessing the sslib block as a recommended, default format)

[lukpueh]

DSSE is in securesystemslib and might replace traditional Metadata at some point. It's not worth to move the two working Metadata implementations from tuf and in-toto Python implementations to securesystemslib.