Add backwards-compatible SCHEMA replacement

[lukpueh]

Adds helper functions to cover all data validation cases in a backwards-compatible manner, where we previously used securesystemslib SCHEMA.

This is part of a series of patches to prepare for removal of legacy securesystemslib interfaces. (see secure-systems-lab/securesystemslib#183 for specific issues with SCHEMA)

Details

Most of the checks are simple type validations, now without the hidden redundancy of SCHEMA. (e.g. NAME_SCHEMA, PATH_SCHEMA, or ANY_STRING_SCHEMA all just checked if the input was a string, now there is only _check_str)

A few checks validate more complex data structures, most notably public key dictionaries. While SCHEMA whitelisted allowed fields and their values, we now just try to deserialise using securesystemslib's Signer API, and re-raise a deserialisation error as format error.

This is more Pythonic (if we can parse it as key, it is a key) and scales nicely, because the Signer API allows to register new public key schemes in user code.

Alternative / future work

This PR is a hack with the goal to remove schema in securesystemslibs without breaking the API in in-toto. It also has the nice side-effect of making validation more explicit and flexible.

A more sustainable solution can be found in the python-tuf Metadata API, where

• most of the data model (including keys and signatures) is represented by classes and validated on deserialization
• API inputs are above all validated to prevent unexpected behavior, or provide better UX
• in many cases, it's okay to just use the exceptions that come from using invalid input data
• type annotations are very useful

However, this would be a more invasive and surely breaking change. So, even though this PR adds a lot of code, I think we should go with it for now, and revise the class model later.

Reviewing commit by commit should make the review straight-forward.

[adityasaky]

LGTM!

[lukpueh]

Speaking of nice side-effect and flexibility. Here's me "registering" sigstore in user code and generating an in-toto link signed with my email address, with 0 changes in the library code. 😎

from securesystemslib.signer import (
    KEY_FOR_TYPE_AND_SCHEME,
    SIGNER_FOR_URI_SCHEME,
    Signer,
    SigstoreKey,
    SigstoreSigner,
)
from in_toto.runlib import in_toto_run

# All what's needed for in-toto to speak sigstore
KEY_FOR_TYPE_AND_SCHEME[("sigstore-oidc", "Fulcio")] = SigstoreKey

# This is only needed to use generic `Signer.from_priv_key_uri` interface.
# We could also load SigstoreSigner directly.
SIGNER_FOR_URI_SCHEME[SigstoreSigner.SCHEME] = SigstoreSigner

uri, key = SigstoreSigner.import_via_auth()

signer = Signer.from_priv_key_uri(uri, key)

link = in_toto_run("hello-sigstore", [], [], ["echo", "hello", "sigstore"], signer=signer)

# Hack to convert a `Key` to the in-toto -flavoured dict representation  
key_dict = key.to_dict()
key_dict["keyid"] = key.keyid

link.verify_signature(key_dict)