Move (most of) Key to Securesystemslib

[jku]

This updates python-tuf sources for latest securesystemslib signer api changes. Most of the changes are in tests (since we test Key quite extensively) but verify_delegate() is also significantly rewritten. The commits are mostly logical -- the one exception is that the original refactoring of role/key lookup in Metadata was not great: it is fixed in a later commit by moving the functionality from Metadata to Targets and Root where it makes more sense.

Changelist:

• Key moves to Securesystemslib and the Key API changes:
  • Key.from_securesystemslib_key() is now SSlibKey.from_securesystemslib_key()
  • Key.verify_signature() arguments and exceptions change (it used to take Metadata as arg)
• Metadata.verify_delegate() has to be rewritten because of Key.verify_signature() changes
• Introduced some new API calls in Targets and Root: get_delegated_role() and get_key(): these simplify the role and key lookup in Metadata.verify_delegate()

WRT API change:

• this is clearly an API change (as Key changes method signatures), but not something users should suffer badly from
• Key, Signer and Signature are still technically part of the metadata API so no regression there: they just come from a securesystemslib module now.
• The Key implementations (like SSlibKey and SSlibSigner) are not currently importable through Metadata API but I don't think that is a problem?

[jku]

Unsure what to do with the documentation: I don't want to reproduce the docs in python-tuf... but I don't know if we can import the Key docs from securesystemslib?

[lukpueh]

but I don't know if we can import the Key docs from securesystemslib?

I think that's a good idea. It's what we for in-toto's key utilities:
https://in-toto.readthedocs.io/en/latest/api.html#key-utilities

[jku]

but I don't know if we can import the Key docs from securesystemslib?

I think that's a good idea. It's what we for in-toto's key utilities: https://in-toto.readthedocs.io/en/latest/api.html#key-utilities

oh nice: we should be doing that for Signer and Signature as well -- basically for the "modern" securesystemslib api.

it's interesting how that autodoc magic seems to work without ever importing securesystemslib...

[jku]

this is now out of date since I modified the securesystemslib PR -- it's still roughly correct, but I am not sure if a python-tuf Key is even needed anymore, maybe not. Will update next week

[coveralls]

Pull Request Test Coverage Report for Build 3939714721

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 48 of 54 (88.89%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-0.5%) to 97.681%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tuf/api/metadata.py	43	49	87.76%

Totals
Change from base Build 3911059358:	-0.5%
Covered Lines:	1313
Relevant Lines:	1336

---

💛 - Coveralls

[jku]

this is testing leftover -- but it does lead to one passing test 🍾

[lukpueh]

Just tried out signing with post quantum keys (sphincs+) using the new signer API. Works like a charm and out of the box! 🎉 cc @rugo

see snippet

# Install `tuf` from this branch and `securesystemslib` from secure-systems-lab/securesystemslib#456
python3 -m pip install git+https://github.com/jku/python-tuf.git@no-key#egg=tuf
python3 -m pip install --force-reinstall git+https://github.com/jku/securesystemslib.git@privkeyuri#egg=securesystemslib[crypto,pynacl,PySPX]

POC: Verify that root metadata is signed by at least one sphincs+ key as required

import os

from securesystemslib.keys import generate_sphincs_key
from securesystemslib.signer import Signer, SSlibKey, SSlibSigner

from tuf.api.metadata import Metadata, Root

# Generate pq keys
pq_key = generate_sphincs_key()
pq_pubkey = SSlibKey.from_securesystemslib_key(pq_key)
pq_signer = SSlibSigner(pq_key)

# # Generic solution: Use `Signer.from_priv_key_uri` instead of `SSlibSigner`,
# #     this also works:
#
# os.environ["PQ_KEY"] = pq_key["keyval"]["private"]
# pq_signer = Signer.from_priv_key_uri("envvar:PQ_KEY", pq_pubkey)

# Generate root role and configure signature requirement
root = Root()
root.add_key(pq_pubkey, Root.type)

# Wrap root in metadata envelope and add signature
root_metadata = Metadata(root)
root_metadata.sign(pq_signer)

# Assert `verify_delegate` indeed needs at least one sig from above key
assert root_metadata.signed.roles["root"].threshold == 1
assert root_metadata.signed.roles["root"].keyids == [pq_pubkey.keyid]

root_metadata.verify_delegate(Root.type, root_metadata)

[jku]

Current state:

• Works with securesystemslib master
• Should not be merged before there is a securesystemslib release (and we start depending on it)
• the main commit really should be split into a few separate ones
• the verify_delegate() refactor needs a good review

[jku]

Ok, PR is ready for review I think -- not for merge though, we do need a securesystemslib release before that. Updated description

[jku]

Rebased

[jku]

Rebased and then added

• commit to fix refactor Metadata._get_role_and_keys() #2272
• one more fix to key usage in examples
• tweak docs build so Key is still documented

[jku]

TODO: Package should depend on securesystemslib >= 0.26 now

[lukpueh]

TODO: Package should depend on securesystemslib >= 0.26 now

I think this is the only thing that's missing.

Otherwise LGTM. 👍

[lukpueh]

Thanks!