Revise schema and formats facility

[lukpueh]

Description of issue or feature request:
Here are several loosely ordered observations in regards to securesystemslib's schema/formats facility:

Current behavior:

• Some schemas sound more specific than they are, e.g. RELPATH_SCHEMA == PATH_SCHEMA == AnyString. This is misleading, when assessing a function's capability of sanitizing inputs.

• Some schemas are an odd replacement for constants, e.g. schemas that define hard-coded strings, like "rsassa-pss-sha256", and are then used to check the same strings, which have been hardcoded into function default arguments.

• Schema validation seems generally overused. In TUF nearly every function runs check_match for each argument. IMHO argument sanitizing is mostly important in public-facing interfaces. Programming errors, on the other, should be caught through extensive testing and code review.
  Using it everywhere makes the code bloated, also because there are a lot of generic comments describing the checks, and the obligatory and also quite generic FormatError, if <arg> is not properly formatted- blocks in the <Exceptions> section of docstrings.

• Schema checking sometimes makes execution branches unreachable (also see Unreachable else best practice code-style-guidelines#18):

  X_OR_Y_SCHEMA.check_match(arg)
  
  if arg == X:
    # ...
  elif arg == Y:
    # ...
  else:
    raise WillNeverBeRaisedError()

  And I've even seen cases where WillNeverBeRaisedError is also listed in the docstring as Exception that might be raised (documentation rot)

• The error messages from checking schemas with the check_match method are often not helpful, because they usually don't show the value of the checked variable or lack context.

Expected behavior:

• Review existing schemas for their validity/strictness, especially when chained.

• Only use in public facing interfaces (open for discussion).
  This would also require a clearer definition of what functions should be public interfaces, which btw. TUF/in-toto integrators would greatly benefit from.

• At least don't blindly add <arg>.<SCHEMA>.check_match to every function. Coordinate with the rest of the function, and its purpose.

• Make sure the error message is helpful, e.g. by:

  • supporting an error message override argument in check_match, or by
  • using matches plus raise FormatError (or maybe even just ValueError) with a custom message instead of check_match.

• Disambiguate schemas and constants.

[lukpueh]

• The error messages from checking schemas with the check_match method are often not helpful, because they usually don't show the value of the checked variable or lack context.

Similar observations were made in theupdateframework/python-tuf#243

[lukpueh]

The many check_match-calls also have an impact on performance. E.g. when delegating 16K hashed bins for warehouse, >5% of the execution time was just check_match-ing.

[lukpueh]

Just discovered another documented, albeit (to me) unexpected behavior:

• Unrecognized keys in an object schema are allowed

>>> from securesystemslib import schema
>>> FOO_SCHEMA = schema.Object("FOO_SCHEMA", foo=schema.AnyString())
>>> FOO_SCHEMA.matches({"foo": "ABC", "bar": "interesting..."}) # Would expect to return False due to "bar"
True

[lukpueh]

While testing a format errors on different Python versions, I just discovered another issue with the Object(Schema):

Given that in Python <3.7 the dict key order is not guaranteed, iterating over the "required" fields ...

securesystemslib/securesystemslib/schema.py

Line 750 in 1c4161e

self._required = list(required.items())

... of a schema object in check_match may yield different error messages, if multiple required fields are missing.

securesystemslib/securesystemslib/schema.py

Lines 759 to 768 in 1c4161e

	for key, schema in self._required:
	# Check if 'object' has all the required dict keys. If not one of the
	# required keys, check if it is an Optional().
	try:
	item = object[key]
	except KeyError:
	# If not an Optional schema, raise an exception.
	if not isinstance(schema, Optional):
	message = 'Missing key ' + repr(key) + ' in ' + repr(self._object_name)