Use CUE to define schema for files

[jsoriano]

What does this PR do?

Uses CUE to define the schema for files instead of JSON schema (only integration manifest and changelog by now).

It includes #384 for semantic rules versioning.

Thoughts on using CUE for schema validation

TLDR; I am not convinced that it provides general advantages.

Pros:

• Expressive and compact syntax.
• Conditional code.
• Nice features like checking backwards compatibility.

Cons:

• No first-level support for descriptions, examples and so on, we would need to work on something else for documentation.
• Only approach for reusability of definitions is the use of packages, but packages loading don't support fs.FS (cue/load: support loading from io/fs.FS cue-lang/cue#607). Hacks needed.
• Syntax behaves a bit unexpectedly sometimes. Examples:
  • Data injection with tags is quite tricky, and doesn't always work.
  • Additional contents are only checked for named definitions, they are not checked when validating against the spec in the root of a file. This forces us to work with definitions, not such a big issue.
• Internals are hidden in internal libraries. No custom Go logic or types can be added, we'd need to fork or continue with semantic validation. Same limitations in that sense than jsonschema at the end.
• Validation of relative paths and data stream names would need to be done with semantic validation (current approach of using custom "formats" in JSON schema is tricky in any case).
• Errors are sometimes quite cryptic, at least in the context of schema validation. Transformations needed.
• Not 1.0 yet, Go APIs may change.

Why is it important?

To support multiple versions of the spec with the same files.

Checklist

• I have added test packages to test/packages that prove my change is effective.
• I have added an entry in versions/N/changelog.yml.
• Validate data stream names.
• Validate relative paths.

Related issues

• Alternative to Introduce format version v2 #380.
• Includes Introduce initial version 2 and stronger versioning support for the spec #384.
• Related to Consider alternatives to jsonschema for validation #287.
• Closes Introduce new package spec version  #345.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-08-15T19:55:47.403+0000

• Duration: 8 min 47 sec

Test stats 🧪

Test	Results
Failed	0
Passed	365
Skipped	0
Total	365

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (7/7)	💚
Files	66.667% (16/24)	❕
Classes	72.222% (26/36)	❕
Methods	57.282% (59/103)	❕
Lines	42.13% (530/1258)	❕
Conditionals	100.0% (0/0)	💚

[jsoriano]

Closing in favor of #384.