[EPIC] CI/CD Enhancements

[aj-stein-nist]

User Story:

As an OSCAL developer, in order to have a quick and seamless experience with development and testing of the official OSCAL repos' content and software, I would like enhancements to the OSCAL CI/CD systems to: simplify the build process with a single command, facilitate automated releases, and improve the modularity of different build steps.

Goals:

• New functionality to auto-increment versions for all OSCAL artifacts and automate as much as possible for steps to create a new release
  • Reach consensus or spike for discussion with team to apply that same tooling for new Metaschema releases as well
• Officially publish docker containers in DockerHub and/or GH registry before standardizing how we use it locally and/or in GHA (Build and Publish Docker Image for NIST and Community Developers #1008)
• Reach consensus and implement a tool to unify all build steps (i.e. make)
  • Potentially unify and clean up the different usages of XProc for local dev, but not for CI/CD pipelines for a lot of tooling that is in the hot-path of building artifacts in CD (Metaschema bootstrapping, OSCAL schema generation, content generation, et cetera)
• Update tooling, documentation, and maybe team workflow to use docker within GitHub Actions like we should be doing with docker-compose on our laptops; without this, we have two very similar but different execution pipelines (with shell scripts in local workstation testing but separate YAML declaration for the GitHub Actions job)
• Spike and determine the viability of packaging up the CI/CD functionality we enhance and publish reusable GitHub Actions for NIST and the larger community to use.

Dependencies:

None at the epic level.

[aj-stein-nist]

Also for consideration:

• Adding, changing, or removing how certain steps in GitHub Actions to increase performance the CI/CD process (i.e. uploader action steps take sometimes minutes for uploading artifacts).
• Exposing all dependency management manifests to ensure everything is checked and vulnerable packages reported (as part of Build and Publish Docker Image for NIST and Community Developers #1008)

[aj-stein-nist]

Interesting opportunity for improved tooling. Not part of the list yet, but for potential inclusion: #1183.

[aj-stein-nist]

Also, given discussion today with release ZIP archive JSON Schema files and source code pulls not working as planned, consider some integration-level tests around sanity-checking released contents and warning when they do not work as expected? Talk to Dave about this for future releases.

[aj-stein-nist]

@aj-stein-nist needs to review the list of work items in this epic for those that are obsolete, change and remove the applicable ones. Some still appear to have value and worth considering (specifically: how do we auto-increment models for release).