Round Trip Content Conversion Not Run in GHA CI/CD

[aj-stein-nist]

Describe the bug

In the course of completing usnistgov/OSCAL#1222, review of the current CI/CD documentation indicates we did or currently do intend to run validate-content-conversion-round-trips.sh, either with run-all.sh as code or as wording suggests independently. The former is not implemented and the latter is currently commented out. Documentation should be updated or pipelines ought to be updated at a later date. This bug was opened to solicit feedback from project leads and development team.

Who is the bug affecting?

Developers who want to modify and test the consistency of round-tripped OSCAL document instances from any combination JSON, XML, and YAML.

What is affected by this bug?

Testing of round-tripping of content conversion.

When does this occur?

Consistently in our hosted CI/CD system, partially works with custom tweaks to shell scripting run-all.sh to restore back round-trip testing of content conversion.

How do we replicate the issue?

1a. Execute GitHub Actions through CI/CD and review output in debugging mode.
1b. Execute bash -x ./build/ci-cd/run-all.sh in local testing environment and observe debug output

Expected behavior (i.e. solution)

Updated documentation or updated pipelines actually testing round tripping functionality.

Other Comments

N/A

[aj-stein-nist]

I talked with Dave and Wendell, and they agree that full round-trip test integration in CI/CD with a neat, clean pass or fail result is complex at this time. Wendell made some good points re the complexities of round-tripping specific fields in JSON/XML/YAML, like any those with a Markdown-ish type of markup-multiline and different implementations. Re the recommended path forward, we agreed given the conversation:

• Leave the script for round-trip testing as-is, don't remove it in Modularize All GitHub Actions Setups OSCAL#1223, when wrapping up the embedded Review Error Behavior in OSCAL CI/CD Scripts and GitHub Actions Automation OSCAL#1222 wrap-up work.
• Update the documentation there to say we have that script but it does not run in CI/CD, the round-trip script, "run all" script, and others that don't.
• Table this and leave it in the backlog to be prioritized accordingly after the requisite further analysis needed is complete with follow-up from AJ.