Discrepancy between NIST OSCAL JSON structure and NIST OSCAL XML structure (outline) on POAM

[vitggsa]

Describe the bug

There is a discrepancy between NIST OSCAL JSON structure (https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/plan-of-action-and-milestones/json-outline/) and NIST OSCAL XML structure (https://pages.nist.gov/OSCAL-Reference/models/v1.1.1/plan-of-action-and-milestones/xml-outline/).

Refer to fedramp-automation ticket for more details: https://github.com/GSA/fedramp-automation/issues/461

Who is the bug affecting

See fedramp-automation ticket for more details: https://github.com/GSA/fedramp-automation/issues/461

What is affected by this bug

Documentation, Metaschema, Website

How do we replicate this issue

Compare links supplied above for comparison between JSON outline and details and XML outline details.

Expected behavior (i.e. solution)

Update outlines or provide clarification on difference.

Other comments

No response

Revisions

No response

[Compton-US]

Note for team - see:

OSCAL/src/metaschema/oscal_assessment-common_metaschema.xml

Lines 1199 to 1201 in f24dd56

	<assembly ref="response" min-occurs="0" max-occurs="unbounded">
	<group-as name="remediations" in-json="ARRAY"/>
	</assembly>

[iMichaela]

It has been discussed in issue #1618 . On Feb 28, NIST OSCAL team issued a solicitation to the community requesting feedback on the issue since the fix would be backwards incompatible.
Here is a copy of the solicitation:
::::::::::::::::::::::::::::::::::::::::
Hello OSCAL Community,

This is a public notice to solicit community feedback for the next 30 days regarding a potential change to the OSCAL Plan of Actions and Milestones JSON Model. The change will the JSON model similar to the OSCAL Plan of Actions and Milestones XML Model. In the JSON model, an element of risk is aliased as remediation, whereas in the POA&M XML model it is response. More details can be found in the GitHub issue below.

#1618

We have determined a change will constitute a break in backwards compatibility. It is the intent of the NIST OSCAL Team to not schedule this change (to alter the JSON element from remediation to response to match the XML equivalent) for the near-term, unless community feedback leads us to review prioritization after 30 days.

If you are a community member, you can provide feedback described in the method below. Primarily, we ask you #1618 (comment).

• If you agree and do not consider this change urgent for you or your collaborators, please select the "thumbs up" (+1) as a reaction to this comment.
• If you disagree and do consider this change urgent for you or your collaborators, please select the "thumbs down" (-1) as a reaction to this comment.
• Add a comment on the issue to provide feedback or ask a question that will assist you in making a decision before the 30 day deadline.

You can provide feedback via alternate methods not listed above. We will review them, but we strongly prefer the method above and only consider the alternative methods on a case-by-case basis.

On 31 March 2023, we will close the feedback window, determine next steps, and apprise the community any change in prioritization.

Regards,
The NIST OSCAL Team
::::::::::::::::::::::::::::::::::::::

RESULTS FROM THE COMMUNITY: ONE VOTE DOWN (aka CHANGE IT)., with the following justification from @vmangat

::::::::::::::::::::::::::::::::::::::
We recommend getting this change implemented sooner.
This also impacts the risk assembly in SAR. This will also impact the NIST Conversion tool JSON -> XML

While most of the activity in the past few months has been on generating OSCAL SSPs, the coming months will see a pickup on POAMs being converted to OSCAL.. As the population of these files increases. particularly with a lot of JSON x XML conversions that will take place, it will benefit to have consistency.

Also #1120 added an assembly to the POAM document which is NOT breaking backward compatibility, currently scheduled for 1.1.0

We request that both changes be included in 1.0.5 which will ensure the POAM conversions to OSCAL that will take place in the next few months are good and clean. apologies for mixing issues.
::::::::::::::::::::::::::::::::::::

Issue #1618 was closed since the community survey ended. A related (duplicate) issue (#1766) was opened in April by @rachkim00 and closed based on the #1618 resolution that NIST OSCAL team will further research a solution with minimum impact to the community. The proposed solution from @rachkim00 was to "to update the 'Remediation' with 'Response'." in the JSON schema. NIST team determined that such change will break the backwards compatibility for the JSON adopters/users, but as promised in the #1618 resolution, NIST team will inform the community of the chosen approach. The fix is simple (see @Compton-NIST note). The impact is significant for the JSON OSCAL community and will call for a major OSCAL release.

[iMichaela]

@vitggsa - Can you please summarize the impact to FedRAMP automation if we move to a major release to address the issue you created (e.g. would FedRAMP support OSCAL v1.1.1 and OSCAL v2.0.0, or will move to OSCAL v 2.0.0 only, etc.)?
Thank you.

[iMichaela]

NOTE TO TEAM: Assessment Results XML outline and JSON outline present the same discrepancy. AR and POAM will need to be addressed in tandem.

@vitggsa - is FedRAMP handling Assessment Results' (AR) XML vs JSON discrepancy programmatically today?

[vitggsa]

This issue was identified by one of our early adopter's workgroup participants. Not sure whether they saw it on AR also. Will follow-up with them on ticket. https://github.com/GSA/fedramp-automation/issues/461

[david-waltermire]

In the following I am representing the FedRAMP perspective, since I have recently started working with them part time.

Background

This bug affects implementers that are supporting both XML and JSON/YAML, for which they will need specific logic to deal with the difference. Implementers that support only XML or only JSON/YAML do not have an issue, other than the visual (and maybe semantic) difference in names, which can be clarified in documentation. The OSCAL CLI, which is Metaschema-based and is not affected by this bug, since it provides for seemless translation between both representations.

Impact to FedRAMP use of OSCAL

While this bug does cause some implementation challenge for implementers supporting both XML and JSON/YAML, FedRAMP believes that supporting both 1.x and 2.x creates an even more significant implementation challenge. This results from having to support multiple namespaces in parsers, split version logic in tools and user interfaces, and adjustments to how FedRAMP validations work across multiple major versions. This burden is much greater than a small change to address this issue or leaving the models as-is. One solution would be to force everyone to using 2.x, but that also causes significant disruption. This would not be a compelling reason to consider adopting a 2.0.0 at this time or in the next year or so.

Suggested Way Forward

Given the above, FedRAMP believes there are two options.

1. Leave the discrepancy as-is and document the discrepancy in OSCAL documentation. Tools supporting both XML and JSON/YAML will need to implement logic to address this difference. Many likely have already done so. The FedRAMP validations will be adjusted to expect this difference.
2. Release a bug fix as v1.2.0 that renames "remediations" to "responses" in JSON. While this does break compatibility with prior v1.x versions, the impact is localized to the AR and POAM models, which are not widely in use today. FedRAMP will adjust their validations to use the new names. Package submissions supporting the prior approach will not be considered valid. OSCAL adoption is still early and this type of bug fix will probably cause only a small amount of disruption.

We plan to raise this issue with the FedRAMP early adopters community to collect input. I'll post a follow on with a summary of this feedback and a final recommendation from FedRAMP on a way forward. Would you please hold any action on this issue until then?

[Compton-US]

Ultimately the decision here is whether it is changed, and if so, when does that change need to occur.

On version numbering:

Since we utilize semantic versioning, a breaking change requires a major increment. This is only a signal to implementers and does not imply a significant feature shift in OSCAL, or increasing level of effort. ("major" vs "minor")

Major version X (X.y.z | X > 0) MUST be incremented if any backward incompatible changes are introduced to the public API. It MAY also include minor and patch level changes. Patch and minor versions MUST be reset to 0 when major version is incremented. - https://semver.org/

We should not place such an emphasis on the major version that we cannot increment it to communicate as intended by semantic versioning. For anyone not paying close attention to these development discussions, it will not be clear if we only increment the minor, particularly over time. This will break trust, and we cannot quantify the impact this may have across the community. Whether it is 1.2.0 or 2.0.0, it is the same change, with the same problem to correct when upgrading or supporting multiple versions. Using 2.0.0 is very clear that something breaking changed, and that change can be highlighted in the release.

In this case, we also do not have to implement multiple breaking changes to make it difficult to move to the next version. Rather than focusing on the number, what I'm hearing is to limit what is broken, so that it is a minimal level of effort to make the adjustment.

[iMichaela]

@david-waltermire-nist - Thank you for your feedback. The background you provided is the reason for NIST OSCAL team not acting on this issue raised already 3 times. We will wait until you will provide the feedback form the FedRAMP early adopters. Can you please let us know when to expect an answer (approximately)?

When we polled the community in writing (issue #1618) or during the OSCAL meeting with the devs, we did not receive an overwhelming number of responses (disappointing, I know) but the answers received were pointing to an immediate fix to minimize the impacted community members which were not focusing yet on the assessment layer (FedRAMP included). We did not act on it because the community's voice was not strong enough (no FedRAMP until now, one CSPs, two GRCs, no 3PAO).

One day (sooner or later) we will need to move to OSCAL 2.x, and when this will happen, NIST will have to support, for a period of time, (OSCAL 1.x + bug fixes) and (OSCAL 2.x + bug fixes + new features), to facilitate a smooth transition. This effort is not negligible at our end either, so we will not make the decision without an internal impact analysis. The length of the time the versions will be maintained before sunsetting v 1.x will also need to be coordinated with the community members (FedRAMP and international community members included). Community members have also the option of locking their operations or support to OSCAL v1.x. and slowly transition during the period we will maintain both versions. OSCAL JSON content can be migrated with a wound trip conversion: JSON-> XML with a current converter (XSLT or oscal-cli) and back XML-> JSON with an OSCAL v2.x converter.

[iMichaela]

11/06/2023: FedRAMP closed today the related issue #461 with the note "completed".

[david-waltermire]

See the comment in GSA/fedramp-automation#461. Based on discussion with the FedRAMP community, FedRAMP recommends that this issue be closed with no further action on this issue until a substantive 2.0 release.