Consistent & Clear Wording in Validations

[ohsh6o]

Extended Description

As a FedRAMP reviewer, to best understand if the validations I and CSP SSP authors will read actually reflect the intended guidance from a formal FedRAMP review, I want to ensure the current validation output messages are consistent and clear in how they communicate errors and important diagnostic information about the SSP.

Preconditions

• Access to example review reports (with SSP feedback) for one or more real FedRAMP CSP packages
• Complete this work, if possible, alongside Allow User to Customize Validation Feedback #166.

Acceptance Criteria

• For high-level validations that map to checklist items match wording as best as possible with real-world examples provided (terminology, phrasing, etc.)
• Create a shared document with a glossary so we can align on terms with the FART: add words or phrases we will define to ask them for periodic review

Story Tasks

• Review User Feedback and add tasks for specific items to fix
• For the reviewer persona, update the following wording in the messaging (@id can stay the same):
  • Generally for rule assertions on lead-in since we are talking prescriptive rules, change This SSP has to A FedRAMP SSP must to make it clear the aspirational nature of the rule.
  • base64 -> attachment file content
  • resource -> supporting artifact found in a citation
  • A FedRAMP SSP has defined a responsible party with no extraneous roles. -> Responsible Role in control does not correspond to any entry in Roles and Responsibilities Table.
  • A FedRAMP SSP must not include implemented controls beyond what is required for the applied baseline. ... A FedRAMP SSP has implemented X extraneous controls not needed given the selected profile.

More Examples
The following adopt a "business rule" style.

A FedRAMP SSP submission must include a Security Assessment Plan (SAP).
A FedRAMP SSP submission must include a Security Assessment Report (SAR).
A FedRAMP SSP submission must include Plans of Action and Milestones (POA&Ms).
A FedRAMP SSP submission must include a Continuous Monitoring Plan (ConMon Plan).
A FedRAMP SSP must use the correct FedRAMP SSP Template.
A FedRAMP SSP must use the correct FedRAMP Deployment Model.
Within a FedRAMP SSP, all controls have at least one implementation status checkbox selected.
Within a FedRAMP SSP, all critical controls are implemented.
Within a FedRAMP SSP, customer responsibilities are clearly identified in the CIS-CRM Tab, as well as the SSP Controls (by checkbox selected and in the implementation description). The CIS-CRM and SSP controls are consistent for customer responsibilities.
Within a FedRAMP SSP, the Initial Authorizing Agency concurs with the CRM (adequacy and clarity of customer responsibilities).
Within a FedRAMP SSP, the Roles Table (User Roles and Privileges) sufficiently describes the range of user roles, responsibilities, and access privileges.
Within a FedRAMP SSP, in the control summary tables, the information in the Responsible Role row correctly describes the required entities responsible for fulfilling the control.
Within a FedRAMP SSP, the appropriate Digital Identity Level is selected.
Within a FedRAMP SSP, the authorization boundary is explicitly identified in the network diagram.
Within a FedRAMP SSP, does the CSO provide components to run on the client side?
Within a FedRAMP SSP, there is a data flow diagram that clearly illustrates the flow and protection of data going in and out of the service boundary and that includes all traffic flows for both internal and external users.
Within a FedRAMP SSP, does the CSP use any third-party or external cloud services that lack FedRAMP Authorization?
Within a FedRAMP SSP, if this is a SaaS or a PaaS, is it "leveraging" another IaaS with a FedRAMP Authorization?
Within a FedRAMP SSP, if 11a is Yes, the "inherited" controls are clearly identified in the control descriptions.
Within a FedRAMP SSP, all interconnections are correctly identified and documented in the SSP.
Within a FedRAMP SSP, all required controls are present.
Within a FedRAMP SSP, the inventory is provided in the FedRAMP Integrated Inventory Workbook.
Within a FedRAMP SSP, the CSO is compliant with DNSSEC. (Controls SC-20 and SC-21 apply).
Within a FedRAMP SSP, the CSO adequately employs Domain-based Message Authentication, Reporting & Conformance (DMARC) requirements according to DHS BOD 18-01.
A FedRAMP SSP must incorporate the FedRAMP Master Acronym and Glossary.
A FedRAMP SSP must incorporate the FedRAMP Applicable Laws and Regulations.
A FedRAMP SSP must incorporate the FedRAMP Logo.
A FedRAMP SSP must incorporate a User Guide.
A FedRAMP SSP must incorporate Rules of Behavior.
A FedRAMP SSP must incorporate a Contingency Plan.
A FedRAMP SSP must incorporate a Configuration Management Plan.
A FedRAMP SSP must incorporate an Incident Response Plan.
A FedRAMP SSP must incorporate a Separation of Duties Matrix.
A FedRAMP SSP must incorporate a policy document for each of the 17 NIST SP 800-54 Revision 4 control families.
A FedRAMP SSP must incorporate a procedure document for each of the 17 NIST SP 800-54 Revision 4 control families.
FedRAMP SSP policy and procedure documents must have unique per-control-family associations.
A FedRAMP SSP must incorporate a Privacy Point of Contact role.
A FedRAMP SSP must have all four PTA questions answered.
A FedRAMP SSP must incorporate one or more NIST CMVP-validated cryptographic modules (FIPS 140).
A FedRAMP SSP must specify a FIPS 199 categorization.
A FedRAMP SSP must specify one or more SP 800-60v2r1 information types.
A FedRAMP SSP must incorporate a Digital Identity Determination.
A FedRAMP SSP must incorporate a system inventory.
A FedRAMP SSP must have a FedRAMP system-id.
A FedRAMP SSP must have a system name.
A FedRAMP SSP must have a short system name.
A FedRAMP SSP must have a FedRAMP authorization type.
A FedRAMP SSP must identify the system owner.
A FedRAMP SSP must identify the authorizing official.
A FedRAMP SSP must identify the system management point of contact.
A FedRAMP SSP must identify the system technical point of contact.
A FedRAMP SSP must identify the system other point of contact.
A FedRAMP SSP must incorporate an authorization boundary diagram.
A FedRAMP SSP must incorporate a network-architecture diagram.
A FedRAMP SSP must incorporate a data-flow diagram.
A FedRAMP SSP employs a FedRAMP OSCAL profile.
Within a FedRAMP SSP, every required control must have an implementation status.
Within a FedRAMP SSP, planned control implementations must have a planned completion date.

Definition of Done

• Acceptance criteria met - Each user story should meet the acceptance criteria in the description
• Unit test coverage of our code > 90% (from QASP) this may be fuzzy and hard to prove
• Code quality checks passed - Enable html tidy with XML code standards as part of the build (from QASP)
  - [ ] Accessibility: (from QASP) as we create guidance or documentation and reports (semantic tagging including aria tags): demonstrate with 0 errors reported for WCAG 2.1 AA standards using an automated scanner and 0 errors reported in manual testing
• Code reviewed - Code reviewed by at least one other team members (or developed by a pair)
• Source code merged - Code that’s demoed must be in source control and merged
  - [ ] Code must successfully build and deploy into staging environment (from QASP): this may evolve from xslt sh pipline into something more
  ~~- [ ] Security reviewed and reported - Conduct vulnerability and compliance scanning. threat modeling? ~~
  - [ ] Code submitted must be free of medium- and high-level static and dynamic security vulnerabilities (from QASP)
  - [ ] Usability tests passed - Each user story should be easy to use by target users (development community? FedRAMP FART team)
  - [ ] Usability testing and other user research methods must be conducted at regular intervals throughout the development process (not just at the beginning or end). (from QASP)
• Code refactored for clarity - Code must be clean, self-documenting
• No local design debt
• Load/performance tests passed - test data needed - saxon instrumentation
• Documentation generated - update readme or contributing markdown as necessary.
  - [ ] Architectural Decision Record completed as necessary for significant design choices

[GaryGapinski]

Ready to merge, but awaiting any last-minute suggestions on message text. Will be closed by #193.