Completing and testing schema documentation production

[wendellpiez]

User Story:

On the Pages site, I can find reliable, complete and legible documentation on the tags (objects, elements) in the respective schemas.

Goals:

A documentation pipeline that produces HTML with internal links is underway; it produces a set of HTML pages suitable for ingestion by Jekyll. We need to finish this production, test it and integrate it into CI so that it runs whenever metaschemas change (warranting updates to their docs).

Dependencies:

This picks up #286. The current code was migrated from earlier code. A stable target is a dependency: now we have the Jekyll/uswds Pages site design.

The effort entails some design work (on the Pages site) as well as implementation; designing it may entail some scoping and definition of functional requirements for users of these resources.

Acceptance Criteria

All active metaschemas are documented on the Pages site, using a means that is sustainable over the long term (as schemas evolve).

Note that the completeness and accuracy of the documentation itself is not a criterion for acceptance.

[wendellpiez]

This is #286 already assigned to me.

[wendellpiez]

Sprint Progress Report March 18 2019

This work has surfaced a number of requirements for schema documentation now underway, including:

• Splitting XML and JSON documentation pages into two parallel structures
• Extending Jekyll navigation to support this parallel view
• Including cross-links (XML to JSON docs and back) at the object level
• Revising language on both sides to reflect readers' terminological expectations
• Extending to work with new modularity design (Develop OSCAL Modular Approach Strategy #316)
  • Documentation sourced from importing metaschemas should (be able to) supplement and/or override imported definitions/docs

Work can be found in my docs-redesign branch. A complication is that in order to complete this there is also a dependency on #327.

[wendellpiez]

Sprint Progress March 21 2019

Much progress here. Getting closer.

Finishing this Issue has a dependency on #327 -- the docs pipeline working with the new modularity model.

[iMichaela]

03/21/2019

@wendel needs to combine different things he did on different branches. @anweiss will help next week. metaschema works on one branch and docs on another branch and they have to come together first.

[howieavp76]

Latest version posted to S3:

http://nist-oscal.s3-website-us-east-1.amazonaws.com/

[iMichaela]

03/28/2019

Work is as completed as it can be at this time, but it has a dependency on #327 and #332.

[wendellpiez]

Progress April 4 2019

Work has been done including new feature to provide for amending documentation when models import definitions from other models. More testing would be useful. Suggest I make a PR for review?

Work is here: https://github.com/wendellpiez/OSCAL/tree/docs-redesign

[brian-ruf]

4/4/2019 - @wendellpiez has completed work, and needs to create a PR for review and acceptance. Also, see Issue #327. @anweiss to clean up navigation issues by COB 4/5.
Goal: Have final draft of site on S3 by COB 4/9 or 4/10, so team can review content before publishing to production by end of April.
Plan is to update Pages for now.
There is a risk that Pages will be unavailable in the near future. A backup plan is necessary long term.

[iMichaela]

04/11/2019
This work is in @anweiss ' PR #326 . This issue addresses the document production. We need a new issue for document editing and improving. @wendellpiez will create an issue for the next sprint, based on the list @brianrufgsa and @david-waltermire-nist worked. The punch list is in #317

[david-waltermire]

4-19-19

The development of the pipeline has been completed. @wendellpiez and @david-waltermire-nist will work on the CI integration today.

[iMichaela]

5/2/2019

Documentation production is stable. More documentation with examples should be written, but this is an ongoing process. Waiting for @david-waltermire-nist to review the documentation production and close it. @howieavp76 will push a preview of the site that has the docuemntation in it eventhough @anweiss did not accept yet @wendellpiez's PR.

[brian-ruf]

5/9/2019

We identified Metadata is appearing as a root element. @wendellpiez will address this.
@david-waltermire-nist will do a review for other bugs.

[wendellpiez]

I addressed the bug -- elements being incorrectly marked as "root" when they are roots of their modules, but not roots of the metaschema into which the module is called.

However in doing so I uncovered another issue. The schema documentation is not caught up with the metaschema processing logic now in master -- and while the adjustment is not difficult (and will be an improvement), it needs a file that is not yet in the development branch (for its 'pre-compiling' metaschema assembly logic).

Advice? I could borrow the file I need perhaps by rebasing one branch on the other. Or (less crazy?) we could complete this adjustment after the merge. Outputs (the schema documentation files) will look the same after this adjustment.

[wendellpiez]

Now fixed in my branch: https://github.com/wendellpiez/OSCAL/tree/docs-redesign

• Root elements/objects e.g. 'catalog' and 'profile' are now listed at the tops of their ToC panels
• Metaschema modularity now working properly (so elements are not erroneously reported as "root" when they are roots of their modules, not of the base metaschema)

[david-waltermire]

This was mostly addressed by PR #326. The CI/CD aspect is addressed by #274. Closing this issue.