Refactored core model classes and support for collections of model in… (

#256)

# Overall

- Relocated many class files and cleaned up many unneeded methods.
- Improved some Javadocs.
- Fixed many Spotbugs, PMD, and Java errors and warnings.
- Performed some code cleanup and formatting.

# Model Refactoring

Refactored core model classes and support for collections of model instances.

- Refactored instance class hierarchy to support grouped and ungrouped model instances, with minimal implementation.
  - Renamed model interfaces for greater consistency.
  - Refactored core model interfaces to reduce inheritance conflicts.
  - Adjusted the module interfaces to promote a cleaner, non-parameterized API.
- Renamed many instance implementation classes to ensure consistent use of terminology. Prior the terms "property" and "instance" were used interchangeably, which could cause some confusion.
- Refactored IBindingContext to move supporting interfaces to be child interfaces.
  - Removed the need for instances of IDataTypeHandler. Now IClassDataTypeHandler functionality is provided by IFeatureComplexItemValueHandler and JavaTypeAdapterDataTypeHandler is provided by IFeatureScalarItemValueHandler.
  - Refactored bound class hierarchy to be more consistent with needed methods.
  - Moved Java field handling to IFeatureJavaField from IBoundNamedModelInstance
  - Moved JSON key and item handling to IBoundModelInstance.
  - Reworked XML and JSON reading and writing.
  - Added IBoundChoiceGroupInstance interface for choice groups.
- Cleaned up IModelPropertyInfo and implementations, relocating supporting classes as child classes.
  - Implemented a IModelInstance item parsing as IModelPropertyInfo.IReadHandler, allowing all item parsing to be implemented in the format-specific parsing classes, i.e. MetaschemaJsonReader, MetaschemaXmlReader.
  - Cleaned up IModelPropertyInfo readItems methods, which now return the parsed value. Eliminated the need for IPropertyCollector instances.
- Refactored constraint ILet statement construction to use static factory methods.
- Cleaned up JSON value key naming code.
- Fixed default value handling which wasn't producing empty collection for unparsed model instances. Cleaned up getDefaultValue and getEffectiveDefaultValue methods.
- Refactored JSON key handling to determine the JSON key on a per item basis. This is needed for ChoiceGroups.
- Refactored container handling code to reuse the core model implementation where possible.
  - Refactored group-as namespace handling to allow for no namespace situations. Implemented getEffectiveGroupAsNamespace to handle the default case of using the module namespace where no group-as support is provided.
- Added support for a model(node) Metapath function that gets a nodeset based on the underlying Metaschema module model for the node.

# Binding Support

- Refactored XML and JSON item writing to use a shared IModelInstanceCollectionInfo.IWriteHandler implementation, allowing all item writing to be implemented in the format-specific classes, i.e. MetaschemaJsonWriter, MetaschemaXmlWriter.
- Implemented choice group writing for XML, JSON, and YAML.
  - Moved JSON reading and writing code from IItemValueHandler to MetaschemaJsonReader and MetaschemaJsonWriter using dispatch methods. This allows are IO code to be implemented withing the JSON format-specific implementation.
  - Moved XML reading and writing code from IItemValueHandler to MetaschemaXmlReader and MetaschemaXmlWriter using dispatch methods. This allows are IO code to be implemented withing the XML format-specific implementation.
  - Cleaned up XML parsing code. Fixed a bunch of bugs causing parsing to not work.
    - Fixed all identified bugs in XML parsing.
    - Added support for parsing XML fields as a root element.
- Cleaned up IBoundGroupedFieldInstance implementations to use only the complex form, since a grouped item must have a bound class to determine its item type.
- Aligned generated annotations with class generation, adding support for many missing annotation entries.
- Added binding support for choice groups.
- Added Metaschema module bindings.
- Adjusted the name used during code generation to use the value key name if available. This provides a more sensible name for Java property get and set methods created during code generation.
- Added support for deprecated on allowed value enumerations.
- Added support for Metaschema module loading that uses the Metaschema module binding instead of the XML parser. Metaschema modules in XML, JSON, and YAML can now be fully read!
  - Added Metaschema module loader support for entity resolution. Tested with OSCAL Metaschema modules.
  - Cleaned up bound module loading in DefaultBindingContext.
  - Relocated the IModulePostProcessor interface to the new IModuleLoader interface.
  - Refactored flag and field name handling for a cleaner implementation. This new approach uses "name" as the binding annotation value for flags, instead of "useName".
- Improved testing of assemblies with various types of fields and flags.
  - Implemented new unit tests to test the model bindings.
- Refactored DefaultJsonDeserializer to read a root element without the need to produce a Metapath item first.
- Adjusted the class generation for grouped instances to allow for extended classes where definitions are used by reference.
- Implemented subclass generation for choice group instances.
- Moved Metaschema module binding classes to a subdirectory to help with change integration.
- Refactored JSON schema generator. Many improvements to the indexing approach used to determine which definitions become global definitions in the JSON schema.
  - Improved JSON documentation to include titles and descriptions on properties.
- JSON and XML schema generation for choice groups is working.

core/pom.xml

@@ -145,8 +145,7 @@
     <build>
         <resources>
             <resource>
-                <directory>
-                    ${project.build.directory}/generated-resources/xmlbeans</directory>
+                <directory>${project.build.directory}/generated-resources/xmlbeans</directory>
             </resource>
             <resource>
                 <directory>${project.build.directory}/generated-resources/schema</directory>
@@ -296,8 +295,7 @@
                             <goal>copy-resources</goal>
                         </goals>
                         <configuration>
-                            <outputDirectory>
-                                target/generated-resources/schema/schema</outputDirectory>
+                            <outputDirectory>target/generated-resources/schema/schema</outputDirectory>
                             <resources>
                                 <resource>
                                     <directory>metaschema/schema</directory>

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/IDataTypeAdapter.java

@@ -191,7 +191,6 @@ default boolean isAtomic() {
    * @return {@code true} if the adapter will parse the element, or {@code false}
    *         otherwise
    */
-  // TODO: implement this
   boolean canHandleQName(@NonNull QName nextElementQName);
 
   /**

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/adapter/DateFormats.java

@@ -30,7 +30,11 @@
 import java.time.format.DateTimeFormatterBuilder;
 import java.time.temporal.ChronoField;
 
-public final class DateFormats {
+/**
+ * Provides common date formats used by the data type implementations.
+ */
+@SuppressWarnings("PMD.DataClass")
+final class DateFormats {
 
   // public static final DateTimeFormatter dateWithOptionalTZ;
   public static final DateTimeFormatter DATE_WITH_TZ;

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/adapter/MetaschemaDataTypeProvider.java

@@ -34,9 +34,10 @@
 import edu.umd.cs.findbugs.annotations.NonNull;
 
 /**
- * Provides for runtime discovery of built-in implementations of the core
- * Metaschema data types.
+ * Provides runtime discovery of built-in implementations of the core Metaschema
+ * data types.
  */
+@SuppressWarnings("PMD.CouplingBetweenObjects")
 @AutoService(IDataTypeProvider.class)
 public final class MetaschemaDataTypeProvider // NOPMD - Used for service initialization
     extends AbstractDataTypeProvider {

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/markup/AbstractMarkupString.java

@@ -72,6 +72,7 @@
 import edu.umd.cs.findbugs.annotations.NonNull;
 import edu.umd.cs.findbugs.annotations.Nullable;
 
+@SuppressWarnings("PMD.CouplingBetweenObjects")
 public abstract class AbstractMarkupString<TYPE extends AbstractMarkupString<TYPE>>
     implements IMarkupString<TYPE> {
   private static final Logger LOGGER = LogManager.getLogger(FlexmarkFactory.class);
@@ -88,7 +89,13 @@ public abstract class AbstractMarkupString<TYPE extends AbstractMarkupString<TYP
   @NonNull
   private final Document document;
 
-  public AbstractMarkupString(@NonNull Document document) {
+  /**
+   * Construct a new markup string based on the provided flexmark AST graph.
+   *
+   * @param document
+   *          the AST graph representing Markdown text
+   */
+  protected AbstractMarkupString(@NonNull Document document) {
     this.document = document;
   }
 
@@ -97,44 +104,25 @@ public Document getDocument() {
     return document;
   }
 
-  // @Override
-  // public void writeHtml(@NonNull XMLStreamWriter2 xmlStreamWriter, @NonNull
-  // String namespace)
-  // throws XMLStreamException {
-  //
-  //
-  // IMarkupString<?> markupString = (IMarkupString<>)value;
-  //
-  // MarkupXmlStreamWriter writingVisitor
-  // = new MarkupXmlStreamWriter(namespace, markupString.isBlock());
-  // writingVisitor.visitChildren(getDocument(), xmlStreamWriter);
-  // xmlStreamWriter.flush();
-  // }
-  //
-  // @Override
-  // public void writeHtml(@NonNull OutputStream os, @Nullable String namespace,
-  // @Nullable String
-  // prefix)
-  // throws XMLStreamException {
-  // XMLOutputFactory2 factory = (XMLOutputFactory2)
-  // XMLOutputFactory.newInstance();
-  // assert factory instanceof WstxOutputFactory;
-  // factory.setProperty(WstxOutputProperties.P_OUTPUT_VALIDATE_STRUCTURE, false);
-  // XMLStreamWriter2 xmlStreamWriter = (XMLStreamWriter2)
-  // factory.createXMLStreamWriter(os);
-  //
-  // String effectiveNamespace = namespace == null ? DEFAULT_HTML_NS : namespace;
-  // String effectivePrefix = prefix == null ? DEFAULT_HTML_PREFIX : prefix;
-  // NamespaceContext nsContext =
-  // MergedNsContext.construct(xmlStreamWriter.getNamespaceContext(),
-  // List.of(NamespaceEventImpl.constructNamespace(null, effectivePrefix,
-  // effectiveNamespace)));
-  // xmlStreamWriter.setNamespaceContext(nsContext);
-  //
-  //
-  // writeHtml(xmlStreamWriter, effectiveNamespace);
-  // }
+  @Override
+  public boolean isEmpty() {
+    return getDocument().getFirstChild() == null;
+  }
 
+  /**
+   * Parse HTML-based text into markdown as a flexmark AST graph.
+   * <p>
+   * This method uses a two-step approach that first translates the HTML into
+   * markdown, and then parses the markdown into an AST graph.
+   *
+   * @param html
+   *          the HTML text to parse
+   * @param htmlParser
+   *          the HTML parser used to produce markdown
+   * @param markdownParser
+   *          the markdown parser
+   * @return the markdown AST graph
+   */
   @NonNull
   protected static Document parseHtml(@NonNull String html, @NonNull FlexmarkHtmlConverter htmlParser,
       @NonNull Parser markdownParser) {
@@ -171,6 +159,15 @@ private boolean isTag(@Nullable org.jsoup.nodes.Node node, @NonNull String tagNa
     return parseMarkdown(markdown, markdownParser);
   }
 
+  /**
+   * Parse markdown-based text into a flexmark AST graph.
+   *
+   * @param markdown
+   *          the markdown text to parse
+   * @param parser
+   *          the markdown parser
+   * @return the markdown AST graph
+   */
   @SuppressWarnings("null")
   @NonNull
   protected static Document parseMarkdown(@NonNull String markdown, @NonNull Parser parser) {

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/markup/IMarkupString.java

@@ -55,6 +55,8 @@ public interface IMarkupString<TYPE extends IMarkupString<TYPE>>
   @NonNull
   Document getDocument();
 
+  boolean isEmpty();
+
   // /**
   // * Write HTML content to the provided {@code xmlStreamWriter} using the
   // provided {@code

core/src/main/java/gov/nist/secauto/metaschema/core/datatype/markup/flexmark/FlexmarkFactory.java

@@ -35,6 +35,7 @@
 
 import edu.umd.cs.findbugs.annotations.NonNull;
 
+@SuppressWarnings("PMD.DataClass")
 public class FlexmarkFactory {
   @NonNull
   private static final FlexmarkFactory SINGLETON = new FlexmarkFactory();
@@ -62,7 +63,7 @@ public FlexmarkFactory() {
   }
 
   @SuppressWarnings("null")
-  public FlexmarkFactory(@NonNull DataHolder config) {
+  public FlexmarkFactory(@SuppressWarnings("exports") @NonNull DataHolder config) {
     this.configuration = config;
     this.markdownParser = Parser.builder(config)
         .customDelimiterProcessor(new FixedEmphasisDelimiterProcessor(Parser.STRONG_WRAPS_EMPHASIS.get(config)))
@@ -74,7 +75,7 @@ public FlexmarkFactory(@NonNull DataHolder config) {
   }
 
   @NonNull
-  public DataHolder getConfiguration() {
+  protected DataHolder getConfiguration() {
     return configuration;
   }
 
@@ -98,6 +99,7 @@ public Formatter getFormatter() {
     return formatter;
   }
 
+  @SuppressWarnings("exports")
   @NonNull
   public FlexmarkHtmlConverter getFlexmarkHtmlConverter() {
     return htmlConverter;

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/ISequence.java

@@ -301,62 +301,62 @@ default <T> T[] toArray(T[] array) {
 
   @Override
   default boolean add(ITEM_TYPE item) {
-    throw new UnsupportedOperationException();
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default boolean remove(Object obj) {
-    throw new UnsupportedOperationException();
+  default void add(int index, ITEM_TYPE element) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default boolean containsAll(Collection<?> collection) {
-    return asList().containsAll(collection);
+  default boolean addAll(Collection<? extends ITEM_TYPE> collection) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default boolean addAll(Collection<? extends ITEM_TYPE> collection) {
-    throw new UnsupportedOperationException();
+  default boolean addAll(int index, Collection<? extends ITEM_TYPE> collection) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default boolean removeAll(Collection<?> collection) {
-    throw new UnsupportedOperationException();
+  default boolean remove(Object obj) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default boolean retainAll(Collection<?> collection) {
-    throw new UnsupportedOperationException();
+  default ITEM_TYPE remove(int index) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default void clear() {
-    throw new UnsupportedOperationException();
+  default boolean containsAll(Collection<?> collection) {
+    return asList().containsAll(collection);
   }
 
   @Override
-  default boolean addAll(int index, Collection<? extends ITEM_TYPE> c) {
-    throw new UnsupportedOperationException();
+  default boolean removeAll(Collection<?> collection) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default ITEM_TYPE get(int index) {
-    return asList().get(index);
+  default boolean retainAll(Collection<?> collection) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default ITEM_TYPE set(int index, ITEM_TYPE element) {
-    throw new UnsupportedOperationException();
+  default void clear() {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override
-  default void add(int index, ITEM_TYPE element) {
-    throw new UnsupportedOperationException();
+  default ITEM_TYPE get(int index) {
+    return asList().get(index);
   }
 
   @Override
-  default ITEM_TYPE remove(int index) {
-    throw new UnsupportedOperationException();
+  default ITEM_TYPE set(int index, ITEM_TYPE element) {
+    throw new UnsupportedOperationException("object is immutable");
   }
 
   @Override

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/MetapathConstants.java

@@ -37,6 +37,7 @@
 /**
  * Provides constant values used in Metapath.
  */
+@SuppressWarnings("PMD.DataClass")
 public final class MetapathConstants {
   @NonNull
   public static final URI NS_METAPATH = ObjectUtils.requireNonNull(
@@ -51,7 +52,7 @@ public final class MetapathConstants {
   public static final URI NS_METAPATH_FUNCTIONS_MATH = ObjectUtils.requireNonNull(
       URI.create("http://csrc.nist.gov/ns/metaschema/metapath-functions/math"));
   @NonNull
-  public static final URI NS_METAPATH_FUNCTIONS_EXTENDED = NS_METAPATH_FUNCTIONS_MATH;
+  public static final URI NS_METAPATH_FUNCTIONS_EXTENDED = NS_METAPATH;
 
   @NonNull
   public static final String PREFIX_METAPATH = "mp";

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/MetapathExpression.java

@@ -63,6 +63,9 @@
 /**
  * Supports compiling and executing Metapath expressions.
  */
+@SuppressWarnings({
+    "PMD.CouplingBetweenObjects" // necessary since this class aggregates functionality
+})
 public class MetapathExpression {
 
   public enum ResultType {

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/StaticContext.java

@@ -179,7 +179,7 @@ public DynamicContext dynamicContext() {
   /**
    * A builder used to generate the static context.
    */
-  public static class Builder {
+  public static final class Builder {
     private URI baseUri;
     @NonNull
     private final Map<String, URI> namespaces = new ConcurrentHashMap<>();

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/cst/BuildCSTVisitor.java

@@ -109,7 +109,10 @@
  * <a href="https://www.antlr.org/">ANTLRv4</a> into a compact syntax tree
  * (CST).
  */
-@SuppressWarnings({ "PMD.GodClass", "PMD.CyclomaticComplexity" }) // acceptable complexity
+@SuppressWarnings({
+    "PMD.GodClass", "PMD.CyclomaticComplexity", // acceptable complexity
+    "PMD.CouplingBetweenObjects" // needed
+})
 public class BuildCSTVisitor
     extends AbstractCSTVisitorBase {
 

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/cst/CSTPrinter.java

@@ -64,7 +64,7 @@ public static String toString(@NonNull IExpression expr) {
     return new CSTPrinterVisitor().visit(expr);
   }
 
-  private static class CSTPrinterVisitor
+  private static final class CSTPrinterVisitor
       extends AbstractExpressionVisitor<String, State> {
 
     @Override

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/cst/DecimalLiteral.java

@@ -57,7 +57,7 @@ public <RESULT, CONTEXT> RESULT accept(IExpressionVisitor<RESULT, CONTEXT> visit
     return visitor.visitDecimalLiteral(this, context);
   }
 
-  // REFACTOR: store decimal item value
+  // REFACTOR: store decimal item value as a field of this class
   @Override
   public ISequence<? extends IDecimalItem> accept(DynamicContext dynamicContext, ISequence<?> focus) {
     return ISequence.of(IDecimalItem.valueOf(getValue()));

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/cst/Quantified.java

@@ -263,7 +263,7 @@ public Iterator<List<T>> iterator() {
       if (size == 0) {
         return Collections.emptyListIterator();
       }
-      return new CartesianProductIterator<T>(dimensions);
+      return new CartesianProductIterator<>(dimensions);
     }
 
     // /**

core/src/main/java/gov/nist/secauto/metaschema/core/metapath/format/IPathFormatter.java

@@ -26,6 +26,7 @@
 
 package gov.nist.secauto.metaschema.core.metapath.format;
 
+import gov.nist.secauto.metaschema.core.metapath.item.node.IAssemblyInstanceGroupedNodeItem;
 import gov.nist.secauto.metaschema.core.metapath.item.node.IAssemblyNodeItem;
 import gov.nist.secauto.metaschema.core.metapath.item.node.IDocumentNodeItem;
 import gov.nist.secauto.metaschema.core.metapath.item.node.IFieldNodeItem;
@@ -103,6 +104,17 @@ default String format(@NonNull IPathSegment segment) {
   @NonNull
   String formatAssembly(@NonNull IAssemblyNodeItem assembly);
 
+  /**
+   * This visitor callback is used to format an individual grouped assembly path
+   * segment.
+   *
+   * @param assembly
+   *          the node to format
+   * @return the formatted text for the segment
+   */
+  @NonNull
+  String formatAssembly(@NonNull IAssemblyInstanceGroupedNodeItem assembly);
+
   /**
    * This visitor callback is used to format a root assembly path segment.
    *