Skip to content

Commit

Permalink
Refactor model loading to allow equal conflicts
Browse files Browse the repository at this point in the history
When using Smithy features like model discovery on the classpath, it's
sometimes difficult to prevent the same Smithy models from being added
to the model assembler multiple times. This is also true when using
artifacts created for the "model" plugin for smithy-build along with
model discovery. To address this, Smithy has been updated to allow
conflicting shape definitions if the shapes, after they are fully built,
are equivalent (meaning, they have the same exact members or references to
other shapes and are the same type).
  • Loading branch information
mtdowling committed Aug 4, 2020
1 parent e5480ef commit f8287d4
Show file tree
Hide file tree
Showing 29 changed files with 1,626 additions and 1,305 deletions.
58 changes: 42 additions & 16 deletions docs/source/1.0/spec/core/model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -149,15 +149,31 @@ together to form a valid semantic model.
Merging model files
===================

Implementations MUST take the following steps to merge models together to load
the semantic model:
Implementations MUST take the following steps when merging two or more
model files to form a semantic model:

#. Merge the metadata objects of all model files using the steps defined in
:ref:`merging-metadata`.
#. Shapes defined in a single model file are added to the semantic model as-is.
#. Shapes with the same shape ID defined in multiple model files are
reconciled using the following rules:

#. All conflicting shapes MUST have the same shape type.
#. Conflicting :ref:`aggregate shapes <aggregate-types>` MUST contain the
same members that target the same shapes.
#. Conflicting :ref:`service shapes <service-types>` MUST contain the same
properties and target the same shapes.
#. Conflicting traits defined in shape definitions or through
:ref:`apply statements <apply-statements>` are reconciled using
:ref:`trait conflict resolution <trait-conflict-resolution>`.

#. Duplicate shape IDs, if found, MUST cause the model merge to fail.
See :ref:`shape-id-conflicts` for more information.
#. Merge any conflicting applied traits using
:ref:`trait conflict resolution <trait-conflict-resolution>`.
#. Merge the metadata objects of both models using the steps defined
in :ref:`merging-metadata`.
.. note::

*The following guidance is non-normative.* Because the Smithy IDL allows
forward references to shapes that have not yet been defined or shapes
that are defined in another model file, implementations likely need to
defer :ref:`resolving relative shape IDs <relative-shape-id>` to
absolute shape IDs until *all* model files are loaded.


.. _metadata:
Expand Down Expand Up @@ -485,6 +501,12 @@ To illustrate, ``com.Foo#baz`` and ``com.foo#BAZ`` are not allowed in the
same semantic model. This restriction also extends to member names:
``com.foo#Baz$bar`` and ``com.foo#Baz$BAR`` are in conflict.

.. seealso::

:ref:`merging-models` for information on how conflicting shape
definitions for the same shape ID are handled when assembling the
semantic model from multiple model files.


.. _simple-types:

Expand Down Expand Up @@ -2153,7 +2175,18 @@ immediately precede a shape. The following example applies the
* Refer to the :ref:`JSON AST specification <json-ast>` for a
description of how traits are applied in the JSON AST.

.. rubric:: Applying traits externally
.. rubric:: Scope of member traits

Traits that target :ref:`members <member>` apply only in the context of
the member shape and do not affect the shape targeted by the member. Traits
applied to a member supersede traits applied to the shape targeted by the
member and do not inherently conflict.


.. _apply-statements:

Applying traits externally
--------------------------

Both the IDL and JSON AST model representations allow traits to be applied
to shapes outside of a shape's definition. This is done using an
Expand Down Expand Up @@ -2200,13 +2233,6 @@ The following example applies the :ref:`documentation-trait` and
treated exactly the same as applying the trait inside of a shape
definition.

.. rubric:: Scope of member traits

Traits that target :ref:`members <member>` apply only in the context of
the member shape and do not affect the shape targeted by the member. Traits
applied to a member supersede traits applied to the shape targeted by the
member and do not inherently conflict.


.. _trait-conflict-resolution:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@
"target": "example.rest#Map"
},
"stringDateTime": {
"target": "smithy.api#StringDateTime"
"target": "smithy.example#StringDateTime"
}
}
},
Expand Down Expand Up @@ -208,7 +208,7 @@
"example.rest#Timestamp": {
"type": "timestamp"
},
"smithy.api#StringDateTime": {
"smithy.example#StringDateTime": {
"type": "timestamp",
"traits": {
"smithy.api#timestampFormat": "date-time"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,15 @@ private TraitCache getTraitCache() {
return cache;
}

/**
* Gets the immutable set of {@code ShapeId} in the model.
*
* @return Returns the shape IDs.
*/
public Set<ShapeId> getShapeIds() {
return shapeMap.keySet();
}

/**
* Gets a set of shapes in the model marked with a specific trait.
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
/*
* Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at
*
* http://aws.amazon.com/apache2.0
*
* or in the "license" file accompanying this file. This file is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
* permissions and limitations under the License.
*/

package software.amazon.smithy.model.loader;

import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import software.amazon.smithy.model.SourceException;
import software.amazon.smithy.model.node.Node;
import software.amazon.smithy.model.shapes.AbstractShapeBuilder;
import software.amazon.smithy.model.shapes.MemberShape;
import software.amazon.smithy.model.shapes.Shape;
import software.amazon.smithy.model.shapes.ShapeId;
import software.amazon.smithy.model.shapes.ShapeType;
import software.amazon.smithy.model.traits.Trait;
import software.amazon.smithy.model.traits.TraitFactory;
import software.amazon.smithy.model.validation.ValidationEvent;

/**
* Base class used for mutable model files.
*/
abstract class AbstractMutableModelFile implements ModelFile {

protected final TraitContainer traitContainer;

// A LinkedHashMap is used to maintain member order.
private final Map<ShapeId, AbstractShapeBuilder<?, ?>> shapes = new LinkedHashMap<>();
private final List<ValidationEvent> events = new ArrayList<>();
private final MetadataContainer metadata = new MetadataContainer(events);
private final TraitFactory traitFactory;

/**
* @param traitFactory Factory used to create traits when merging traits.
*/
AbstractMutableModelFile(TraitFactory traitFactory) {
this.traitFactory = Objects.requireNonNull(traitFactory, "traitFactory must not be null");
traitContainer = new TraitContainer.TraitHashMap(traitFactory, events);
}

/**
* Adds a shape to the ModelFile, checking for conflicts with other shapes.
*
* @param builder Shape builder to register.
*/
void onShape(AbstractShapeBuilder<?, ?> builder) {
if (shapes.containsKey(builder.getId())) {
AbstractShapeBuilder<?, ?> previous = shapes.get(builder.getId());
// Duplicate shapes in the same model file are not allowed.
ValidationEvent event = LoaderUtils.onShapeConflict(builder.getId(), builder.getSourceLocation(),
previous.getSourceLocation());
throw new SourceException(event.getMessage(), event.getSourceLocation());
}

shapes.put(builder.getId(), builder);
}

/**
* Adds metadata to be reported by the ModelFile.
*
* @param key Metadata key to set.
* @param value Metadata value to set.
*/
final void putMetadata(String key, Node value) {
metadata.putMetadata(key, value);
}

/**
* Invoked when a trait is to be reported by the ModelFile.
*
* @param target The shape the trait is applied to.
* @param trait The trait shape ID.
* @param value The node value of the trait.
*/
final void onTrait(ShapeId target, ShapeId trait, Node value) {
traitContainer.onTrait(target, trait, value);
}

/**
* Invoked when a trait is to be reported by the ModelFile.
*
* @param target The shape the trait is applied to.
* @param trait The trait to apply to the shape.
*/
final void onTrait(ShapeId target, Trait trait) {
traitContainer.onTrait(target, trait);
}

@Override
public final List<ValidationEvent> events() {
return events;
}

@Override
public final Map<String, Node> metadata() {
return metadata.getData();
}

@Override
public final Set<ShapeId> shapeIds() {
return shapes.keySet();
}

@Override
public final Collection<Shape> createShapes(TraitContainer resolvedTraits) {
List<Shape> resolved = new ArrayList<>(shapes.size());

// Build members and add them to top-level shapes.
for (AbstractShapeBuilder<?, ?> builder : shapes.values()) {
if (builder instanceof MemberShape.Builder) {
ShapeId id = builder.getId();
AbstractShapeBuilder<?, ?> container = shapes.get(id.withoutMember());
if (container == null) {
throw new RuntimeException("Container shape not found for member: " + id);
}
for (Trait trait : resolvedTraits.getTraitsForShape(id).values()) {
builder.addTrait(trait);
}
container.addMember((MemberShape) builder.build());
}
}

// Build top-level shapes.
for (AbstractShapeBuilder<?, ?> builder : shapes.values()) {
if (!(builder instanceof MemberShape.Builder)) {
// Try/catch since shapes could have problems building, like an invalid Shape ID.
try {
for (Trait trait : resolvedTraits.getTraitsForShape(builder.getId()).values()) {
builder.addTrait(trait);
}
resolved.add(builder.build());
} catch (SourceException e) {
events.add(ValidationEvent.fromSourceException(e).toBuilder()
.shapeId(builder.getId()).build());
}
}
}

return resolved;
}

@Override
public final ShapeType getShapeType(ShapeId id) {
return shapes.containsKey(id) ? shapes.get(id).getShapeType() : null;
}
}
Loading

0 comments on commit f8287d4

Please sign in to comment.