Fixes and additions to schema introspection, for comparative fuzzing (#…

…899)

* Align descriptions of built-ins with graphql-js. This makes comparating testing of introspection possible
* includeDeprecated argument is nullable
* Introspection: serialize defaultValue without indentation. Align with graphql-js
* Introspection: that field is called specifiedByURL, not specifiedBy
* Introspection: __Type.possibleTypes only includes object types. If an interface type implements another interface, it is not listed here
* Convenience for introspection fuzzing
* Introspection: as a precaution, validate unconditionally newly crafted executable documents, rather than only with debug assertions

crates/apollo-compiler/src/built_in_types.graphql

@@ -1,12 +1,19 @@
+"A GraphQL Schema defines the capabilities of a GraphQL server. It exposes all available types and directives on the server, as well as the entry points for query, mutation, and subscription operations."
 type __Schema {
   description: String
+  "A list of all types supported by this server."
   types: [__Type!]!
+  "The type that query operations will be rooted at."
   queryType: __Type!
+  "If this server supports mutation, the type that mutation operations will be rooted at."
   mutationType: __Type
+  "If this server support subscription, the type that subscription operations will be rooted at."
   subscriptionType: __Type
+  "A list of all directives supported by this server."
   directives: [__Directive!]!
 }
 
+"The fundamental unit of any GraphQL Schema is the type. There are many kinds of types in GraphQL as represented by the `__TypeKind` enum.\n\nDepending on the kind of a type, certain fields describe information about that type. Scalar types provide no information beyond a name, description and optional `specifiedByURL`, while Enum types provide their values. Object and Interface types provide the fields they describe. Abstract types, Union and Interface, provide the Object types possible at runtime. List and NonNull types compose other types."
 type __Type {
   kind: __TypeKind!
   name: String
@@ -27,17 +34,27 @@ type __Type {
   specifiedByURL: String
 }
 
+"An enum describing what kind of type a given `__Type` is."
 enum __TypeKind {
+  "Indicates this type is a scalar."
   SCALAR
+  "Indicates this type is an object. `fields` and `interfaces` are valid fields."
   OBJECT
+  "Indicates this type is an interface. `fields`, `interfaces`, and `possibleTypes` are valid fields."
   INTERFACE
+  "Indicates this type is a union. `possibleTypes` is a valid field."
   UNION
+  "Indicates this type is an enum. `enumValues` is a valid field."
   ENUM
+  "Indicates this type is an input object. `inputFields` is a valid field."
   INPUT_OBJECT
+  "Indicates this type is a list. `ofType` is a valid field."
   LIST
+  "Indicates this type is a non-null. `ofType` is a valid field."
   NON_NULL
 }
 
+"Object and Interface types are described by a list of Fields, each of which has a name, potentially a list of arguments, and a return type."
 type __Field {
   name: String!
   description: String
@@ -47,22 +64,26 @@ type __Field {
   deprecationReason: String
 }
 
+"Arguments provided to Fields or Directives and the input fields of an InputObject are represented as Input Values which describe their type and optionally a default value."
 type __InputValue {
   name: String!
   description: String
   type: __Type!
+  "A GraphQL-formatted string representing the default value for this input value."
   defaultValue: String
   isDeprecated: Boolean!
   deprecationReason: String
 }
 
+"One possible value for a given Enum. Enum values are unique values, not a placeholder for a string or numeric value. However an Enum value is returned in a JSON response as a string."
 type __EnumValue {
   name: String!
   description: String
   isDeprecated: Boolean!
   deprecationReason: String
 }
 
+"A Directive provides a way to describe alternate runtime execution and type validation behavior in a GraphQL document.\n\nIn some cases, you need to provide options to alter GraphQL's execution behavior in ways field arguments will not suffice, such as conditionally including or skipping a field. Directives provide this by describing additional information to the executor."
 type __Directive {
   name: String!
   description: String
@@ -71,25 +92,45 @@ type __Directive {
   isRepeatable: Boolean!
 }
 
+"A Directive can be adjacent to many parts of the GraphQL language, a __DirectiveLocation describes one such possible adjacencies."
 enum __DirectiveLocation {
+  "Location adjacent to a query operation."
   QUERY
+  "Location adjacent to a mutation operation."
   MUTATION
+  "Location adjacent to a subscription operation."
   SUBSCRIPTION
+  "Location adjacent to a field."
   FIELD
+  "Location adjacent to a fragment definition."
   FRAGMENT_DEFINITION
+  "Location adjacent to a fragment spread."
   FRAGMENT_SPREAD
+  "Location adjacent to an inline fragment."
   INLINE_FRAGMENT
+  "Location adjacent to a variable definition."
   VARIABLE_DEFINITION
+  "Location adjacent to a schema definition."
   SCHEMA
+  "Location adjacent to a scalar definition."
   SCALAR
+  "Location adjacent to an object type definition."
   OBJECT
+  "Location adjacent to a field definition."
   FIELD_DEFINITION
+  "Location adjacent to an argument definition."
   ARGUMENT_DEFINITION
+  "Location adjacent to an interface definition."
   INTERFACE
+  "Location adjacent to a union definition."
   UNION
+  "Location adjacent to an enum definition."
   ENUM
+  "Location adjacent to an enum value definition."
   ENUM_VALUE
+  "Location adjacent to an input object type definition."
   INPUT_OBJECT
+  "Location adjacent to an input object field definition."
   INPUT_FIELD_DEFINITION
 }
 
@@ -108,47 +149,36 @@ directive @include(
 "Marks an element of a GraphQL schema as no longer supported."
 directive @deprecated(
   """
-  Explains why this element was deprecated, usually also including a
-  suggestion for how to access supported similar data. Formatted using
-  the Markdown syntax, as specified by
-  [CommonMark](https://commonmark.org/).
+  Explains why this element was deprecated, usually also including a suggestion for how to access supported similar data. Formatted using the Markdown syntax, as specified by [CommonMark](https://commonmark.org/).
   """
   reason: String = "No longer supported"
 ) on FIELD_DEFINITION | ARGUMENT_DEFINITION | INPUT_FIELD_DEFINITION | ENUM_VALUE
 
-"Exposes a URL that specifies the behaviour of this scalar."
+"Exposes a URL that specifies the behavior of this scalar."
 directive @specifiedBy(
-    "The URL that specifies the behaviour of this scalar."
+    "The URL that specifies the behavior of this scalar."
     url: String!
 ) on SCALAR
 
 """
-The `Int` scalar type represents non-fractional signed whole numeric values. Int
-can represent values between -(2^31) and 2^31 - 1.
+The `Int` scalar type represents non-fractional signed whole numeric values. Int can represent values between -(2^31) and 2^31 - 1.
 """
 scalar Int
 
 """
-The `Float` scalar type represents signed double-precision fractional values as
-specified by [IEEE 754](https://en.wikipedia.org/wiki/IEEE_floating_point).
+The `Float` scalar type represents signed double-precision fractional values as specified by [IEEE 754](https://en.wikipedia.org/wiki/IEEE_floating_point).
 """
 scalar Float
 
 """
-The `String` scalar type represents textual data, represented as UTF-8 character
-sequences. The String type is most often used by GraphQL to represent free-form
-human-readable text.
+The `String` scalar type represents textual data, represented as UTF-8 character sequences. The String type is most often used by GraphQL to represent free-form human-readable text.
 """
 scalar String
 
 "The `Boolean` scalar type represents `true` or `false`."
 scalar Boolean
 
 """
-The `ID` scalar type represents a unique identifier, often used to refetch an
-object or as key for a cache. The ID type appears in a JSON response as a
-String; however, it is not intended to be human-readable. When expected as an
-input type, any string (such as `\"4\"`) or integer (such as `4`) input value
-will be accepted as an ID.
+The `ID` scalar type represents a unique identifier, often used to refetch an object or as key for a cache. The ID type appears in a JSON response as a String; however, it is not intended to be human-readable. When expected as an input type, any string (such as `"4"`) or integer (such as `4`) input value will be accepted as an ID.
 """
 scalar ID

crates/apollo-compiler/src/execution/introspection_execute.rs

@@ -22,6 +22,7 @@ use std::sync::OnceLock;
 /// Obtained from [`SchemaIntrospectionSplit::split`].
 ///
 /// [schema introspection]: https://spec.graphql.org/October2021/#sec-Schema-Introspection
+#[derive(Clone, Debug)]
 pub struct SchemaIntrospectionQuery(pub(crate) Valid<ExecutableDocument>);
 
 impl std::ops::Deref for SchemaIntrospectionQuery {
@@ -135,7 +136,7 @@ impl<'a> SchemaWithCache<'a> {
             .get_or_init(|| self.schema.implementers_map())
             .get(interface_name)
             .into_iter()
-            .flat_map(Implementers::iter)
+            .flat_map(|implementers| &implementers.objects)
     }
 }
 
@@ -301,7 +302,7 @@ impl_resolver! {
             schema::ExtendedType::Enum(_) |
             schema::ExtendedType::InputObject(_) => return Ok(ResolvedValue::null()),
         };
-        let include_deprecated = args["includeDeprecated"].as_bool().unwrap();
+        let include_deprecated = include_deprecated(args);
         Ok(ResolvedValue::list(fields
             .values()
             .filter(move |def| {
@@ -353,7 +354,7 @@ impl_resolver! {
         let schema::ExtendedType::Enum(def) = self_.def else {
             return Ok(ResolvedValue::null());
         };
-        let include_deprecated = args["includeDeprecated"].as_bool().unwrap();
+        let include_deprecated = include_deprecated(args);
         Ok(ResolvedValue::list(def
             .values
             .values()
@@ -370,7 +371,7 @@ impl_resolver! {
         let schema::ExtendedType::InputObject(def) = self_.def else {
             return Ok(ResolvedValue::null());
         };
-        let include_deprecated = args["includeDeprecated"].as_bool().unwrap();
+        let include_deprecated = include_deprecated(args);
         Ok(ResolvedValue::list(def
             .fields
             .values()
@@ -433,7 +434,7 @@ impl_resolver! {
     fn possibleTypes() { Ok(ResolvedValue::null()) }
     fn enumValues() { Ok(ResolvedValue::null()) }
     fn inputFields() { Ok(ResolvedValue::null()) }
-    fn specifiedBy() { Ok(ResolvedValue::null()) }
+    fn specifiedByURL() { Ok(ResolvedValue::null()) }
 }
 
 impl_resolver! {
@@ -450,7 +451,7 @@ impl_resolver! {
     }
 
     fn args(&self_, args) {
-        let include_deprecated = args["includeDeprecated"].as_bool().unwrap();
+        let include_deprecated = include_deprecated(args);
         Ok(ResolvedValue::list(self_
             .def
             .arguments
@@ -489,7 +490,7 @@ impl_resolver! {
     }
 
     fn args(&self_, args) {
-        let include_deprecated = args["includeDeprecated"].as_bool().unwrap();
+        let include_deprecated = include_deprecated(args);
         Ok(ResolvedValue::list(self_
             .def
             .arguments
@@ -556,7 +557,9 @@ impl_resolver! {
     }
 
     fn defaultValue(&self_) {
-        Ok(ResolvedValue::leaf(self_.def.default_value.as_ref().map(|val| val.to_string())))
+        Ok(ResolvedValue::leaf(self_.def.default_value.as_ref().map(|val| {
+            val.serialize().no_indent().to_string()
+        })))
     }
 
     fn isDeprecated(&self_) {
@@ -567,3 +570,12 @@ impl_resolver! {
         Ok(deprecation_reason(self_.def.directives.get("deprecated")))
     }
 }
+
+/// Although it should be non-null, the `includeDeprecated: Boolean = false` argument is nullable
+fn include_deprecated(args: &JsonMap) -> bool {
+    match &args["includeDeprecated"] {
+        serde_json_bytes::Value::Bool(b) => *b,
+        serde_json_bytes::Value::Null => false,
+        _ => unreachable!(),
+    }
+}

crates/apollo-compiler/src/execution/introspection_split.rs

@@ -26,6 +26,7 @@ use indexmap::map::Entry;
 /// Result of [`split`][Self::split]ting [schema introspection] fields from an operation.
 ///
 /// [schema introspection]: https://spec.graphql.org/October2021/#sec-Schema-Introspection
+#[derive(Clone, Debug)]
 pub enum SchemaIntrospectionSplit {
     /// The selected operation does *not* use [schema introspection] fields.
     /// It should be executed unchanged.
@@ -61,6 +62,7 @@ pub enum SchemaIntrospectionSplit {
     },
 }
 
+#[derive(Debug)]
 pub enum SchemaIntrospectionError {
     SuspectedValidationBug(SuspectedValidationBug),
     Unsupported {
@@ -237,13 +239,9 @@ fn make_single_operation_document(
         fragments,
     };
     new_document.operations.insert(new_operation);
-    if cfg!(debug_assertions) {
-        new_document
-            .validate(schema)
-            .expect("filtering a valid document should result in a valid document")
-    } else {
-        Valid::assume_valid(new_document)
-    }
+    new_document
+        .validate(schema)
+        .expect("filtering a valid document should result in a valid document")
 }
 
 fn get_fragment<'doc>(

crates/apollo-compiler/src/execution/resolver.rs

@@ -79,7 +79,10 @@ macro_rules! impl_resolver {
                         },
                     )*
                     _ => Err(crate::execution::resolver::ResolverError {
-                        message: format!("unexpected field name: {field_name}")
+                        message: format!(
+                            "unexpected field name: {field_name} in type {}",
+                            self.type_name()
+                        )
                     }),
                 }
             }

crates/apollo-compiler/src/execution/response.rs

@@ -7,7 +7,7 @@ use serde::Deserialize;
 use serde::Serialize;
 
 /// A [GraphQL response](https://spec.graphql.org/October2021/#sec-Response-Format)
-#[derive(Debug, Clone, Serialize, Deserialize)]
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
 #[serde(deny_unknown_fields)]
 pub struct Response {
     // <https://spec.graphql.org/October2021/#note-6f005> suggests serializing this first
@@ -26,7 +26,7 @@ pub struct Response {
 }
 
 /// The `data` entry of a [`Response`]
-#[derive(Debug, Clone, Deserialize)]
+#[derive(Debug, Clone, PartialEq, Deserialize)]
 #[serde(from = "Option<JsonMap>")]
 pub enum ResponseData {
     /// Execution returned an object.
@@ -143,7 +143,7 @@ impl GraphQLError {
 
 impl ResponseData {
     /// For serde `skip_serializing_if`
-    fn is_absent(&self) -> bool {
+    pub fn is_absent(&self) -> bool {
         matches!(self, Self::Absent)
     }
 

crates/apollo-compiler/src/validation/mod.rs

@@ -360,6 +360,7 @@ impl DiagnosticData {
                 ExecutableBuildError::ConflictingFieldName(_) => "ConflictingFieldName",
                 ExecutableBuildError::ConflictingFieldArgument(_) => "ConflictingFieldArgument",
             }),
+            Details::RecursionLimitError => Some("RecursionLimitError"),
             _ => None,
         }
     }