Proposal: Support union scalar types

[stubailo]

It makes sense often to have a field that could return, for example, an Int or a String. It would be convenient to be able to use a resolveType-like mechanism to determine how the value is serialized. This would be even more useful for custom scalars.

The specification currently says: "GraphQL Unions represent an object that could be one of a list of GraphQL Object types", which basically means that unions are the same as interfaces, but without any shared fields. Supporting unions of scalars would further differentiate unions from interfaces.

[rmosolgo]

Could you share a couple of concrete use cases? I ask because, personally, I'm usually reading values right out of MySQL (or running those values through transformations with predictably-typed output), so I haven't needed this yet!

[migueloller]

@rmosolgo,

Here is a concrete use case that we've come across at my team. I have a couple more if you need them.

We've got an algorithm that produces results based on location. We have an enum AreaOfTown that contains all possible areas of town to filter the results. Here's the catch, the algorithm can also filter locations by "near me". Because we can't simply make a union that is AreaOfTown | "NEAR_ME" for the input type, then we have to create a completely new enum that contains all of the values in AreaOfTown and "NEAR_ME". This is undesirable because if we ever change the AreaOfTown enum by adding more (this will 100% happen in the future), then we have to make sure we also update our other enum.

If we could write scalar unions then it would solve this problem for us, among others (we would like to have constructs like Int | [Int!] or Int | String like @stubailo mentioned).

[stubailo]

In our case, it's because we were returning a structure that represented a JSON tree, where keys can be integers (in the case of arrays) or strings (in the case of objects). We ended up compromising for now to just use strings, but I can imagine how this would be extra useful in the case where you want to return one of a set of custom scalars, which might have different serialization logic.

[migueloller]

Here are other relevant discussions:

• GraphQLUnionType as input type graphql-js#207
• Union of Non-Object Types graphql-js#291
• Union input types #202

@leebyron made some good points in his comments here.

[marcintustin]

Allowing uniontypes to contain primitives allows us to model any kind of source that can be polymorphic in its source data. For example, avro data or elasticsearch.

This is a severe limitation on exposing existing data, rather than creating new systems that are defined by graphql from the ground up.

[leebyron]

What's still unclear to me is what a valid query would look like which encounters such a union, and how should well-typed clients determine what to expect from such a query?

Consider:

type MyType {
  field: String
}

union MyUnion = String | MyType

type Query {
  example: MyUnion
}

[marcintustin]

Like with any union - it has to be prepared for any item in the union. I don't see that this poses any special problems.

…

On Wed, Feb 8, 2017 at 7:52 PM Lee Byron ***@***.***> wrote: What's still unclear to me is what a valid query would look like which encounters such a union, and how should well-typed clients determine what to expect from such a query? Consider: type MyType { field: String } union MyUnion = String | MyType type Query { example: MyUnion } — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#215 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLqONw9c-XUFzYcBYSTtPU2GGBsr5J7ks5ramNMgaJpZM4KLagd> .

[leebyron]

For the above schema consider the query:

{ example }

It's clear what this query should return if the example field returns a String at runtime, but what should it return if the value was a MyType?

Similarly:

{ 
  example {
    ... on MyType {
      field
    }
  }
}

It's clear what this would return for MyType, but what should it return for a String?

[marcintustin]

The exact same thing as querying a value that doesn't exist on a member of a union.

…

On Wed, Feb 8, 2017 at 8:36 PM Lee Byron ***@***.***> wrote: For the above schema consider the query: { example } It's clear what this query should return if the example field returns a String at runtime, but what should it return if the value was a MyType? Similarly: { example { ... on MyType { field } } } It's clear what this would return for MyType, but what should it return for a String? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#215 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABLqONSR3whAYxVVrPBjdCmGOT_Py57tks5ram2rgaJpZM4KLagd> .

[leebyron]

It's not clear to me what that means. What would a valid query look like that would clearly give a value for a primitive type or an object type?

[leebyron]

Specifically http://facebook.github.io/graphql/#sec-Leaf-Field-Selections is what I'm referring to as "valid query"

[stubailo]

I think the original issue is about having unions where every member is a scalar right? Not unions between scalers and objects?

[OlegIlyenko]

just an idea. Maybe it would make sense to introduce special inline fragment semantics for scalars:

{ 
  example {
    ... on String {
      value
    }
    ... on MyType {
      field
    }
  }
}

This can be generalized by allowing scalars to be used as leafs (current semantics) and as objects with a single field (value). A huge advantage of this approach is that one can use __typename introspection on scalars.

In our API we have a similar issue. For some parts of the schema, we even have introduced object types like StringAttribute, IntAttribute, etc. in order to unify object attributes ("attributes" are a part of user-defined data structures which our service supports) and scalar attributes and provide same introspection capabilities for both.

[migueloller]

What if the result was a scalar then the result would simply ignore the selection set, and if it is an object type then it would consider the selection set. Statically, the query can be analyzed because the schema should have enough information to know if the union contains a scalar or an object or both.

If there is no selection set and the value is an object then the returned value can just be {}. If there is a selection set and the value is a scalar then the returned value can just be the scalar value (ignoring the selection set, just like it is done for object type unions).

On the client most type system allow unions of arbitrary types, so the client can expect a result that is string | { name: string } in Flow, for example.

Nevertheless, this starts getting into the design choices of the simplicity of GraphQL, which I think @leebyron made great points here: graphql/graphql-js#207 (comment)

[calebmer]

Mixing scalars and object types would require a change to the schema which is too complex to consider. Schemaless GraphQL clients depend on the guarantee the { field } is scalar and { field { a b c } } is composite. Given that a scalar field could be an object this is the only reliable way to tell a scalar apart from a composite value without looking at the schema. By breaking that guarantee you introduce a lot of unnecessary complexity into the GraphQL client when you could easily just add a composite StringBox type or similar:

type StringBox {
  value: String
}

[migueloller]

@calebmer, agreed! It's a balance between feature-set and simplicity and I think the GraphQL designers wanted to err on the side of simplicity. Please correct me if I'm wrong @leebyron.

Given this, I think that the original discussion of scalar-scalar (or more specifically, leaf nodes) unions is what we should perhaps consider (as opposed to object-scalar unions).

[rmosolgo]

It seems like the Box approach could solve most of these issues without any changes to the spec, right?

# These two fields are mutually exclusive:
type AreaOfTownBox {
  stringValue: String 
  enumValue: AreaOfTown
}

# or 
type IntOrStringBox {
  stringValue: String
  intValue: Int 
}

It pushes the complexity to the schema but keeps GraphQL simple and predictable. Are there downsides to that approach other than verbosity?

[migueloller]

@rmosolgo, you are right that it is possible but it comes at the cost of added complexity in the schema, verbosity in the queries, and having to check each value in the client side at runtime to determine which field was returned.

[calebmer]

Another approach inspired by @rmosolgo’s recommendation would to have a type system that looks like:

union IntOrString = IntBox | StringBox

type IntBox {
  value: Int
}

type StringBox {
  value: String
}

With a query that looks like:

fragment intOrString on IntOrString {
  __typename
  ... on IntBox { intValue: value }
  ... on StringBox { stringValue: value }
}

[IvanGoncharov]

Because we can't simply make a union that is AreaOfTown | "NEAR_ME" for the input type, then we have to create a completely new enum that contains all of the values in AreaOfTown and "NEAR_ME". This is undesirable because if we ever change the AreaOfTown enum by adding more (this will 100% happen in the future), then we have to make sure we also update our other enum.

@migueloller This problem can be solved directly in your code without any change to GraphQL.
Wehn you define your types in source code just create a small utility function like extendEnumValues in the example below:

new GraphQLEnumType({
  name: 'NewName',
  values: extendEnumValues(AreaOfTown.getValues(), ['NEAR_ME']);
});

As for IDL, I think it would be great to have support for it there, for example allowing extend to be used on enums.
But this is a topic for separate issue/PR.

Allowing uniontypes to contain primitives allows us to model any kind of source that can be polymorphic in its source data. For example, avro data or elasticsearch.
This is a severe limitation on exposing existing data, rather than creating new systems that are defined by graphql from the ground up.

@marcintustin There are number common JSON patterns that you can't express in GraphQL: key-value pairs (aka Maps), Tuples, etc.
I believe this is a bad idea to make GraphQL a superset of all popular formats/protocols.

Given this, I think that the original discussion of scalar-scalar (or more specifically, leaf-nodes) unions is what we should perhaps consider (as opposed to object-scalar unions).

@migueloller There are many cases when API client can't detect type based on the value returned.
For example, union of enums having the same value. It is pretty common for enums to have common values like None, Undefined, etc.
Another example is a union of custom scalars with the same base type like AbsoluteUrl|RelativeUrl.

Also, API client should explicitly specify types which he can handle using ... on TypeName { construction. Without it, if you change Number|Boolean to Number|Boolean|String it will break client which don't expect the value to be a string.

That mean you can't serialize union of scalar types as a scalar value and forced to support some equivalents of __typename and ... on ScalarType { constructs. So the end result will look very similar to solution proposed by @calebmer

Personally, I think unions of scalar types are frequently abused.
A common example of this is returning an array of strings or the single string only because array with one element looks not so nice :)
It forces all API clients to add few more lines of code for every instance of such anti-pattern.

For the cases when you return arbitrary data you can always fall-back to providing value as a JSON string. Such pattern is already used by GraphQL introspection to return defaultValue
http://facebook.github.io/graphql/#sec-The-__InputValue-Type

Also, in SQL, you can have only one type per column and it doesn't prevent it from being dominate language for databases.

[ianks]

A common example of this is returning an array of strings or the single string only because array with one element looks not so nice

There are legitimate use cases that are not simply cosmetic, however.

Here is an example from the Facebook Marketing API. When attempting to model filtering in the API, you have an object that represents a filter expression, like so:

filtering: [{ field: "clicks", operator: "LESS_THAN", value: 42 }]

However, value is by nature polymorphic, as it has to support expressions like this, as well:

filtering: [{ field: "campaign_id", operator: "IN", value: ["1", "2", ...] }]

Currently, this is not possible to express safely AFAIK. Scalar-union types would make this implementation trivial. Falling back to a JSON string for this seems like a poor solution. Some downsides:

1. validate against the schema
2. interactive tools like graphiql become less useful
3. introducing another data serialization format into your resolvers, which can be a point of failure
4. lack of type safety

---

I believe this is a bad idea to make GraphQL a superset of all popular formats/protocols.

This is a strawman. The argument does not necessitate GQL to be a superset of all protocols. Adding scalar union inputs certainly would not make GQL a superset of all protocols.

---

It seems like the Box approach could solve most of these issues without any changes to the spec, right?

# These two fields are mutually exclusive:
type AreaOfTownBox {
  stringValue: String 
  enumValue: AreaOfTown
}

# or 
type IntOrStringBox {
  stringValue: String
  intValue: Int 
}

@rmosolgo The problem I see with this is you assume that the two fields are mutually exclusive; however, there is no way to statically define that in the current type system.

[IvanGoncharov]

A common example of this is RETURNING an array of strings or the single string only because array with one element looks not so nice

@ianks The entire issue is about returning data, it was clearly stated in the initial comment

It makes sense often to have a field that could RETURN, for example, an Int or a String

It has nothing to do with input objects since they are not only missing unions of scalars but don't support unions at all. If you feel that supporting unions of types (including scalars) is essential for GraphQL please open a separate issue.

Here is an example from the Facebook Marketing API

In this example, you know all possible fields in advance, moreover, you know which filtering is possible on which field. So you can represent this in terms of the current GraphQL spec like below:

filtering: {
  clicks: {
    LESS_THAN: 42 
  },
  campaign_id: {
    IN: ["1", "2"]
  }
}

Advantages of this are much better autocompletion and validation on the client side than when using unions of scalars.

[calebmer]

@ianks here’s some related issues on union input types: #202 and graphql/graphql-js#207

[RomanHotsiy]

@calebmer

Another approach inspired by @rmosolgo’s recommendation would to have a type system that looks like:

union IntOrString = IntBox | StringBox

type IntBox {
  value: Int
}

type StringBox {
  value: String
}

There is one more alternative approach. You can just use custom scalars. GraphQL doesn't specify how custom scalars should be serialised so you can return anything, even free-form JSON. For example graphql-type-json and corresponding article in Apollo docs
Custom scalars lack types info though. But this is not an issue for using them. A notable example is GitHub API that has a number of custom scalars like DateTime, HTML, URI and information about type of those scalars (string) is present ONLY in description.

[j0nd0n7]

I was trying to do filtering by a single ID or an array of IDs and I get here :-(
my case:

union IDFilter = ID | [ID]
friends(id:IDFilter): [Friend]

now I have to go like this:
friends(id:ID, ids:[ID]): [Friend]

In my opinion and preference, the first option is better, you always filter using id:xxx

[josepot]

@jdonzet In your case, wouldn't it be better to just do:

friends(idsFilter: [ID!]): [Friend]

If idsFilter is null there is no filter, you return all Friends... If it's not null, the Array must have at least one ID...

[acjay]

I posed a somewhat related question here: https://stackoverflow.com/q/47933512/807674

Although, my question deals with the awkwardness of modeling union variants that should contain no data other than their mere presence (i.e. singletons). I do think it would be extremely helpful to have one blessed syntactic way of approaching this data modeling question, rather than a hodgepodge of ad hoc approaches.

At the end of the day, these all seem to be special cases of the question of "how far GraphQL should go from a data modeling perspective?" I would argue that obvious ways of representing algebraic data types would be extremely helpful. Mostly, this boils down to union allowing singletons (and at that point, may as well allow all scalars too). There's already the concept of an enum, which is really just a union of singletons. There's just no obvious bridge between that and proper union.

More than just a question of syntactic convenience, it means bumpy transitions for data models as they grow in complexity.

[Tiedye]

This feature would be a big help in migrating schemas that don't use the ID type to schemas that do without introducing breaking changes.

[acao]

@benjie it seems ~1+ year later @oneOf RFC would be the place for this, yes?

[benjie]

I'm not sure what @Tiedye has in mind, but I don't think @OneOf can achieve that.

[Tiedye]

@benjie For migrating argument types: union scalar types would allow it to be done more directly as only one intermediate schema would be required to support the old and new types. My use case is migrating Integer ids to ID ids. For example:

# with union scalar types
type Query {
  post(id: Integer!): Post
}
# to
type Query {
  post(id: Integer! | ID!): Post
}
# to
type Query {
  post(id: ID!): Post
}

# without union types
type Query {
  post(id: Integer!): Post
}
# to
type Query {
  post(id: Integer, new_id: ID): Post # schema no longer encompasses that an id is required
}
# to
type Query {
  post(id: ID, new_id: ID): Post
}
# to
type Query {
  post(id: ID!): Post # front-end has to migrate twice instead of once, very annoying
}

[benjie]

With @oneOf:

type Query {
  post(by: PostSpec!): Post
}
input PostSpec @oneOf {
  pk: Integer
}
# to
input PostSpec @oneOf {
  pk: Integer
  id: ID
}
# to
input PostSpec @oneOf {
  id: ID
}

I don't think there's any way to support union scalar types in general (without boxing like @OneOf does) because GraphQL cannot tell the difference between a String, an ID and a custom scalar that's encoded as a string.

[Tiedye]

Why would it need to tell the difference? Argument types are specified in the query, and there is no overloading in GraphQL

[benjie]

Keep in mind custom scalars can do basically anything. Say for example you have:

scalar Base64EncodedJSON

When you send this to the server, you'll send something like: "eyJoZWxsbyI6IndvcmxkIn0="

When the server receives it, it would then "parse" it into an internal representation, which may well be an object such as (JSON syntax): {"hello":"world"}.

From a client perspective, Base64EncodedJSON and ID and String all look the same - they're strings.

From a schema point of view, there's nothing that tells GraphQL that the argument to myField(with: Base64EncodedJSON! | String!): ReturnType is not valid. The schema author might even expect it to work - give me a string, or give me the JSON. However, when we come to CoerceVariableValues() or similar, GraphQL does not know what parse method to call - is it dealing with a string, or base64-encoded JSON? How can it know?

[Tiedye]

I think it is acceptable to run all applicable parse methods and if multiple succeed either: throw an error, or preferably have some tiebreak like first type in the union. There is likely a more refined way of doing the tiebreaks, but I think it is far from not doable. The extra execution cost is minimal and worth it for the use case I see myself using it for.

[rivantsov]

being on the server side, I agree with @benjie, more troubles (recognizing what the heck is this token) than benefits. A simple oneOf wrapper type would provide equivalent functionality without ambiquities

[Tiedye]

oneOf just doesn't solve the problem sadly :(

[aitorllj93]

Another approach inspired by @rmosolgo’s recommendation would to have a type system that looks like:

union IntOrString = IntBox | StringBox

type IntBox {
  value: Int
}

type StringBox {
  value: String
}

With a query that looks like:

fragment intOrString on IntOrString {
  __typename
  ... on IntBox { intValue: value }
  ... on StringBox { stringValue: value }
}

This sounds like ValueObjects to me so I think it's the best approach right now:

union ScalarValue = IntValue | StringValue;

type IntValue{
  value: Int
}

type StringValue{
  value: String
}

fragment scalarValue on ScalarValue {
  __typename

  ... on IntValue { 
     value 
  }
  ... on StringValue { 
    value 
  }
}

[ThisIsRuddy]

I too would enjoy the benefits of using enums in unions, it would allow me to modularise my directive inputs options which I currently have to use one big enum for 😅

[psirenny]

The lack of a union scalar type exacerbates the null/undefined problem and API authors/consumers largely avoid boxing because of how unergonomic it is. A None enum combined with scalars would allow to disambiguate between the many meanings of null.

[CRC-Mismatch]

Please correct me if I'm wrong at any rate, but I believe a use-case example would be the GraphQL spec's own "error type" specification in itself... According to the examples in https://spec.graphql.org/draft/#sec-Errors, the error's path element would be a list of both Strings and Ints. As it currently stands, a GraphQL API would be unable to reproduce its own GraphQL errors in a valid schema as-is (for example if I want to reproduce such errors in a log query of sorts).