Implement subscriptions for real-time updates

[KyleAMathews]

In the subscriptions channel on Slack, a few of us (@taion, @skevy, and I) were discussing what are the missing pieces needed to implement subscriptions.

One important piece in Relay is the ability for a real-time network layer to apply "mutations" to the Relay Store. People will use different technologies to push data to their client. In each there needs to be a way to tell Relay this field changed or this node should be prepended to this connection, etc.

[taion]

I think it'd be clearer to say "apply mutation/subscription payloads from the network layer" rather than more generally "from userland".

[josephsavona]

The way we've been thinking of subscriptions is as a mix of push/pull semantics. In particular, the client would explicitly request updates from the server for a particular event, and the server could then notify the client when the event occurs (until the client closes the subscription).

Here's the rough API we're considering:

Public API

The public API would be similar to mutations. The main difference is that subscriptions would have fixed/static queries (unlike mutations which intersect the fat query with those fields queried by the application).

1. Define a subscription by subclassing Relay.Subscription:

class StoryLikeSubscription extends Relay.Subscription {
  // Return a static query listing all fields to fetch when the event occurs
  getSubscription() {
    return Relay.QL`
      subscription {
        storyLikeSubscription(storyId: $storyId) {
          likeCount
        }
      }
    `;
  }
  // Describe how to handle the response payload (just like mutations)
  getConfigs() { ... }
  // Get the variables to send along with the query (e.g. the value of `$storyId`)
  getVariables() { ... }
}

2. Initiate a subscription, requesting updates from the server:

// Relay.subscribe(subscription, callbacks)
var subscription = Relay.subscribe(
  new StoryLikeSubscription({storyId: ...}), 
  {
    onNext: (data) => { ... }, // called whenever data is received
    onError: (data) => { ... }, // called if an error occurs
  }
);
subscription.dispose(); // notify the server to stop sending updates

Internal Changes

The rough steps to implementing the above are:

• Update the graphql dependency to the latest version which supports parsing subscription { .. }.
• Modify babel-relay-plugin to handle parsing those queries, outputting GraphQL.Subscription objects (note that GraphQL.Subscription and the corresponding RelayQuery.Subscription are already implemented ;-)
• Create a Relay.Subscription base class similarly to Relay.Mutation.
• Create a RelaySubscriptionRequest class, similar to RelayMutationRequest for describing a subscription to send to the network layer. The public API would be roughly:

type RelaySubscriptionRequest = {
  getDebugName: () => string;
  getSubscription: () => RelayQuery.Subscription;
  onNext: (data) => void; // called by the network layer whenever data is received from the server
  onError: (err) => void; // called by the network layer if an error occurs. this ends the subscription.
}

• Provide a stub implementation in RelayDefaultNetworkLayer which throws when subscriptions are requested
• Docs, examples, tests
• Last but not least, a working example implementation.

[taion]

@josephsavona

I think that makes a good low-level API. My concern is that this doesn't necessarily match how I might want to think about subscriptions from the front end.

Echoing @KyleAMathews's comment, I feel like I really think more about wanting live updates for a particular query, rather than wanting to subscribe to a specific named subscription. Moreover, I think having specific subscriptions requires closer coupling between the front end and the back end - unlike with mutations and few fields, if the server starts defining a new subscription type that I'm not handling from my front end, I'll lose some set of updates, some of which I might care about.

Do you think it might be possible to define some higher-level subscription API that works on the level of nodes rather than on the level of "subscriptions"? I definitely want to define mutation-style getConfigs for connections, but I feel like there should be some concept of "here is a generic update to this node that you don't know how to handle, just re-fetch everything in the Relay store for this node from the GraphQL server".

[taion]

I guess to be more concrete, suppose there were StoryLikeCountSubscription, StoryLikeStringSubscription, and StoryLikersSubscription, which respectively covered the like count, the like string, and the list of likers. Would I have to make multiple subscriptions from my front end if I needed to listen to more than one of those? Would the back end have to know to update each of those subscriptions (and any additional ones that get added) every time someone likes a story?

[KyleAMathews]

This is the issue for the @live directive idea graphql/graphql-js#188

[josephsavona]

I really think more about wanting live updates for a particular query, rather than wanting to subscribe to a specific named subscription

This is an ideal end-state for subscriptions. However, this can be difficult to scale fully and correctly. For Relay core, we'd like to add support for patterns that we or the community have proven to be effective and practical to implement. For now, we'll pursue (and accept contributions for) the event-based subscription API that I described above because we've found this to be effective in practice.

That said, we certainly welcome you to explore other options such as @live queries - get something working and send a pull request so we can discuss!

[KyleAMathews]

@josephsavona what are the steps to add support for a @live directive?

[skevy]

Just FYI to all those on this issue...I'm actively working on implementing the general concepts @josephsavona outlined above. Nothing in PR state yet...but hopefully it won't take me toooo much longer to get it there.

[josephsavona]

@KyleAMathews thanks for starting this discussion! I've created a new task with the API described above to help guide contributions. Let's continue discussion at #541

[josephsavona]

also cc @skevy

[gilesbradshaw]

@taion 's idea is feasible if each node type is subscribable to. I did something like this with Entity Framework and signalR where each change to the database was published.

So any tree of graphQL nodes on the client would result in a tree of subscriptions to the server. I then did additional filtering on the client so the server didn't do 'live queries' it just published events when a node's related nodes changed.