Instrument GraphQL and expose data with built-in type

[KyleAMathews]

Was just brainstorming how cool it'd be if GraphQL was self-instrumented. Imagine if GraphQL logged every query + resolve timings for each field AND exposed all this data as a built-in type.

On top of this you could build:

• Monitoring (e.g. if query times for this customer critical query goes above 100 ms, send an alert)
• Something analogous to DBs' slow query log
• Build tool similar to GraphiQL but for ops. Connect to your production system and watch a live log of queries. Show histogram of 20 slowest queries over the past week. Zoom in on trouble spots and prioritize speeding those queries.
• Audit logs for users.
• etc.

The default implementation would be in-memory but easily overridable for any serious install so logs could be persisted.

Thoughts? What does Facebook do for instrumenting/logging GraphQL?

This is potentially a huge advantage to using GraphQL. Monitoring/alerting/auditing is freaking hard and this would standardize a big chunk of it.

[KyleAMathews]

And the live updating monitor of queries would also be huge to use during development as it'd give a lot of insight to what's actually happening under the hood. Very helpful to those new to GraphQL (read everyone) to help understand how the system works and also very helpful when prototyping new types.

[johanatan]

+1 to this.

[rmosolgo]

Wow, that's a great idea! Even if it doesn't make it into the spec, seems like servers could implement it and expose it through the extensions key (http://facebook.github.io/graphql/#sec-Response-Format)

[KyleAMathews]

@rmosolgo as I understand it, there wouldn't need to be any changes to the spec. The metrics gathering / logging would be handled internally and be exposed as built-in types e.g. queryLogType

[KyleAMathews]

So you could do queries something like:

query slowQueries {
  queryLog(durationGreaterThan: "100", since: "2015-08-08T18:58:27+00:00") {
    type
    duration
    fields {
      name
      description
      duration
    }
  }
}

[leebyron]

We do something really similar to what's proposed here!

We actually do not query the results of query executions using GraphQL itself, but we do have a logging interface throughout GraphQL which allows us to measure access and performance on a per-field basis.

[KyleAMathews]

@leebyron do you see value in standardizing a type or two for exposing this data? I think that'd be a boon to the GraphQL ecosystem as it'd allow a lot of new or existing Ops tools to be build on top of or integrated with GraphQL which is absolutely necessary for widespread adoption.

[leebyron]

I think it's way too early to standardize that. We have our internal representation of this which is extremely specific to our internal metric tools. Standardizing what we have would mean no one would pay attention to the standard. Standardizing something else would mean the thing we have is not really standard.

If a standard emerges across many implementations that tools are taking advantage of, then we should consider a standardization effort. Otherwise I think this fits well into the realm of experimentation.

[leebyron]

It's also not entirely clear to me that this kind of metadata needs to be exposed through GraphQL itself. We have lots of ops tools that measure GraphQL usage but do not expose their data through GraphQL since our client applications do not need access to that information - but our infrastructure ops systems do and GraphQL is not the right tool for their data access needs.

The important piece that I'm excited to add here is a logging interface which will allow for flexibility in connecting any type of ops service to measure GraphQL usage.

[KyleAMathews]

I agree with your argument that it'd be too early to standardize any sort of types. I'll experiment with creating types internally and report back.

[KyleAMathews]

Huey Petersen just wrote this http://hueypetersen.com/posts/2015/11/06/instrumenting-graphql-js/

[clayallsopp]

I used the information above and hackily implemented a simple API in GraphQLHub.

Some ideas on a first-class API:

// on the Schema objects
let schema = new GraphQLSchema();
schema.instrumentation((instrumentation) => {
});

// at the graphql fn level
// opt-in is probably best to remove perf overhead
graphql(schema, query, { instrumentation : true }).then((result) => {
  console.log(result.instrumentation);
});

// perhaps support it at the express-graphql level, which returns it inside `extensions.instrumentation`)
graphqlHTTP({ schema: MyGraphQLSchema, instrumentation: true });
// or via a callback
let handleInstrumentation = (instrumentation) => ...
graphqlHTTP({ schema: MyGraphQLSchema, instrumentation: handleInstrumentation });

[clayallsopp]

Another possibility (alluded to by @KyleAMathews) is adding a __duration field to every field:

query {
  viewer {
    __duration
    name {
      __duration
    }
  }
}

Might be too complex/backwards-incompatible (i.e. now name can either be a scalar or an instrumented type). But it has the benefit that now all you need to do is traverse the GraphQL data response to get instrumentation data, instead of a per-implementation API

[sibelius]

I think Apollo Optics is the first one to do it.

we can get some insights from them

[selvathiruarul]

Any progress on this? Any suggestions for instrumenting graphql-js?

[IvanGoncharov]

I think #1516 would a good first step and allow 3rd-party dev to produce middlewares that will work with any graphql-js based server. After that as Lee suggested we need to observe and standardize emerging pattern.

It's the only realistic goal that we have ATM so I'm closing this issue in favor of #1516