Converting compiler to ppx.

[sainthkh]

A little story

When I first released and promoted reasonql, I got this question a lot.

Why is it a compiler? Not a ppx? 

My answer was this:

To support fragments, we need to open all files and check the graph of GraphQL codes in files. 

And I thought it was impossible to support fragments with ppx, because ppx only replaces some elements in abstract syntax tree, so it cannot walk through entire file tree and find the appropriate fragment.

I had believed that idea until this morning and was thinking how to persuade other developers. But somehow, I realized that I was wrong if I give up some compile speed for initial and clean compilation. (And I'm not so sure now, but it seems that this speed down isn't that big.)

The Goal

Let's use the fragments snippet in this repo. (I removed Button.re here because it's not necessary for now.)

/* App.re */
let query = ReasonQL.gql({|
  query AppQuery {
    posts @singular(name: "post") {
      ...PostFragment_post
    }
  }
|})

/* Post.re */
let query = ReasonQL.gql({|
  fragment PostFragment_post on Post {
    title
    summary
    slug
  }
|})

Currently, it generates modules like below:

/* AppQuery.re */
/* Generated by ReasonQL Compiler, PLEASE EDIT WITH CARE */

/* Original Query
query AppQuery {
    posts @singular(name: "post") {
      ...PostFragment_post
    }
  }
fragment PostFragment_post on Post {
    title
    summary
    slug
  }
*/
let query = {|query AppQuery{posts{...F0}}fragment F0 on Post{title
summary
slug}|}

type post = {
  f_post: PostFragment.post,
};

type queryResult = {
  posts: array(post),
};

type variablesType = Js.Dict.t(Js.Json.t);
let encodeVariables: variablesType => Js.Json.t = vars => Js.Json.object_(vars);

[%%raw {|
var decodePost = function (res) {
  return [
    PostFragment_decodePost(res),
  ]
}

var decodeQueryResult = function (res) {
  return [
    decodePostArray(res.posts),
  ]
}

var decodePostArray = function (arr) {
  return arr.map(item =>
    decodePost(item)
  )
}

var PostFragment_decodePost = function (res) {
  return [
    res.title,
    res.summary,
    res.slug,
  ]
}
|}]

[@bs.val]external decodeQueryResultJs: Js.Json.t => queryResult = "decodeQueryResult";
let decodeQueryResult = decodeQueryResultJs;

/* PostFragment.re */
/* Generated by ReasonQL Compiler, PLEASE EDIT WITH CARE */

type post = {
  title: string,
  summary: string,
  slug: string,
};

Currently, it copies fragment codes to the query module. But when we change it like this, we don't need this copying process.

/* AppQuery.re */

module AppQuery = {
  let query = {|query AppQuery{posts{...Post_Fragment_post}}|} ++ Post.Fragment.query;

  type post = {
    f_post: Post.Fragment.post,
  };

  type queryResult = {
    posts: array(post),
  };

  type variablesType = Js.Dict.t(Js.Json.t);
  let encodeVariables: variablesType => Js.Json.t = vars => Js.Json.object_(vars);

  type postJs = Js.Json.t;

  type queryResultJs = Js.t({.
    posts: array(postJs),
  });

  let decodePost: postJs => post  = res => {
    f_post: Post.Fragment.decodePost(Obj.magic(res)),
  };

  let decodeQueryResult: queryResultJs => queryResult = res => {
    posts: res##posts |> Array.map(post => decodePost(post)),
  };
}

/* PostFragment.re */

module Fragment = {
  let query = {|fragment Post_Fragment_post on Post{
  title
  summary
  slug}|}

  type post = {
    title: string,
    summary: string,
    slug: string,
  };

  type postJs = Js.t({.
    title: string,
    summary: string,
    slug: string,
  });

  let decodePost: postJs => post = res => {
    title: res##title,
    summary: res##summary,
    slug: res##slug,
  }
}

Then, it would work without any problem. To handle fragments, we need Obj.magic. Even if a fragment changes, every other code will call the changed code without any problem when we set this rule:

The name of fragment should be written in this format:
[FileName]_[ModuleName]_[DataTypeName]

So, the PostSummary_post should be named in new reasonql_ppx like this:

Post_Fragment_post.

When we add this small rule, everything will be OK.

Todo

• ReasonML port of graphql-js. Currently, graphql_ppx can parse graphql client query but cannot parse the schema. reason-graphql has the most similar parser. But it doesn't parse location. Maybe we need to create a reference implementation of the graphql-js. -> Result: Needs implementation. 1) graphql_ppx doesn't parse schema definition. 2) reason-graphql and ocaml-graphql-server don't compile interface. => Check https://github.com/sainthkh/graphql-reason

• ppx that generates types and codecs for graphql queries.

• move the reasonql.config.js to bsconfig.json

• add the option to show generate code on files. -> As reasonql_ppx generates a lot of things, sometimes, we want to see what is going on.

[thangngoc89]

graphql_ppx currently parse introspection query, to save you the trouble of making a spec compliant SDL parser, maybe you could use that directly? It contains identical information to SDL

[sainthkh]

@thangngoc89
I was thinking about, too. And I believe it can save us some time. But I think we need the ported library.

1. We need to write or borrow the parser for client query anyway. In the end, we'll copy here and there of the official code. (Yes, we can borrow some code from graphql_ppx. But when something needs to be tweaked, we need to read graphql-js.)

2. SDL is much much easier to read than JSON. Currently, graphql_ppx interprets GraphQL SDL AST written in JSON. For computers, SDL or JSON doesn't matter. But to human eyes, They cannot be compared. And from SDL, we can understand what we can do with the server faster.

And when we use JSON, users need to remember to generate JSON file with npm commands even when they can access their own GraphQL file. (In most monorepo projects, it's really easy to get access to the schema file.) It might be the reason why Google autocompletes graphql_ppx with "graphql_ppx schema file not found".

(As for public APIs, we can provide JS tool to convert JSON back to SDL and start from there. We can use the print function.)

3. Finally, we need this library for future Reason GraphQL developers. Currently, every GraphQL library/tool implements partially-featured GraphQL parser for themselves. For future developers, this reality can be overwhelming. We need the parser that they can rely on and create what they want.