Add GraphQL support

[kinow]

As per discussion in Riot, we are now ready to start porting the work from @dwsutherland with GraphQL + Flask to here, with GraphQL + Tornado.

As both the Flask and the Tornado implementations are provided as extensions of the Python Graphene project, it should be fairly easy to port most of the code.

Flask has an approach of a slim core, with extensions installed via pip for things like CORS, HTTP Basic, etc. Whereas Tornado includes a slim core too, but without official modules for things like CORS or some authentication mechanisms.

So this may be a bit of a challenge.

See:

• https://pypi.org/project/graphene-tornado/
• https://github.com/graphql-python/graphene-tornado

[kinow]

Link to compare all changes in that branch: https://github.com/dwsutherland/cylc/compare/9ca798d815934966d28a65438d7fd7645ffc174f..34a87b1b8b73277ed810cc365db7bda9ad658054

[kinow]

@dwsutherland looks to me like there are small changes required for GraphQL + Flask. The majority of the changes are in Cylc to have the right objects required to populate the GraphQL response.

Q: about the schema.py.

I believe this is where we have the complete API contract, available through our GraphQL endpoint for clients.

Where should we put it? I think it will go to cylc/cylc. The reason for that is to avoid cyclic dependency between the Python modules.

This current project, cylc/cylc-jupyterhub, may use cylc/cylc as dependency, to start commands, scan suites, etc. So we have a dependency cylc-jupyterhub --- to ---> cylc.

But if we have schema.py here, I think we would have to import it from cylc/cylc. Causing a cyclic/circular dependency. Which looks like we can try to hack our way out of circular dependencies in Python by changing where/how we import stuff, but it would still be harder to maintain.

Another alternative would be to have cylc/cylc/, cylc/cylc-jupyterhub, and a cylc/cylc-graphql or something module, where both cylc and cylc-jupyterhub would depend on.

WDYT?

[kinow]

Work started here: https://github.com/kinow/cylc-jupyterhub/tree/add-graphql

So far have only added graphene-tornado. Created a new virtualenv, then installed through pip install -e . with no issues. Already have the base dependency for the GraphQL endpoint. Just need to confirm where the schema will be (see comment above)

[dwsutherland]

@dwsutherland looks to me like there are small changes required for GraphQL + Flask. The majority of the changes are in Cylc to have the right objects required to populate the GraphQL response.

This is a breaking change, and we will no longer be able to use the Flask branch, we will divorce from it here on out. Yeah, we need to setup the feed from cylc8.

Q: about the schema.py.

I believe this is where we have the complete API contract, available through our GraphQL endpoint for clients.

Where should we put it? I think it will go to cylc/cylc. The reason for that is to avoid cyclic dependency between the Python modules.

This current project, cylc/cylc-jupyterhub, may use cylc/cylc as dependency, to start commands, scan suites, etc. So we have a dependency cylc-jupyterhub --- to ---> cylc.

But if we have schema.py here, I think we would have to import it from cylc/cylc. Causing a cyclic/circular dependency. Which looks like we can try to hack our way out of circular dependencies in Python by changing where/how we import stuff, but it would still be harder to maintain.

Another alternative would be to have cylc/cylc/, cylc/cylc-jupyterhub, and a cylc/cylc-graphql or something module, where both cylc and cylc-jupyterhub would depend on.

WDYT?

The schema is no longer needed in the suite (we will transport the full field objects) and resolved them in the UI Server. So I'll make a pull request to cylc/cylc-jupyterhub when I integrate it after your done with Tornado (unless there's good argument for having GraphQL at both ends)..

Just use a hello world schema until then..

All good?

[kinow]

Just use a hello world schema until then..

Oh, much easier for now. Thanks @dwsutherland !

[kinow]

Done.

[kinow]

Tested WebSockets today. Used the simple example from the Tornado documentation, which just prints back whatever you say on the message.

Not possible to use that endpoint in a browser, unless you write JS code. For now the WebSocket is insecure, but we should try to use a secure WebSocket in production.

Then tested without a cookie/authentication.

And that worked, as the endpoint was not extending JupyterHub's HubOAuthenticated class/trait.

Extending the right class, we can't connect without logging in:

And finally, once logged in, the WebSocket worked again! 🎉

So I believe we are all good with WebSockets support for Tornado, with integration with the Hub. @dwsutherland I think the only thing missing now is the graphql-python/graphql-ws#25 👍

[dwsutherland]

@kinow - Yes, it can't use the same HTTP authentication.. but authentication is done at the hub right? else we may be able to use other types, or bespoke (as you mentioned)

[kinow]

Authentication done in the hub. We can still change who is authorised to see what however.

[kinow]

Merged. Remaining work in #14, and later more work on data model.