Modernizing the server

[paulmelnikow]

Since the legacy services have almost all been rewritten and the modern services are well abstracted from the server, I'd like to start work on modernizing the server itself.

The server is not a lot of code, though it is a bit tricky, and it's code that I'd like to be as transparent as possible to new people who are reading it.

Scoutcamp has served this project well for a long time, and much respect is due to @espadrine and @jankeromnes for creating something that has performed so well for so long. However there are some downsides, mainly that the framework isn't fully documented, the code is difficult to read, and it hasn't gotten wide adoption. I'd like to move forward with something else.

Criteria would be:

• Good documentation, active maintenance, and high test coverage.
• Native support for async/await support.
• Works well with path-to-regexp and provides an implementation path for trie-based routing.

I've started playing around with micro, micro-router, and routington. One idea is to create a micro-trie-router using routington and based on code from micro-router.

[paulmelnikow]

One piece that should be tackled as part of this task is generating JSON badges directly, instead of indirecting through a template.

[chris48s]

The concept of replacing scoutcamp with something else has been on my mind again.

I think last time we discussed it in much detail we said one of the bits of work we wanted to get done first was replacing request with a modern async/await based http client. This was done (slowly) under #4655

In common with that issue, I guess there are actually two jobs here: One is the legwork of replacing scoutcamp with $SOMETHINGELSE, which is going to be a big and difficult job. However before that, we have to decide what $SOMETHINGELSE is. Tbh, working that out could take as long so I figured it is useful to start that discussion.. has anyone on the core team got any thoughts on this?

[paulmelnikow]

A couple weeks back, had an offline discussion with Chris about this, and wanted to propose a next step.

Basically our proposal is that $SOMETHINGELSE be Express. It’s not the hottest or fastest-growing framework, but it’s maintained, well-documented, and will be around for a long time.

Shields basically already has a custom framework, with few points of contact between scout camp and the service classes. Express is really flexible, so I think it’s a pretty good candidate for gluing in a higher-level framework like ours.

In my professional work, I use Express similarly – with a higher-level framework Postgraphile – and it works very well.

(For what it's worth, finishing this work is basically rewriting core/server/server.js and the various base service class register() functions. We'll also have to adjust how we're dealing with cache headers, and there are tricky edge cases here and there (which we may get wrong on the first try). But, I don't think it will be such a hard swap.)

[paulmelnikow]

I had a go of this in #7877.

I haven't done anything with trie-based routing, but that could be addressed in a new issue.