MVP Epic (Walking Skeleton)

[kjdelisle]

Background

The first preview release of LoopBack4 will enable developers to start building extensions for our new framework version. The next logical step is a release enabling developers to actually build applications (REST API servers). We would like to ship this version in February 2018 (but the date can change!). When we factored in our current team velocity and the amount of work needed, it became clear that we must significantly reduce scope (number of features) if we want to release early next year.

To help us with deciding what to include and what to postpone, we decided to use a mixture of Minimum Viable Product and Walking Skeleton approaches. We picked a small Todo application shown in our loopback4-example-getting-started as the MVP application that should be easy to build with the MVP version of LoopBack4. When grooming our backlog, we can use the following question to drive our priorities: is the user story in question a missing piece that will prevent people from building our Todo app? If it's not required for the Todo app, then it is almost certainly out of scope of the MVP release.

Please note that minimum does not mean hacky and difficult to use! We want to give LoopBack users the best developer experience from the beginning, to get them interested in the new version of our framework, motivated to give it a try and provide us with valuable feedback that we can use to prioritize which features to implement next.

Scope

As a developer creating a small Todo application with REST API/JSON payload only (see loopback4-example-getting-started), I would like to have:

• CLI tooling to rapidly scaffold my application project
  • create my project directory, package.json, and subdirectories for me, with some boilerplate application definition and index.ts
  • provide functions for generating controllers and models (using both a prompt-driven approach, as well as CLI arguments and flags)
• runtime APIs that are easy to use and robust (this means functions like app.server, app.controller, etc)
  • avoid needing to perform configuration in multiple locations (maintain inversion of control principles)
  • application startup should not require overriding start (unless some async changes are desired by the developer)
• convention-based bootstrapper that will pick controllers/models/etc. files from pre-defined directories in my project and add them to my app automatically for me
  • instead of needing to call app.controller(MyController), we'll look in /src/controllers and find my.controller.ts
  • should happen on application startup (but before server(s) startup!)
• solid documentation to ensure that I have great examples every step of the way
  • documentation should lead you down a "garden path" from "I don't know what I'm doing" to "I can write a microservice in loopback"
  • pages should link to the next step (or back to somewhere that can take you to the next step)
• automatic validation of my inputs against my Swagger specification
  • If the swagger definition for the endpoint says that the body's "foo" parameter must be a string, and the client sends a request with property "foo" set to a number, then the framework automatically catch this and throw the appropriate error (most likely a 400 Bad Request).

High-level focus areas:

• CLI tooling
• Conventional bootstrapper
• Upgrade from Swagger v2 to OpenAPI Spec v3
• Runtime improvements (validation of input args)
• Docs improvements
• Project infrastructure

Out of scope

• ORM/juggler refactorings - we will keep using existing loopback-datasource-juggler
• Documentation on advanced topics going beyond MVP scope - we will cover best practices/recommended paths only.
• Authentication and authorization, ACL
• Other transports/protocols (gRPC, etc.), we will support REST API only
• Automagic REST API for model relations
• Writing applications in pure ES2017
• Migration from LB3
• Top-down (api-spec-first) approach for building applications
• Self-hosted explorer (e.g. http://localhost:3000/explorer)

[virkt25]

• Does Model + Controller Generation also include OpenAPI Spec generation?
• Is OpenAPI Spec support part of MVP? (instead of using swagger2openapi as we currently do). I think it's key to have first class support for OpenAPI as it's the latest standard
• Can you clarify runtime API's? (Is that .server, .controller, .bind (+ variants and withOptions / Extension PR's)?)

[kjdelisle]

Bonus points to @virkt25 for being the first to ask questions. :D

I'm answering them out-of-order since some questions answer others.

Is OpenAPI Spec support part of MVP? (instead of using swagger2openapi as we currently do). I think it's key to have first class support for OpenAPI as it's the latest standard

Originally, yes, but @bajtos and I spoke and we don't think v3 can make it into the MVP given the extra work to revalidate. I think it should be near the top of our stretch goals, but it currently looks like we won't have time for it.

Does Model + Controller Generation also include OpenAPI Spec generation?

We already generate the v2 swagger spec for them, and if the v3 support is out-of-scope, then this will be as well.

Can you clarify runtime API's? (Is that .server, .controller, .bind (+ variants and withOptions / Extension PR's)?)

This question has a two-part answer. The runtime APIs do include things like .server, .controller and so on. I don't know if things like .withOptions will make it into the MVP, though we need to decide that as a part of our planning meetings since it forces us to pick an approach with respect to things like multi-server support per application (IMO)

[kjdelisle]

@raymondfeng @bajtos
Is HTTPS support in the MVP?

[virkt25]

I think HTTPS Support should be MVP because security is important. Also might come in handy for future when implementing HTTP2 as it's required by browsers supporting HTTP2 ...

[b-admike]

• provide functions for generating controllers and models

Does this mean provide prompts to users which in turn call these functions?

• convention-based bootstrapper that will pick controllers/models/etc. files from pre-defined directories in my project and add them to my app automatically for me

Would the bootstrapper run in the background for all applications and add these files whenever they're added?

• automatic validation of my inputs against my Swagger specification

What does automatic validation mean in this case? Does it come as a test case for users' generated applications using our testlab swagger validator?

[jannyHou]

application startup should not require overriding start (unless some async changes are desired by the developer)

Could you elaborate more on this? --> "overriding start" do you mean function app.start or?

[kjdelisle]

@b-admike

Does this mean provide prompts to users which in turn call these functions?
Given that the current CLI implementation that Raymond made uses yeoman, it'd probably be a similar CLI-prompt style implementation (lb controller -> What is the name of your controller? -> etc...)

It should also support args flags so that we can programmatically verify it's behaviour without needing to write a hacky engine that inputs arrow keys (cough cough)

Would the bootstrapper run in the background for all applications and add these files whenever they're added?

It would be a runtime step, much like how loopback 3 does it now. On app start, we scan those directories and build up the bindings. It would probably need to be a behaviour of the Application class (since it, or a subclass of it, will always be a part of any LB4 app).

What does automatic validation mean in this case? Does it come as a test case for users' generated applications using our testlab swagger validator?

It means that if you input a body with a property { "foo": 2 } but the swagger defined for that endpoint says that the body's "foo" parameter must be a string, that it will automatically catch this and throw the appropriate error (most likely a 400 Bad Request).

[kjdelisle]

@jannyHou

Could you elaborate more on this? --> "overriding start" do you mean function app.start or?

Yes. Unless someone wants to do something very atypical, they shouldn't need to add this to their Application subclass:

async start() {
  // some stuff is happening here
  await super.start();
}

[kjdelisle]

@shimks 📟

[kjdelisle]

@dhmlau 📟

[bajtos]

👀 📟 👀
I edited the issue description at the top, added "Background" section explaining the motivation for this MVP approach. I also captured Kevin's comment explaining automatic validation.

[bajtos]

What does automatic validation mean in this case? Does it come as a test case for users' generated applications using our testlab swagger validator?

It means that if you input a body with a property { "foo": 2 } but the swagger defined for that endpoint says that the body's "foo" parameter must be a string, that it will automatically catch this and throw the appropriate error (most likely a 400 Bad Request).

TodoController#createTodo method is a good example:

@post('/todo')
@param.body('todo', TodoSchema)
async createTodo(todo: Todo) {
  if (!todo.title) {
    return Promise.reject(new HttpErrors.BadRequest('title is required'));
  }
  const result = await this.todoRepo.create(todo);
  return result;
}

The todo payload should be validated by the REST transport (extension) using Swagger/OpenAPI metadata provided by the developer. In this case, TodoSchema should mark title as a required property, and that should be all that's needed to get the input validated.

[shimks]

runtime APIs that are easy to use and robust (this means functions like app.server, app.controller, etc)

avoid needing to perform configuration in multiple locations (maintain inversion of control principles)

Can you clarify what you mean by this? I'm assuming that you want an alternative to what we currently have (explicit binding)?

[jannyHou]

automatic validation of my inputs against my Swagger specification

Does the validation happen when the corresponding function is invoked or we want to catch it before invoking a controller function?

[bajtos]

automatic validation of my inputs against my Swagger specification

Does the validation happen when the corresponding function is invoked or we want to catch it before invoking a controller function?

IMO the validation (and conversion/coercion) should happen before the controller method (or route handler) is invoked. One option is to implement this as part of parseParams sequence action.

[dhmlau]

Closing as done.