[Spike] Drop LB3 model files into LB4 app

[bajtos]

The migration approaches described in #485 and #1849 have a major problem: they require developers to migrate everything in one big change, there is no way how to migrate in smaller self-contained steps. I would like to look into ways how to make the migration more incremental.

Background

Essentially, LB3 consists of three parts: loopback-datasource-juggler providing ORM (data modelling and database access), strong-remoting providing REST API layer, and loopback glueing all things together. In LB4, we are still using the same juggler under the hood. strong-remoting was replaced with @loopback/rest and the "glue" has been completely rewritten.

The idea

Carry over LB3 models (.json and .js files) into LB4. Use juggler v4 under the hood, there were very little changes from LB3 there. Write a compatibility layer that will process strong-remoting metadata and build LB4 controllers and/or handler routes to expose LB3-flavored models via the new REST API layer.

The envisioned user experience

• Take the existing LB3 app
• Create a new LB4 project, or perhaps modify existing LB3 project infrastructure to LB4
• Add @loopback/compat-v3 extension to the LB4 app
• Done. All models are exposed via LB4 app.

The plan

Things we will need to implement (or copy from LB3):

• registry of models (app.registry)
• base models (Model and PersistedModel)
• parts of loopback-boot responsible for model definition & registration

Things to implement anew:

• Code converting strong-remoting metadata into LB4 routes. We may leverage loopback-swagger to convert strong-remoting metadata into OpenAPI spec that's used as the remoting format by LB4.

Things that will not work (in the first version)

• Authorization and related built-in models like User and AccessTokens
• Boot scripts
• More items may be discovered (and documented!) during the spike

Task list

• Registry of models and datasources, configure the model in the app
• Infrastructure for defining remote methods (strong-remoting metadata), define few CRUD endpoints
• Expose remote methods via @loopback/rest
• Boot models from JSON + JS files
• Example app: migrated loopback-getting-started
• Summarize the work, write a high-level description of the PoC implementation.
• Open a pull request to discuss the findings: RFC: use LB3 models in an LB4 app [SPIKE - DO NOT MERGE] #2274
• Open a pull request to fix the gaps discovered in loopback-datasource-juggler type definitions - see Improve Model type definitions loopback-datasource-juggler#1685
• Create GitHub issues for follow-up user stories

[bajtos]

work in progress: https://github.com/strongloop/loopback-next/tree/spike/lb3

[bajtos]

working notes: https://github.com/strongloop/loopback-next/blob/spike/lb3/lb3api.md

[bajtos]

the pull request: #2274

[bajtos]

Cross-posting #2274 (comment)

I have discussed the spike with @raymondfeng & @dhmlau and we agreed to stop pursuing this direction. While it does provide relatively easy migration path for LB3 users, there are too many downsides for us. Most importantly, we are re-introducing LB3 codebase that we wanted to get rid of, plus we are delaying the time when LB3 users will finally move to the new LB4 design.

As a short-term solution, we will allow LB3 users to mount their entire LB3 app in an LB4 project - see #2301, #2318 and the list of follow-up tasks posted in #2318 (comment):

• Mount Express router with OpenAPI spec Mount Express router with OpenAPI spec #2389
• Docs: request-handling pipeline Docs: request-handling pipeline #2390
• Boot and mount LB3 app Boot and mount LB3 app #2391
• Example: mount LB3 app in LB4 Example: mount LB3 app in LB4 #2392
• Migration guide: mount LB3 in LB4 Migration guide: mount LB3 in LB4 #2393
• Spike: dynamically updated LB3 API (routes + spec) Spike: dynamically updated LB3 API (routes + spec) #2394

For longer-term, we will look into ways how to simplify migration from LB3 models to LB4. I'll create follow-up spike stories for that soon.

[bajtos]

The list of follow-up stories:

• Import LB3 model JSON file to LB4 model class Import LB3 model JSON file to LB4 model class #2480
• From model definition to REST API with no repository/controller classes From model definition to REST API with no custom repository/controller classes #2036
• From datasource config to Service REST API with no proxy/controller classes Spike: From datasource config to Service REST API with no proxy/controller classes #2481
• Expose custom remote methods contributed by CRUD connectors on top of DAO methods Expose custom remote methods contributed by CRUD connectors on top of DAO methods #2482
• From Relation definition to REST API with no repository/controller classes [EPIC] From relation definition to REST API with auto-generated repository/controller classes #2483
• Spike: Discover and define models at runtime Spike: Discover and define models at runtime #2484
• Unregister runtime artifacts Unregister runtime artifacts #2485

[bajtos]

I am closing the spike as done.