Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: mongoose extension #5201

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions extensions/mongoose/.npmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
package-lock=true
27 changes: 27 additions & 0 deletions extensions/mongoose/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
Copyright (c) IBM Corp. 2020.
Node module: @loopback/mongoose
This project is licensed under the MIT License, full text below.

--------

MIT License

MIT License Copyright (c) IBM Corp. 2020

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice (including the next
paragraph) shall be included in all copies or substantial portions of the
Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
160 changes: 160 additions & 0 deletions extensions/mongoose/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# @loopback/mongoose

## Stability: ⚠️Experimental⚠️

> Experimental packages provide early access to advanced or experimental
> functionality to get community feedback. Such modules are published to npm
> using `0.x.y` versions. Their APIs and functionality may be subject to
> breaking changes in future releases.

[![LoopBack](<https://github.com/strongloop/loopback-next/raw/master/docs/site/imgs/branding/Powered-by-LoopBack-Badge-(blue)[email protected]>)](http://loopback.io/)

## ⚠️Warnings⚠️

As of `0.0.1` of this extension, the type definitions of `@types/mongoose` are
several signifigant minor versions out of date: it supports mongoose `5.7.12`,
and the latest stable version of mongoose is `5.9.10`

## Using your existing Mongoose Schemas in Loopback

Using mongoose to devine your mongodb schemas is relatively straightforward,
though at this early stage it's still a number of steps:

- Import the component, and it's binding:
`import {LoopbackMongooseComponent, MongooseBindings} from '@loopback/mongoose'`

- Register your configuration (see below)
```ts
this.configure(MongooseBindings.COMPONENT).to(configuration);
```
- Add the component to your application
```ts
this.component(LoopbackMongooseComponent);
```
- Finally, `@inject` your Entity Models for use
```ts
public async MyService(@inject('loopback-mongoose-component.model.ModelName') ModelName) {
const result = await ModelName.findOne({});
}
```

## Configuration

There are three pieces of information that the Mongoose Component needs in order
to create a connection to your database and set up the mongoose models:

- the `uri` of the connection
- an array of Schemas
- optionally, an array of Discriminator Schemas

The configuration looks like the following:

```ts
this.configure(MongooseBindings.COMPONENT).to({
uri: 'mongodb://localhost:27017',
connectionOptions: {} // anything valid to mongoose.createConnection(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might consider to use settings from a datasource.

schemas: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's better to declare mongoose.schemas as an extension point to register Mongoose schemas as extensions.

{
bindingKey: 'my-custom.binding.key',
schema: myImportedSchema,
name: 'MyModel'
}
],
discriminators: [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each schema/model itself can be an extension point for its discriminators.

{
bindingKey: 'my-custom.discriminator.binding.key',
modelKey: 'my-custom.binding.key',
schema: myImportedDiscriminatorSchema,
value: 'MyModelDiscriminator'
}
]
})
```

For multiple connections, pass an array instead of an object:

```ts
this.configure(MongooseBindings.COMPONENT).to([
{
uri: 'mongodb://localhost:27017',
schemas: [...]
},
{
uri: 'mongodb://127.0.0.12:27017',
schemas: [...]
Copy link
Contributor

@raymondfeng raymondfeng Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit similar as how we make AJV configurable and extensible. In https://github.com/strongloop/loopback-next/blob/master/packages/rest/src/validation/ajv-factory.provider.ts, we use @inject (or @config) for options, keywords, and formats. Both keywords and formats can be either passed in via options, or their own extension points.

@config({fromBinding: 'datasources.mongodb'})
private connectionOptions;

@extensions('mongoose.schemas')
private mongooseSchemas: Getter<MongooseSchema[]>;

Copy link
Contributor Author

@mschnee mschnee Apr 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The extension system/pattern is going to be new to me: is the linked AVJ code a good example to base off of as a starting point?

The question mark I have is that schemas are orthogonal to connections: a model object represents a schema defined on a specific connection, so schemas can be used across multiple connections. I've seen systems that export all the schemas from an schems/index.js file, and then have a separate connection that only uses a handful of them to create models specific to that connection.

And then there's having both mongoose and typegoose models (two different extensions) use the same mongoose connection. This is actually important to me here because I have an API server that has both mongoose and typegoose models (sharing one mongoose connection so that the references can be properly determined)

My first thought is to change the uri configuration to be

uriOrConnection: string | mongoose.Connection

to allow you to pass an existing connection instead.

Second thought: having a connection manager separate from mongoose/typegoose components but used by both.

Third thought: try to use the mongoose/mongodb internals to determine if the given uri already references a connection. Unlikely since nothing I've seen in the documentation can be used for an exact match- we'd have to make the connection first to determine if we're connecting to the same database.

These patterns are new to me- I'd love to get some direction :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mschnee I have updated the TypeORM extension to leverage the ideas for more flexible configuration. See https://github.com/strongloop/loopback-next/blob/typeorm/extensions/typeorm/src/services/connection-manager.ts.

As we show in https://github.com/strongloop/loopback-next/blob/master/packages/rest/src/validation/ajv-factory.provider.ts as well, the configuration be injected. Extensions can contribute additional pieces to the configuration.

We can build connection option adapter for data source settings such as existing mongodb datasources can be discovered and referenced for mongoose usage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #5801, we are moving to a different direction than presented in https://github.com/strongloop/loopback-next/blob/typeorm/extensions/typeorm/src/services/connection-manager.ts. Instead of keeping a list of connection options at LoopBack side, we are introducing a new application API for registering a new connection and then we keep the actual connection object around.

https://github.com/strongloop/loopback-next/blob/d714638984da4deced7c36b4eb25457492759c69/packages/typeorm/src/mixin.ts#L37-L44

In this new design, users can use regular @inject, @inject.getter etc. decorators to receive connection instances.

Second thought: having a connection manager separate from mongoose/typegoose components but used by both.

Can we use LoopBack's context as the connection manager, the same way as @loopback/repository uses application context to keep track of datasources?

And then there's having both mongoose and typegoose models (two different extensions) use the same mongoose connection.

IIUC, typegoose is using mongoose under the hood, expecting app authors to pick & install a compatible version (mongoose is in peerDependency). I think that means the LoopBack extension should be using mongoose APIs to create connections and then find a way how to use those connections with typegoose models?

The question mark I have is that schemas are orthogonal to connections: a model object represents a schema defined on a specific connection, so schemas can be used across multiple connections. I've seen systems that export all the schemas from an schems/index.js file, and then have a separate connection that only uses a handful of them to create models specific to that connection.

@loopback/repository is already designed this way:

  • a datasource represents a MongoDB connection
  • a model class describes a schema
  • a repository class binds a model with a datasource
  • it's possible for a single model to be attached to multiple datasources

Can you perhaps use a similar design for Mongoose/Typegoose too?

}
])
```

Remember, models are **always** scoped to a single connection. See the
[Mongoose Documentation for Multiple Connections](https://mongoosejs.com/docs/connections.html#multiple_connections)
for more information.

### uri

This is a valid mongodb connection uri string. See the
[MongoDB Documentation](https://docs.mongodb.com/manual/reference/connection-string/)
for more information.

### connectionOptions

_(optional)_

Optionally, these are options that can be passed to the mongoose
`createConnection` function. See the
[Mongoose Documentation](https://mongoosejs.com/docs/api.html#mongoose_Mongoose-createConnection)
for more details.

### Schemas

The Mongoose Component will create the schemas for you. Each item in this array
contains the information necessary to so do:

**schema** The schema class itself, created from `new mongoose.Schema({})`

**name** The model name passed to `connection.model(name, schema)`

**bindingKey** **optional** Optionally, you can create your own binding key. The
default binding key will be `loopback-mongoose-extensions.model.${name}`. If you
have multiple connections, the binding key will include the connection index:
`loopback-mongoose-extension.connection.0.model.${name}`

Using the same binding key for a model on multiple connections is not supported,
and doing so will throw an `InvalidArgumentException`.

## Discriminators

Creating discriminators requires having already created a model to create the
discriminator from. The configuration to do so requires the bindingKey for the
original model, as well as the binding key for the new model, as well as the
name of the discriminator.

**schema**

The schema class itself, created from `new mongoose.Schema({})`

**name**

The model name passed to `model.discriminator(name, schema)`

**modelKey**

This must be the same binding key used for the discriminator base model.

**bindingKey** _(optional)_

Optionally, you can create your own binding key. The default binding key will be
`loopback-mongoose-extensions.model.${name}`. If you have multiple connections,
the binding key will include the connection index:
`loopback-mongoose-extension.connection.0.model.${name}`

**value** **(optional)**

The string stored in the `discriminatorKey` property. See the
[mongoose documentation for discriminators](https://mongoosejs.com/docs/api.html#model_Model.discriminator).

# Using the Mongoose Extension

There is a full REST server used for
[integration tests](./src/__tests__/integration), which contains an example of
how to set up and use mongoose models and discriminators.
6 changes: 6 additions & 0 deletions extensions/mongoose/index.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright IBM Corp. 2020. All Rights Reserved.
// Node module: @loopback/mongoose
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

export * from './dist';
6 changes: 6 additions & 0 deletions extensions/mongoose/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright IBM Corp. 2020. All Rights Reserved.
// Node module: @loopback/mongoose
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

module.exports = require('./dist');
6 changes: 6 additions & 0 deletions extensions/mongoose/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// Copyright IBM Corp. 2020. All Rights Reserved.
// Node module: @loopback/mongoose
// This file is licensed under the MIT License.
// License text available at https://opensource.org/licenses/MIT

export * from './src';
Loading