Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(mongodb): MongoDB Aggregation improvements #3366

Merged
merged 26 commits into from
May 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2698b30
Update docs
DaddyWarbucks Dec 1, 2023
8a4840f
Allow _get method to use pipeline
DaddyWarbucks Dec 2, 2023
39ac4a1
Add comment
DaddyWarbucks Dec 2, 2023
a2a1d55
Disallow pipeline with mongodb param
DaddyWarbucks Dec 2, 2023
47962fb
Use findOneAndReplace in _update method
DaddyWarbucks Dec 5, 2023
39b0387
Use findOneAndUpdate in patch method
DaddyWarbucks Dec 5, 2023
4795929
Allow $sort with _delete method
DaddyWarbucks Dec 5, 2023
bf252e5
Allow pipeline and better mongo methods in _delete
DaddyWarbucks Dec 5, 2023
8a49c54
Sort before select in pipeline
DaddyWarbucks Dec 5, 2023
60c1819
USe mongo projection in create instead of common select
DaddyWarbucks Dec 5, 2023
ec45ecf
Simplify getProjection method
DaddyWarbucks Dec 5, 2023
c0e3c05
Start working on pipeline count, add pipeline to _update
DaddyWarbucks Dec 5, 2023
55100a4
Improve _update and _get, add tests
DaddyWarbucks Dec 5, 2023
d0b5b11
Cleanup formatting
DaddyWarbucks Dec 5, 2023
ca59178
Better pipeline in _patch
DaddyWarbucks Dec 5, 2023
f77ff98
Count documents
DaddyWarbucks Dec 9, 2023
64ee399
Cleanup comments
DaddyWarbucks Dec 9, 2023
ced2cdc
Fix count and pagination
DaddyWarbucks Jan 24, 2024
06790a0
Add tests
DaddyWarbucks Jan 24, 2024
cd067fd
Use aggregation in create
DaddyWarbucks Mar 31, 2024
a7f683d
Update docs
DaddyWarbucks Mar 31, 2024
4f52ad8
Add test
DaddyWarbucks Mar 31, 2024
cf0c9d5
Fix typos
DaddyWarbucks Apr 18, 2024
dd6fcc4
Add $sort/$limit tests
DaddyWarbucks Apr 25, 2024
3908eee
Remove unused method.
DaddyWarbucks Apr 26, 2024
4be451b
Fix tests
DaddyWarbucks May 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions docs/api/databases/mongodb.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,7 @@ MongoDB adapter specific options are:

The [common API options](./common.md#options) are:

- `id {string}` (_optional_, default: `'_id'`) - The name of the id field property. By design, MongoDB will always add an `_id` property.
- `id {string}` (_optional_) - The name of the id field property (usually set by default to `id` or `_id`).
- `id {string}` (_optional_, default: `'_id'`) - The name of the id field property. By design, MongoDB will always add an `_id` property. But you can choose to use a different property as your primary key.
- `paginate {Object}` (_optional_) - A [pagination object](#pagination) containing a `default` and `max` page size
- `multi {string[]|boolean}` (_optional_, default: `false`) - Allow `create` with arrays and `patch` and `remove` with id `null` to change multiple items. Can be `true` for all methods or an array of allowed methods (e.g. `[ 'remove', 'create' ]`)

Expand All @@ -79,31 +78,31 @@ There are additionally several legacy options in the [common API options](./comm

### aggregateRaw(params)

The `find` method has been split into separate utilities for converting params into different types of MongoDB requests. By default, requests are processed by this method and are run through the MongoDB Aggregation Pipeline. This method returns a raw MongoDB Cursor object, which can be used to perform custom pagination or in custom server scripts, if desired.
The `find` method has been split into separate utilities for converting params into different types of MongoDB requests. When using `params.pipeline`, the `aggregateRaw` method is used to convert the Feathers params into a MongoDB aggregation pipeline with the `model.aggregate` method. This method returns a raw MongoDB Cursor object, which can be used to perform custom pagination or in custom server scripts, if desired.

### findRaw(params)

`findRaw(params)` is used when `params.mongodb` is set to retrieve data using `params.mongodb` as the `FindOptions` object. This method returns a raw MongoDB Cursor object, which can be used to perform custom pagination or in custom server scripts, if desired.
`findRaw(params)` This method is used when there is no `params.pipeline` and uses the common `model.find` method. It returns a raw MongoDB Cursor object, which can be used to perform custom pagination or in custom server scripts, if desired.

### makeFeathersPipeline(params)

`makeFeathersPipeline(params)` takes a set of Feathers params and converts them to a pipeline array, ready to pass to `collection.aggregate`. This utility comprises the bulk of the `aggregateRaw` functionality, but does not use `params.pipeline`.
`makeFeathersPipeline(params)` takes a set of Feathers params and converts them to a pipeline array, ready to pass to `model.aggregate`. This utility comprises the bulk of the `aggregateRaw` functionality, but does not use `params.pipeline`.

### Custom Params

The `@feathersjs/mongodb` adapter utilizes two custom params which control adapter-specific features: `params.pipeline` and `params.mongodb`.
The `@feathersjs/mongodb` adapter utilizes three custom params which control adapter-specific features: `params.pipeline`, `params.mongodb`, and `params.adapter`.

#### params.adapter

Allows to dynamically set the [adapter options](#options) (like the `Model` collection) for a service method call.

#### params.pipeline

Used for [aggregation pipelines](#aggregation-pipeline).
Used for [aggregation pipelines](#aggregation-pipeline). Whenever this property is set, the adapter will use the `model.aggregate` method instead of the `model.find` method. The `pipeline` property should be an array of [aggregation stages](https://www.mongodb.com/docs/manual/reference/operator/aggregation-pipeline/).

#### params.mongodb

When making a [service method](/api/services.md) call, `params` can contain an`mongodb` property (for example, `{upsert: true}`) which allows modifying the options used to run the MongoDB query. The adapter will use the `collection.find` method and not the [aggregation pipeline](#aggregation-pipeline) when you use `params.mongodb`.
When making a [service method](/api/services.md) call, `params` can contain an`mongodb` property (for example, `{ upsert: true }`) which allows modifying the options used to run the MongoDB query. This param can be used for both find and aggregation queries.

## Transactions

Expand Down Expand Up @@ -198,17 +197,17 @@ See the MongoDB documentation for instructions on performing full-text search us

## Aggregation Pipeline

In Feathers v5 Dove, we added support for the full power of MongoDB's Aggregation Framework and blends it seamlessly with the familiar Feathers Query syntax. All `find` queries now use the Aggregation Framework, by default.
In Feathers v5 Dove, we added support for the full power of MongoDB's Aggregation Framework and blends it seamlessly with the familiar Feathers Query syntax. The `find` method automatically uses the aggregation pipeline when `params.pipeline` is set.

The Aggregation Framework is accessed through the mongoClient's `collection.aggregate` method, which accepts an array of "stages". Each stage contains an operator which describes an operation to apply to the previous step's data. Each stage applies the operation to the results of the previous step. It’s now possible to perform any of the [Aggregation Stages](https://www.mongodb.com/docs/upcoming/reference/operator/aggregation-pipeline/) like `$lookup` and `$unwind`, integration with the normal Feathers queries.
The Aggregation Framework is accessed through the mongoClient's `model.aggregate` method, which accepts an array of "stages". Each stage contains an operator which describes an operation to apply to the previous step's data. Each stage applies the operation to the results of the previous step. It’s now possible to perform any of the [Aggregation Stages](https://www.mongodb.com/docs/upcoming/reference/operator/aggregation-pipeline/) like `$lookup` and `$unwind`, integration with the normal Feathers queries.

Here's how it works with the operators that match the Feathers Query syntax. Let's convert the following Feathers query:

```ts
const query = {
text: { $regex: 'feathersjs', $options: 'igm' },
$sort: { createdAt: -1 },
$skip: 0,
$skip: 20,
$limit: 10
}
```
Expand Down
Loading
Loading