Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Mongo Panache Extension] Dynamically select database #14789

Closed
ch3rub1 opened this issue Feb 3, 2021 · 14 comments
Closed

[Mongo Panache Extension] Dynamically select database #14789

ch3rub1 opened this issue Feb 3, 2021 · 14 comments

Comments

@ch3rub1
Copy link

ch3rub1 commented Feb 3, 2021

Description
A few month ago, I've submitted an issue regarding the implemention of multitenancy within quarkus mongo panache extension (see #5183).
It seems that quarkus mongo panache extension now support some kinds of multitenancy implementations like multiple mongo client or database selection through @MongoEntity annotation.
However, it seems not possible to implement multitenancy by using a database by tenant (on the same cluster).
For example :

  • A service receive a request triggering some database operations.
  • The targeted tenant name is contained by a HTTP header or a JWT claim.
  • We need to select mongo database matching the tenant name.

I managed to implement this scenario only with mongo client extension by :

  • Creating a request scopped producer which produce an injectable Tenant object, created from the RoutingContext.
  • Creating a request scopped producer which produce an injectable MongoDatabase object, based on the injected Tenant object.
  • Injecting the MongoDatabase object into a request scopped repository. The repository then use this injected database to perform operations to it.

Does this solution can be considered as "good" ? Or do you recommend another approach ?

Is it possible, today, to have similar feature with mongo panache extension ? Or this require an evolution of it ?

Implementation ideas
I saw that some extensions, like oidc or hibernate, offer the possibility to implement a TenantResolver interface. Having such a possibility to configure the mongo client will likely allow to implement this kind of multitenancy.

@ch3rub1 ch3rub1 added the kind/enhancement New feature or request label Feb 3, 2021
@ghost
Copy link

ghost commented Feb 3, 2021

/cc @FroMage, @evanchooly, @loicmathieu

@loicmathieu
Copy link
Contributor

Yes, dynamic selection of MongoClient / MongoDatabase is not possible at the moment.

Having a way to resolve it dynamically, as you proposed with your TenantResolver, could be a good addition but it needs to be designed with care as it needs to interact gracefully with the existing features of MongoDB with Panache. We can inspire ourself with the way tenants works in Hibernate but I'm not even sure it is compatible with Hibernate with Panache.

@ch3rub1
Copy link
Author

ch3rub1 commented Feb 5, 2021

To facilitate the feature integration, we can also imagine a MongoDatabaseResolver instead of a TenantResolver.
With this, developpers could simply provide an implementation of this interface to determine which database will be used by Mongo Panache. These mongo databases can be on the same cluster, on different clusters... developers have the liberty to implement any kind of multitenancy whitout introducing a tenant notion to Mongo Panache extension.

It's just another implementation idea 😃

@m4k4veli
Copy link

m4k4veli commented Mar 4, 2021

I managed to implement this scenario only with mongo client extension by :

  • Creating a request scopped producer which produce an injectable Tenant object, created from the RoutingContext.
  • Creating a request scopped producer which produce an injectable MongoDatabase object, based on the injected Tenant object.
  • Injecting the MongoDatabase object into a request scopped repository. The repository then use this injected database to perform operations to it.

Hi, i was wondering if you could provide a code example on how to implement resolving mongodb database name based on tenant information. I'm currently working on a proof of concept, where having different db is mandatory for each tenant using panache. I'm struggling implementing a solution. Therefore i would be very grateful if you could provide some more information.

@ch3rub1
Copy link
Author

ch3rub1 commented Mar 5, 2021

I'm currently working on a proof of concept, where having different db is mandatory for each tenant using panache.

@m4k4veli I managed to implement this kind of multitenancy only with the mongodb client extension, not with panache.
I'll be glad to provide you a code example but it it'll not help you if you absolutly want to use panache for your proof of concept.
Do you still want it ?

@ch3rub1
Copy link
Author

ch3rub1 commented Mar 5, 2021

@m4k4veli I written a sample quarkus app with mongo multitenancy implemented.
You can find it here

@m4k4veli
Copy link

m4k4veli commented Mar 5, 2021

Many thanks for your effort. This helped me alot, and your solution shows what i expected on how to implement multi-tenancy via mongoclient, at least ss long as panache cannot dynamically select the database.
Props to you!

@Chanizo
Copy link

Chanizo commented May 4, 2022

Hi ! We encountered the same difficulties, is there any news on this issue ?
We need multi tenancy with 1 tenant <=> 1 database.
We removed Panache and used your repository for the moment @ch3rub1, thank you very much !

@pedroh-pereira
Copy link
Contributor

I also have more ideas in mind.
Maybe could exists a way to insert a collection prefix dinamically, because some companies implement multi-tenancy in this way.
For example:
ClientA and ClientB. They will use the same database and the collections could be something like: ClientA.users and ClientB.users

Another approach that I wasn't able to reproduce is mongo client name dynamic selection, we need to pass a fixed database for each MongoEntity. So maybe a MongoClientNameResolver could be implemented too, maybe we can follow the same approach of the PR and on processor we can check if there is a MongoClientNameResolver implementation, all the application.properties connections configs should be added to be lazily produced

@evanchooly
Copy link
Member

IMO, multitenancy should absolutely not be a concern for quarkus/panache. These are ODM concerns and we (well, I ...) really don't want panache to slowly accrete ODM functions when there are functional ODMs out there already.

@pedroh-pereira
Copy link
Contributor

pedroh-pereira commented Nov 10, 2022

@evanchooly I understood your point. So how about do we have at least the capability to dynamically select the database?
I think it’ll be sufficient for the most common cases as mine, for example me and my team are developing some POCs to replace the Spring and win all the amazing features and benefits from Quarkus, currently we are providing a multi-tenancy product, certainly we can provide it using the mongo client, but would be great to use Panache and have something in an out box way. Today, If I’m going to use Panache, I’ll need to create a new extension creating a layer over panache and provide BasePanacheMongoEntity and BasePanacheMongoRepository, because these classes are using a singleton final instance from mongo operations, so we can’t provide a custom MongoOperations implementation.
If the idea is not to bring features like this to panache itself, the possibility to providing a custom MongoOperations implementation may be would be enough.
I know it’s a different case compared to JPA that already have a hibernate supporting it behind the scenes, but maybe it can help many people as me.
What do you think about that?
Thanks for the answer!

@loicmathieu
Copy link
Contributor

We want MongoDB with Panache to keep it simple, but if we can have some kind (not all kind) of multi-tenancy with low cost it's OK for me. Your proposed PR seems to be simple enhought.
But maybe the multi-tenancy should be at MongoDB client side directly (like multi-tenancy is done at Hibernate ORM side not Hibernate with Panache side) as it would also be usable for people using MongoDB without Panache.

@evanchooly I usually agree for advanced features to be for an ODM solution, but multi-tenancy IMHO is not such feature as it is a low level feature to select wich cluster/database/... to use dynamically.
I think it should be supported, at least at the client and/or database side, at the collection side (like prefixing it) it seems more like an ODM feature.

pedroh-pereira added a commit to pedroh-pereira/quarkus that referenced this issue Nov 11, 2022
@evanchooly
Copy link
Member

The PR itself seems reasonably non-invasive though I do agree that it would probably be better to do at the mongo-client level instead. However, we don't really deal with databases there but just the clients themselves so this is probably only really doable inside panache. At a minimum it offloads any "unwanted" complexity to users' code. I left some questions on the PR.

pedroh-pereira added a commit to pedroh-pereira/quarkus that referenced this issue Nov 14, 2022
…of github.com:pedroh-pereira/quarkus into feat/quarkusio#14789-adding-panache-database-resolver
pedroh-pereira added a commit to pedroh-pereira/quarkus that referenced this issue Nov 14, 2022
pedroh-pereira added a commit to pedroh-pereira/quarkus that referenced this issue Nov 14, 2022
loicmathieu added a commit that referenced this issue Jan 12, 2023
…che-database-resolver

MongoDb with Panache: Add multi-tenancy support to MongoDb Panache throught dynamic database selection
@loicmathieu
Copy link
Contributor

Fixed by #29665

@loicmathieu loicmathieu added this to the 2.17 - main milestone Jan 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants