Multitenancy with mongodb panache extension

[ch3rub1]

Description
We're building an app which implement multitenancy by having different mongo databases.

Currently, there seems to be no way to specify which database should be used by PanacheMongoRepository implementations depending on dynamic values like http requests headers or path params.

Currently, since Panache get the database value by calling ConfigProvider.getConfig(), the only workaround we see is to implement a custom microprofile ConfigSource to provide database value depending on ThreadLocal content set by a request filter earlier.

Is there any better way to do it or do you plan to implement this feature in future releases ?

Implementation ideas
Injecting a custom database provider into PanacheMongoRepository implementations to resolve database to use at runtime may be a way to resolve this issue.

Thanks for your input on the matter.

[machi1990]

@loicmathieu this one is for you.

[loicmathieu]

@machi1990 there is two ways to solve this issue:

1. Providing a MongoDatabaseResolver as suggested
2. Providing a way to resolve different values for a config property based on a ThreadLocal value

As we rely on Vert.x i don't know if the second is easily feasible, ping @cescoffier, anyway this will impact the config resolution and I don't know if something for this already exist for MP Config.

If we implement the first aproach, at some point we will want to implement the same for Hibernate with Panache, so maybe @FroMage have some feedback on this ?

There is also an open PR for multi-database support for MongoDB (#3343) , these two issues are orthogonal but must works together

[FroMage]

Do you need one database per entity class or multiple databases per entity class?

[ch3rub1]

We need to be able to define, dynamically, the database on which an entity will be persisted to.

Database selection depends on a request context like http header ("X-Tenant" for instance) or URI path param ("/api/v1/{tenant}/...").

Having a mechanism like our missed KeycloakConfigResolver to resolve database depending on request would be great 😃

[FroMage]

OK, this is more than #3343 because the choice of DB is dynamic. Pretty sure the list of DBs is static, though, right? The client can't point you to a DB you haven't set up in configuration, right?

So this question is then more general and could apply to ORM as well @Sanne @emmanuelbernard and in general I guess the way I would do it is via a filter or interceptor that we could make extendable which would override the DataSource/EntityManager/MongoDbClient that would be otherwise injected in the current request.

We need a new API for that. And even if it's not shareable by ORM/MongoDb, it would be very similar, so needs to be coordinated.

[Sanne]

Funny I was having similar thoughts yesterday regarding Hibernate ORM multitenancy.

In Hibernate / upstream, the capability to use a "dynamically selected" database is actually popular; the way it works is there's a SPI and the user needs to inject his custom "switch" implementation during bootup.

We can also pick up such an implementation by classname / configuration but as far as I can see it's easiest to inject the constructed instance, so that it can include references to datasources and whatever else is useful to it. I'd guess this would be the easiest approach in Quarkus as well, especially considering we have no JNDI.

Multitenancy is a very overloaded term and can be used in multiple ways; only some need to have a different Datasource configuration; take for example the "schema" or "role" aka "username" in some databases: you'd share the same DS configuration properties (even same pool), and yet such a custom implementation could switch user and schema names dinamically, including to a schema or user which was not known at boottime.

Quarkus doesn't support these modules yet, personally I think we'll need to make some prototypes.

[loicmathieu]

For DataSources, I implement a routing datasource once, that was configured with a table with routing keys and datasource names, and delegate to the right datasource bean based on a routing criteria.
For this to works every datasources needs to be CDI beans + the routing datasource.

I remember that the configuration part (in XML) was a mess ;)

So first, we need a multi-database/multi-mongo/... implementation, it is already done for Agroal and WIP for MongoDB.
Then some kind of provider that will be used instead of the direct call to CDI to retrieve the datasource/mongodb client.
Then a way to describe the routing table.
Finally, a way to the user to set the routing criteria and retrieve the right bean from CDI.

Example configuration:

quarkus.mongodb.db1.connection-string=localhost:2707
quarkus.mongodb.db2.connection-string=localhost:2708
quarkus.mongodb.routing-criterias=KEY1:db1,KEY2:db2

Example RoutingCriteriaResolver bean:

@ApplicationScopped
public class CustomRoutingCriteriaResolver implements RoutingCriteriaResover {
    @Override
   public String routingCriteria(HttpServletRequest req) { //wich context should be allowed here ? At least some HTTP req ?
       return req.getHeader("X-Routing-Context") == "CXT-1");
   }
}

Then at the injection points we have three options:

1. Injecting it magically only with @Inject
2. Injecting it with @Inject and a qualifier annotation @Routing
3. Injecting a provider @Inject RoutingMongoClientProvider mongoClientProvider

This will be where the implementation part will be the more complex I think :)

[Sanne]

yes using a CDI bean looks like the natural choice.

Having a routing criteria return a String might not suit all needs though; for example in ORM it could be a schema name, or it could be a name of something else like the role. In Hibernate ORM the tenantId is a String type, so having such a RoutingCriteriaResolver could be easily plugging in, but :

• it needs to be coupled with a custom org.hibernate.engine.jdbc.connections.spi.MultiTenantConnectionProvider as beyond having the Id there's multiple strategies to choose among
• we'd need a way to know which RoutingCriteriaResover to be applied to each SessionFactory / EntityManagerFactory; not supporting more than one today but we hope to resolve that soon.

So +1 for the API proposal, what you posted id probably the core for it.

When it comes to allow people to configure their custom MultiTenantConnectionProvider I would suppose that could also be defined as a CDI bean; in that case I'd have people define any configuration property they need as a custom application config entry though. So their bean would inject whatever else they need to configure it, rather than us pre-defining which configuration keys to use. But that's possibly specific to Hibernate's need.

[ch3rub1]

Multitenancy is a very overloaded term and can be used in multiple ways; only some need to have a different Datasource configuration; take for example the "schema" or "role" aka "username" in some databases: you'd share the same DS configuration properties (even same pool), and yet such a custom implementation could switch user and schema names dinamically, including to a schema or user which was not known at boottime.

The approche described above is exactly what we looking for.
For now, we only have one datasource but we need to switch schema (= mongodb database) dynamicaly, even if the schema/username/database is not known at boot time (mongodb client creates a database at first use).

[loicmathieu]

@realDrCastafolte MongoDB with Panache currently only allow to use multiple database on the same MongoDB client (so on the same MongoDB clsuster).

Multiple MongoDB Client with Quarkus just lands in thanks to #4529, I will now works on integrating this functionality inside MongoDB with Panache, so stay tuned ;)