Multi tenancy support for hibernate search

[edwinmol]

Description

Hibernate search has support for multi tenancy via the config property:

hibernate.search.backend.multi_tenancy.strategy = discriminator

However, this property is not mapped to quarkus configuration for hibernate search.

Quarkus already supports multi tenancy for hibernate ORM and OIDC, seems logical to extend this support to hibernate search.

Implementation ideas

No response

[quarkus-bot]

/cc @gsmet, @yrodiere

[yrodiere]

Note that starting with Hibernate Search 6.1 (currently 6.1.0.Alpha1), multi-tenancy no longer requires explicit settings. So this setting will become all but obsolete.

However, I agree it would be a good idea to add the setting to Quarkus before we upgrade to Hibernate Search 6.1: if nothing else, the integration tests will remain useful after the upgrade.

[edwinmol]

Mmh, happy to see that it will be supported in 6.1. But I think it would be beneficial to support extra tenants just for hibernate search (next to the tenants for OIDC and database).

I'm currently investigating how we can do zero-downtime upgrades (even with schema changes) for a quarkus service that uses hibernate search. Having multi-tenant support could be a solution for this use case.
When upgrading the service in Kubernetes, the new pods can use a new tenant id for hibernate search and do mass indexing on startup. The service itself does not have to be multi-tenant for db and OIDC.

Or is this use case (zero downtime upgrades) supported in another way?
Currently, we plan to use multiple elasticsearch clusters, which is not ideal.

[yrodiere]

That's an interesting use case for multi-tenancy. I can't say I ever thought of it. It feels a bit like a hack as you will bloat your schema and documents with a tenant id just for schema updates, but I suppose it could work.

The major problem with zero-downtime upgrades is the loss of index updates. You will need to make sure old pods are made read-only. Otherwise, index updates from old pods might result in out-of-sync indexes, since these updates will not be propagated to the indexes used by new pods. Another solution would be to perform two full reindexings: one while you're still having old and new pods in your cluster, before exposing the new pods, in order to get a somewhat decent index for your new pods ; and another after eliminating old pods, to make sure to catch up with whatever change the old pods made to the database.

There's a solution to zero-downtime reindexing here in the reference documentation, but it doesn't work well if there are schema upgrades, unless you can 1) make your old pods read-only so they don't try to write documents anymore and 2) accept search query errors for a short time while switching.

I think the best way to address your problem would be to use a custom index layout strategy (it's supported in Quarkus). Just suffix the read/write index aliases of new pods with some application-specific version, e.g. myindex-read-v1.0/myindex-write-v1.0. Then you will be sure that new pods use strictly separate indexes from old pods. But as mentioned above, you will still need to address the problem of old pods that continue to write to the database.

Note that all user-provided components in Hibernate Search can be CDI beans, so you can perfectly well make your custom index layout strategy a CDI bean and inject the version of your application into that bean using a configuration property.

[edwinmol]

Thank you for pointing that out! I think that could work for us.

[gsmet]

@yrodiere is this an issue we should close with 6.1 coming soon?

[yrodiere]

I guess we can leave this here to remind me to at least add an integration test with multi-tenancy + Hibernate Search?