Skip to content

Commit

Permalink
Add Hibernate Search management endpoint
Browse files Browse the repository at this point in the history
- to reindex either all or subset of indexed entities
  • Loading branch information
marko-bekhta committed Sep 21, 2023
1 parent 76348f3 commit 795bba0
Show file tree
Hide file tree
Showing 20 changed files with 954 additions and 1 deletion.
106 changes: 106 additions & 0 deletions docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -1015,6 +1015,112 @@ You can enable AWS request signing in Hibernate Search by adding a dedicated ext
See link:{hibernate-search-orm-elasticsearch-aws-guide}#aws-configuration-reference[the documentation for the Hibernate Search ORM + Elasticsearch AWS extension]
for more information.

[[management]]
== Management endpoint

[CAUTION]
====
Hibernate Search's management endpoint is considered preview.
In _preview_, backward compatibility and presence in the ecosystem is not guaranteed.
Specific improvements might require changing configuration or APIs, or even storage formats,
and plans to become _stable_ are under way.
Feedback is welcome on our https://groups.google.com/d/forum/quarkus-dev[mailing list]

Check warning on line 1028 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1028, "column": 80}}}, "severity": "INFO"}
or as issues in our https://github.com/quarkusio/quarkus/issues[GitHub issue tracker].
====

The Hibernate Search extension provides an HTTP endpoint to reindex your data through the xref:./management-interface-reference.adoc[management interface].

Check warning on line 1032 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'? Raw Output: {"message": "[Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'?", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1032, "column": 50}}}, "severity": "WARNING"}
By default, this endpoint is not available. It can be enabled through configuration properties as shown below.

Check warning on line 1033 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1033, "column": 85}}}, "severity": "INFO"}

[source,properties]
----
quarkus.management.enabled=true <1>
quarkus.hibernate-search-orm.management.enabled=true <2>
----
<1> Enable the xref:./management-interface-reference.adoc[management interface].
<2> Enable Hibernate Search specific management endpoints.

Once the management is enabled, data can be re-indexed via `/q/hibernate-search/reindex`, where `/q` is the default management root path

Check warning on line 1043 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'? Raw Output: {"message": "[Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'?", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1043, "column": 70}}}, "severity": "WARNING"}
and `/hibernate-search` is the default Hibernate Search root management path.
It (`/hibernate-search`) can be changed via configuration property as shown below.

Check warning on line 1045 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsWarnings] Consider using 'through', 'by', 'from', 'on', or 'by using' rather than 'via' unless updating existing content that uses the term. Raw Output: {"message": "[Quarkus.TermsWarnings] Consider using 'through', 'by', 'from', 'on', or 'by using' rather than 'via' unless updating existing content that uses the term.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1045, "column": 30}}}, "severity": "WARNING"}

Check warning on line 1045 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1045, "column": 57}}}, "severity": "INFO"}

[source,properties]
----
quarkus.hibernate-search-orm.management.root-path=custom-root-path <1>
----
<1> Use a custom `custom-root-path` path for Hibernate Search's management endpoint.
If the default management root path is used then the reindex path becomes `/q/custom-root-path/reindex`.

Check warning on line 1052 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'? Raw Output: {"message": "[Quarkus.Spelling] Use correct American English spelling. Did you really mean 'reindex'?", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1052, "column": 43}}}, "severity": "WARNING"}

This endpoint accepts `POST` requests with `application/json` content type only.
All indexed entities will be re-indexed if an empty request body is submitted.

Check warning on line 1055 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.SentenceLength] Try to keep sentences to an average of 32 words or fewer. Raw Output: {"message": "[Quarkus.SentenceLength] Try to keep sentences to an average of 32 words or fewer.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1055, "column": 69}}}, "severity": "INFO"}
If only a subset of entities must be re-indexed or
if there is a need to have a custom configuration of the underlying mass indexer

Check warning on line 1057 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'. Raw Output: {"message": "[Quarkus.Fluff] Depending on the context, consider using 'Rewrite the sentence, or use 'must', instead of' rather than 'need to'.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1057, "column": 4}}}, "severity": "INFO"}
then this information can be passed through the request body as shown below.

Check warning on line 1058 in docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc

View workflow job for this annotation

GitHub Actions / Linting with Vale

[vale] reported by reviewdog 🐶 [Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'. Raw Output: {"message": "[Quarkus.TermsSuggestions] Depending on the context, consider using 'because' or 'while' rather than 'as'.", "location": {"path": "docs/src/main/asciidoc/hibernate-search-orm-elasticsearch.adoc", "range": {"start": {"line": 1058, "column": 51}}}, "severity": "INFO"}

[source,json]
----
{
"filter": {
"types": ["EntityName1", "EntityName2", "EntityName3", ...], <1>
},
"massIndexer":{
"typesToIndexInParallel": 1, <2>
}
}
----
<1> An array of entity names that should be re-indexed. If unspecified or empty, all entity types will be re-indexed.
<2> Sets the number of entity types to be indexed in parallel.

The full list of possible filters and available mass indexer configurations is presented in the example below.

[source,json]
----
{
"filter": { <1>
"types": ["EntityName1", "EntityName2", "EntityName3", ...], <2>
"tenants": ["tenant1", "tenant2", ...] <3>
},
"massIndexer":{ <4>
"typesToIndexInParallel": 1, <5>
"threadsToLoadObjects": 6, <6>
"batchSizeToLoadObjects": 10, <7>
"cacheMode": "IGNORE", <8>
"mergeSegmentsOnFinish": false, <9>
"mergeSegmentsAfterPurge": true, <10>
"dropAndCreateSchemaOnStart": false, <11>
"purgeAllOnStart": true, <12>
"idFetchSize": 100, <13>
"transactionTimeout": 100000, <14>
}
}
----
<1> Filter object that allows to limit the scope of reindexing.
<2> An array of entity names that should be re-indexed. If unspecified or empty, all entity types will be re-indexed.
<3> An array of tenant ids, in case of multi-tenancy. If unspecified or empty, all tenants will be re-indexed.
<4> Mass indexer configuration object.
<5> Sets the number of entity types to be indexed in parallel.
<6> Sets the number of threads to be used to load the root entities.
<7> Sets the batch size used to load the root entities.
<8> Sets the cache interaction mode for the data loading tasks.
<9> Whether each index is merged into a single segment after indexing.
<10> Whether each index is merged into a single segment after the initial index purge, just before indexing.
<11> Whether the indexes and their schema (if they exist) should be dropped and re-created before indexing.
<12> Whether all entities are removed from the indexes before indexing.
<13> Specifies the fetch size to be used when loading primary keys if objects to be indexed.
<14> Specifies the timeout of transactions for loading ids and entities to be re-indexed.
+
Note all the properties in the json are optional, and only those that are needed should be used.

For more detailed information on mass indexer configuration see the
link:{hibernate-search-docs-url}#indexing-massindexer-parameters[corresponding section of the Hibernate Search reference documentation].

Submitting the reindexing request will trigger indexing in the background. Mass indexing progress will appear in the application logs.
For testing purposes, it might be useful to know when the indexing finished. Adding `wait_for=finished` query parameter to the URL
will result in the management endpoint returning a chunked response that will report when the indexing starts and then when it is finished.

When working with multiple persistence units, the name of the persistence unit to reindex can be supplied through the
`persistence_unit` query parameter: `/q/hibernate-search/reindex?persistence_unit=non-default-persistence-unit`.

== Further reading

If you are interested in learning more about Hibernate Search 6,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-dev-ui-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-deployment</artifactId>
<optional>true</optional>
</dependency>

<!-- test dependencies -->
<dependency>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,11 @@
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchBuildTimeConfigPersistenceUnit.ElasticsearchIndexBuildTimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRecorder;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementConfig;
import io.quarkus.runtime.configuration.ConfigUtils;
import io.quarkus.runtime.configuration.ConfigurationException;
import io.quarkus.vertx.http.deployment.NonApplicationRootPathBuildItem;
import io.quarkus.vertx.http.deployment.RouteBuildItem;

@BuildSteps(onlyIf = HibernateSearchEnabled.class)
class HibernateSearchElasticsearchProcessor {
Expand Down Expand Up @@ -435,4 +438,21 @@ void devServicesDropAndCreateAndDropByDefault(
}
}

@Record(ExecutionTime.RUNTIME_INIT)
@BuildStep(onlyIf = HibernateSearchManagementEnabled.class)
void createManagementRoutes(BuildProducer<RouteBuildItem> routes,
NonApplicationRootPathBuildItem nonApplicationRootPathBuildItem,
HibernateSearchElasticsearchRecorder recorder,
HibernateSearchManagementConfig managementConfig) {

routes.produce(nonApplicationRootPathBuildItem.routeBuilder()
.management()
.route(
managementConfig.rootPath() + (managementConfig.rootPath().endsWith("/") ? "" : "/")
+ "reindex")
.routeConfigKey("quarkus.hibernate-search-orm.management.root-path")
.handler(recorder.managementHandler())
.displayOnNotFoundPage()
.build());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
package io.quarkus.hibernate.search.orm.elasticsearch.deployment;

import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchBuildTimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementConfig;

/**
* Supplier that can be used to only run build steps
* if the Hibernate Search extension and its management is enabled.
*/
public class HibernateSearchManagementEnabled extends HibernateSearchEnabled {

private final HibernateSearchManagementConfig config;

HibernateSearchManagementEnabled(HibernateSearchElasticsearchBuildTimeConfig config,
HibernateSearchManagementConfig managementConfig) {
super(config);
this.config = managementConfig;
}

@Override
public boolean getAsBoolean() {
return super.getAsBoolean() && config.enabled();
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,12 @@
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfigPersistenceUnit.ElasticsearchBackendRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.HibernateSearchElasticsearchRuntimeConfigPersistenceUnit.ElasticsearchIndexRuntimeConfig;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.bean.HibernateSearchBeanUtil;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.management.HibernateSearchManagementHandler;
import io.quarkus.hibernate.search.orm.elasticsearch.runtime.mapping.QuarkusHibernateOrmSearchMappingConfigurer;
import io.quarkus.runtime.annotations.Recorder;
import io.quarkus.runtime.configuration.ConfigurationException;
import io.vertx.core.Handler;
import io.vertx.ext.web.RoutingContext;

@Recorder
public class HibernateSearchElasticsearchRecorder {
Expand Down Expand Up @@ -165,6 +168,10 @@ public SearchSession get() {
};
}

public Handler<RoutingContext> managementHandler() {
return new HibernateSearchManagementHandler();
}

private static final class HibernateSearchIntegrationStaticInitInactiveListener
implements HibernateOrmIntegrationStaticInitListener {
private HibernateSearchIntegrationStaticInitInactiveListener() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
package io.quarkus.hibernate.search.orm.elasticsearch.runtime.management;

import io.quarkus.runtime.annotations.ConfigPhase;
import io.quarkus.runtime.annotations.ConfigRoot;
import io.smallrye.config.ConfigMapping;
import io.smallrye.config.WithDefault;

@ConfigMapping(prefix = "quarkus.hibernate-search-orm.management")
@ConfigRoot(phase = ConfigPhase.BUILD_AND_RUN_TIME_FIXED)
public interface HibernateSearchManagementConfig {

/**
* Root path for reindexing endpoints.
* This value will be resolved as a path relative to `${quarkus.management.root-path}`.
*
* @asciidoclet
*/
@WithDefault("hibernate-search/")
String rootPath();

/**
* If management interface is turned on the reindexing endpoints will be published under the management interface.
* This property allows to enable this functionality by setting it to ``true`.
*
* @asciidoclet
*/
@WithDefault("false")
boolean enabled();

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
package io.quarkus.hibernate.search.orm.elasticsearch.runtime.management;

import java.util.Locale;

import io.quarkus.arc.Arc;
import io.quarkus.arc.ManagedContext;
import io.vertx.core.Handler;
import io.vertx.core.http.HttpHeaders;
import io.vertx.core.http.HttpMethod;
import io.vertx.core.http.HttpServerRequest;
import io.vertx.ext.web.RoutingContext;

public class HibernateSearchManagementHandler implements Handler<RoutingContext> {

@Override
public void handle(RoutingContext routingContext) {
ManagedContext requestContext = Arc.container().requestContext();
if (requestContext.isActive()) {
doHandle(routingContext);
} else {
requestContext.activate();
try {
doHandle(routingContext);
} finally {
requestContext.terminate();
}
}
}

private void doHandle(RoutingContext ctx) {
HttpServerRequest request = ctx.request();

if (!HttpMethod.POST.equals(request.method())) {
errorResponse(ctx, 406, "Http method [" + request.method().name() + "] is not supported. Use [POST] instead.");
return;
}

String contentType = request.getHeader(HttpHeaders.CONTENT_TYPE);
if (contentType != null && !contentType.toLowerCase(Locale.ROOT).startsWith("application/json")) {
errorResponse(ctx, 406, "Content type [" + contentType + " is not supported. Use [application/json] instead.");
return;
}

new HibernateSearchPostRequestProcessor().process(ctx);
}

private void errorResponse(RoutingContext ctx, int code, String message) {
ctx.response()
.setStatusCode(code)
.setStatusMessage(message)
.end();
}
}
Loading

0 comments on commit 795bba0

Please sign in to comment.