Skip to content

Commit

Permalink
Adds management interface support.
Browse files Browse the repository at this point in the history
It allows exposing selected routes (management routes) to a different HTTP server.
It avoids exposing these management routes on the main HTTP server, which could lead to leaks and undesired access to these endpoints.

Enabling/Disabling the management interface is a build-time property.
However, the interface, port, and SSL... are runtime values.

The management interface is not intended to be used using native transport (as high concurrency is rarely a need for these endpoints).
Also, the access log and same site cookie are not supported yet.

The management interface does not expose plain and secured endpoints.
It's either using HTTP or HTTPS.

At the moment are considered management routes:

* health routes (but not the health-ui)
* prometheus routes
* metrics routes

The management interface is, when enabled, exposed on the port 9000 (9001 in test mode).

When deploying to Kubernetes and Openshift, the `management` port is also exposed.
The Prometheus scrape url and the health checks probes are configured to use that `management` port.

The documentation and configuration javadoc of the SmallRye Metrics, SmallRye health, Micrometer, Vert.x HTTP extensions has been extended to mention the configuration differences when the management interface is enabled.
Typically, the health/metrics root paths are not resolved from the same root.
The non-application endpoint paths are not resolved the same way.

When using the dev ui (old and new) with the management interface enabled, the paths are resolved accordingly (for the health and prometheus extensions).
  • Loading branch information
cescoffier committed Mar 12, 2023
1 parent 3a328dc commit e99c418
Show file tree
Hide file tree
Showing 68 changed files with 3,014 additions and 523 deletions.
2 changes: 1 addition & 1 deletion .github/native-tests.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@
{
"category": "HTTP",
"timeout": 95,
"test-modules": "elytron-resteasy, resteasy-jackson, elytron-resteasy-reactive, resteasy-mutiny, resteasy-reactive-kotlin/standard, vertx, vertx-http, vertx-web, vertx-web-jackson, vertx-graphql, virtual-http, rest-client, rest-client-reactive, rest-client-reactive-stork, rest-client-reactive-multipart, websockets",
"test-modules": "elytron-resteasy, resteasy-jackson, elytron-resteasy-reactive, resteasy-mutiny, resteasy-reactive-kotlin/standard, vertx, vertx-http, vertx-web, vertx-web-jackson, vertx-graphql, virtual-http, rest-client, rest-client-reactive, rest-client-reactive-stork, rest-client-reactive-multipart, websockets, management-interface",
"os-name": "ubuntu-latest"
},
{
Expand Down
12 changes: 11 additions & 1 deletion docs/src/main/asciidoc/http-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,16 @@ As an example, if an extension configures a `service` path, that endpoint will b

The link:https://quarkus.io/blog/path-resolution-in-quarkus/[Path Resolution in Quarkus] blog post further explains how path resolution works for both user and extension defined paths.

[IMPORTANT]
.Management Interface
====
`quarkus.http.root-path` is only used for the main HTTP server.
If you enabled the management interface (using the `quarkus.management.enabled=true` property), you can configure the root path of the management interface using:
`quarkus.management.root-path`.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

[[ssl]]
== Supporting secure connections with SSL

Expand Down Expand Up @@ -261,7 +271,7 @@ include::{generated-dir}/config/quarkus-vertx-http-config-group-filter-config.ad
== Support 100-Continue in vert.x

In order to support `100-continue`, the `quarkus.http.handle-100-continue-automatically` option needs to be enabled explicitly
For additional information check https://datatracker.ietf.org/doc/html/rfc7231#section-5.1.1[100-continue= and the related
For additional information check https://datatracker.ietf.org/doc/html/rfc7231#section-5.1.1[100-continue] and the related
https://vertx.io/docs/apidocs/io/vertx/core/http/HttpServerOptions.html#DEFAULT_HANDLE_100_CONTINE_AUTOMATICALLY[Vert.x documentation].

[source,properties]
Expand Down
180 changes: 180 additions & 0 deletions docs/src/main/asciidoc/management-interface-reference.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Management interface reference
include::_attributes.adoc[]
:categories: observability
:summary: Management interface configuration
:numbered:
:sectnums:
:sectnumlevels: 4

By default, Quarkus exposes the _management_ endpoints under `/q` on the main HTTP server.
The same HTTP server provides the application endpoints and the management endpoints.

This document presents how you can use a separate HTTP server (bound to a different network interface and port) for the management endpoints.
It avoids exposing these endpoints on the main server and, therefore, prevents undesired accesses.

== Enabling the management interface

To enable the management interface, use the following **build-time** property:

[source, properties]
----
quarkus.management.enabled=true
----

By default, management endpoints will be exposed on: `http://0.0.0.0:9000/q`.
For example, `http://0.0.0.0:9000/q/health/ready` for the readiness probe.

SmallRye Health Checks, SmallRye Metrics, and Micrometer endpoints will be declared as management endpoints when the management interface is enabled.

== Configure the host, port and scheme

By default, the management interface is exposed on the interface: `0.0.0.0` (all interfaces) and on the port `9000` (`9001` in test mode).
It does not use TLS (`https`) by default.

You can configure the host, ports, and TLS certificates using the following properties:

* `quarkus.management.host` - the interface / host
* `quarkus.management.port` - the port
* `quarkus.management.test-port` - the port to use in test mode
* `quarkus.management.ssl` - the TLS configuration, xref:http-reference#ssl[same as for the main HTTP server].

Here is a configuration example exposing the management interface on _https://localhost:9002_:

[source, properties]
----
quarkus.management.enabled=true
quarkus.management.host=localhost
quarkus.management.port=9002
quarkus.management.ssl.certificate.key-store-file=server-keystore.jks
quarkus.management.ssl.certificate.key-store-password=secret
----

IMPORTANT: Unlike the main HTTP server, the management interface does not handle _http_ and _https_ at the same time.
If _https_ is configured, plain HTTP requests will be rejected.

== Configure the root path

Management endpoints are configured differently than standard HTTP endpoints.
They use a unique root path, which is `/q` by default.
This management root path can be configured using the `quarkus.management.root-path property`.
For example, if you want to expose the management endpoints under `/management` use:

[source, properties]
----
quarkus.management.enabled=true
quarkus.management.root-path=/management
----

The mounting rules of the management endpoints slightly differ from the ones used when using the main HTTP server:

* Management endpoints configured using a _relative_ path (not starting with `/`) will be served from the configured root path.
For example, if the endpoint path is `health` and the root path is `management`, the resulting path is `/management/health`
* Management endpoints configured using an _absolute_ path (starting with `/`) will be served from the root.
For example, if the endpoint path is `/health`, the resulting path is `/health`, regardless of the root path
* The management interface does not use the HTTP root path from the main HTTP server.

[IMPORTANT]
====
The `quarkus.http.root-path` property is only applied to the main HTTP server and not to the management interface.
In addition, the `quarkus.http.non-application-root-path` property is not used for endpoint exposed on the management interface.
====

== Create a management endpoint

SmallRye Health Checks, SmallRye Metrics, and Micrometer endpoints will be declared as management endpoints when the management interface is enabled.

NOTE: if you do not enable the management interface, these endpoints will be served using the main HTTP server (under `/q` by default).

Extensions can create a management endpoint by defining a _non application_ route and calling `management()` method:

[source, java]
----
@BuildStep
void createManagementRoute(BuildProducer<RouteBuildItem> routes,
NonApplicationRootPathBuildItem nonApplicationRootPathBuildItem,
MyRecorder recorder) {
routes.produce(nonApplicationRootPathBuildItem.routeBuilder()
.management() // Must be called BEFORE the routeFunction method
.routeFunction("my-path", recorder.route())
.handler(recorder.getHandler())
.blockingRoute()
.build());
//...
}
----

If the management interface is enabled, the endpoint will be exposed on: `http://0.0.0.0:9000/q/my-path`.
Otherwise, it will be exposed on: `http://localhost:8080/q/my-path`.

IMPORTANT: Management endpoints can only be declared by extensions and not from the application code.

== Management Interface Configuration

include::{generated-dir}/config/quarkus-management-management-management-interface-build-time-config.adoc[leveloffset=+1, opts=optional]

include::{generated-dir}/config/quarkus-management-management-management-interface-configuration.adoc[leveloffset=+1, opts=optional]


[[reverse-proxy]]
== Running behind a reverse proxy


Quarkus can be accessed through proxies that generate headers (e.g. `X-Forwarded-Host`) to preserve information about the original request.
Quarkus can be configured to automatically update information like protocol, host, port and URI to use the values from those headers.

IMPORTANT: Activating this feature can expose the server to security issues like information spoofing.
Activate it only when running behind a reverse proxy.

To set up this feature for the management interface, include the following lines in `src/main/resources/application.properties`:
[source,properties]
----
quarkus.management.proxy.proxy-address-forwarding=true
----

To constrain this behavior to the standard `Forwarded` header (and ignore `X-Forwarded` variants) by setting `quarkus.management.proxy.allow-forwarded` in `src/main/resources/application.properties`:
[source,properties]
----
quarkus.management.proxy.allow-forwarded=true
----

Alternatively, you can prefer `X-Forwarded-*` headers using the following configuration in `src/main/resources/application.properties` (note `allow-x-forwarded` instead of `allow-forwarded`):
[source,properties]
----
quarkus.management.proxy.proxy-address-forwarding=true
quarkus.management.proxy.allow-x-forwarded=true
quarkus.management.proxy.enable-forwarded-host=true
quarkus.management.proxy.enable-forwarded-prefix=true
----

Supported forwarding address headers are:

* `Forwarded`
* `X-Forwarded-Proto`
* `X-Forwarded-Host`
* `X-Forwarded-Port`
* `X-Forwarded-Ssl`
* `X-Forwarded-Prefix`

If both header variants (`Forwarded` and `X-Forwarded-*`) are enabled, the `Forwarded` header will have precedence.

IMPORTANT: Using both `Forwarded` and `X-Forwarded` headers can have security implications as it may allow clients to forge requests with a header that is not overwritten by the proxy.

Ensure that your proxy is configured to strip unexpected `Forwarded` or `X-Forwarded-*` headers from the client request.

== Kubernetes

When Quarkus generates the Kubernetes metadata, it checks if the management interface is enabled and configures the probes accordingly.
The resulting descriptor defines the main HTTP port (named `http`) and the management port (named `management`).
Health probes (using HTTP actions) and Prometheus scrape URLs are configured using the `management` port.

[IMPORTANT]
.KNative
====
Until https://github.com/knative/serving/issues/8471[KNative#8471] is resolved, you cannot use the management interface, as KNative does not support containers will multiple exposed ports.
====
30 changes: 30 additions & 0 deletions docs/src/main/asciidoc/micrometer.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,15 @@ When should you use a Gauge? Only if you can't use something else. Never gauge s
less straight-forward to use than counters. If what you are measuring can be counted (because the value always
increments), use a counter instead.

[NOTE]
.Management interface
====
By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

=== Counters

Counters are used to measure values that only increase. In the example below, you will count the number of times you
Expand Down Expand Up @@ -598,6 +607,27 @@ implementation("org.eclipse.microprofile.metrics:microprofile-metrics-api")

NOTE: The MP Metrics API compatibility layer will be moved to a different extension in the future.

== Management interface

By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by setting `quarkus.management.enabled=true` in your application configuration.
Note that this property is a build-time property.
The value cannot be overridden at runtime.

If you enable the management interface without customizing the management network interface and port, the metrics are exposed under: `http://0.0.0.0:9000/q/metrics`.
You can configure the path of each exposed format using:
[source, properties]
----
quarkus.micrometer.export.json.enabled=true # Enable json metrics
quarkus.micrometer.export.json.path=metrics/json
quarkus.micrometer.export.prometheus.path=metrics/prometheus
----

With such a configuration, the json metrics will be available from `http://0.0.0.0:9000/q/metrics/json`.
The prometheus metrics will be available from `http://0.0.0.0:9000/q/metrics/prometheus`.

Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.

== Configuration Reference

include::{generated-dir}/config/quarkus-micrometer.adoc[opts=optional, leveloffset=+1]
21 changes: 21 additions & 0 deletions docs/src/main/asciidoc/smallrye-health.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,15 @@ The general `status` of the health check is computed as a logical AND of all the
declared health check procedures. The `checks` array is empty as we have not specified
any health check procedure yet so let's define some.

[NOTE]
.Management interface
====
By default, the health checks are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

== Creating your first health check

In this section, we create our first simple health check procedure.
Expand Down Expand Up @@ -410,6 +419,18 @@ The Quarkus `smallrye-health` extension ships with `health-ui` and enables it by

image:health-ui-screenshot01.png[alt=Health UI]

== Management interface

By default, the health checks are exposed on the main HTTP server.
You can expose them on a separate network interface and port by setting `quarkus.management.enabled=true` in your application configuration.
Note that this property is a build-time property.
The value cannot be overridden at runtime.

If you enable the management interface without customizing the management network interface and port, the health checks are exposed under: `http://0.0.0.0:9000/q/health`.
You can configure the _path_ (the `health` segment in the previous URL) using the `quarkus.smallrye-health.root-path` property.

Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.

== Conclusion

SmallRye Health provides a way for your application to distribute information
Expand Down
9 changes: 9 additions & 0 deletions docs/src/main/asciidoc/smallrye-metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,15 @@ You will receive a response such as:

NOTE: If you prefer an OpenMetrics export rather than the JSON format, remove the `-H"Accept: application/json"` argument from your command line.

[NOTE]
.Management interface
====
By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

.Configuration Reference

include::{generated-dir}/config/quarkus-smallrye-metrics.adoc[opts=optional, leveloffset=+1]
Original file line number Diff line number Diff line change
Expand Up @@ -131,4 +131,4 @@ public void postBuild(ContainerImageInfoBuildItem image, List<ContainerImageBuil
//So, we now always perform this step
ExecUtil.exec("kind", "load", "docker-image", image.getImage());
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -118,4 +118,4 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
startupPath,
roles, roleBindings, customProjectRoot);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
/**
* A build item for selecting which port to use for probes using an {@literal HTTP get} action.
*/
public class KubernetesProbePortNameBuildItem extends SimpleBuildItem {
public final class KubernetesProbePortNameBuildItem extends SimpleBuildItem {

private final String name;

Expand Down
4 changes: 4 additions & 0 deletions extensions/kubernetes/vanilla/deployment/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-deployment-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-client-internal-deployment</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,4 @@ private static int getStablePortNumberInRange(String input, int min, int max) {
throw new RuntimeException("Unable to generate stable port number from input string: '" + input + "'", e);
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -367,4 +367,4 @@ private static List<DecoratorBuildItem> createVolumeDecorators(Optional<Project>
});
return result;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -809,4 +809,4 @@ private static Map<String, Integer> verifyPorts(List<KubernetesPortBuildItem> ku
}
return result;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -372,4 +372,4 @@ void externalizeInitTasks(
decorators);
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -286,4 +286,4 @@ void externalizeInitTasks(
decorators);
}
}
}
}
Loading

0 comments on commit e99c418

Please sign in to comment.