Enable metrics in vert.x extension

[volalex]

It would be nice if quarkus will provide a vert.x metrics in MP Vendor metrics. For now vert.x metrics are disabled.

[gsmet]

@cescoffier this one looks like a good issue for beginners but we would need more pointers.

[jmartisk]

I guess I could have a look at this, after #5272 is finished and merged, because the work here will need to based on that PR

[jmartisk]

I briefly started analysing this even though the PR with general metric infrastructure is not yet merged, to see what issues will arise. I was looking at metrics using the Micrometer module: https://vertx.io/blog/eclipse-vert-x-metrics-now-with-micrometer-io/
TLDR: it's a bit more complicated than expected, but not impossible. Will definitely need more thinking.

One issue to resolve is that the Metrics extension itself depends on quarkus-vertx-http, so making quarkus-vertx-http (or -core) depend on the Metrics extension (which we need to expose Metrics) would cause a cyclic dependency. I guess we could solve it by introducing a new extension, for example quarkus-vertx-metrics, that would depend on quarkus-smallrye-metrics and therefore also quarkus-vertx-http and would provide the metrics. The problem with this is that it will not be quite consistent with our approach for other extensions, because to enable Vert.x metrics, the user will have to import a specific extension just for that, other than simply enable it by a config property.

Another problem is that there are currently two Vert.x instances in Quarkus, each provides a different set of metrics. It is a work-in-progress to merge them into one. If we wanted to implement this now, we would probably need to expose metrics from each Vert.x instance separately, so it would probably be better to wait until they are merged before we implement the metrics.

Another problem is that some metrics are generated dynamically after they are first measured and so the complete list is not available in advance, so we can't initialize all of them in STATIC_INIT. We will need to enhance the general metric mechanism with something that will pick up new metrics at runtime whenever they appear in the Vert.x Micrometer registry. Basically something that will mirror the Micrometer registry into our usual metric registry. Or we could aggregate multi-dimensional metrics into one (for example, we could expose transform all metrics like {name='vertx.http.server.requests', tags=[tag(method=GET)]} for all different method values into just one metric named vertx.http.server.requests without a method tag - this way we will know in advance that we can register this metric in STATIC_INIT, but we lose a lot of granularity of the data.

Another problem is that it seems that metrics are not collected at all by Vert.x Micrometer Metrics unless there is a backend defined (which can either be Prometheus, JMX or InfluxDB). If none of these backends is defined, the values are all zero: https://github.com/vert-x3/vertx-micrometer-metrics/blob/3.8.4/src/main/java/io/vertx/micrometer/backends/BackendRegistries.java#L76 We don't need any of these backends though, because we just want to programmatically access the values and feed them into our own mechanisms. Joel's blog post (linked in the first paragraph) says that collecting metrics should be possible without any backend configured, but for some reason it doesn't work for me until I configure Prometheus backend. Maybe it is connected to the fact that we have two Vert.x instances that clash over the same static registry. I'll look more into it.

[ebullient]

Jan, you'll want to emulate what I did for the micrometer metrics extension (to that end, it should stay w/in the smallrye metrics extension as an MP-metrics-specific thing): https://github.com/ebullient/quarkus-micrometer-extension/tree/master/runtime/src/main/java/dev/ebullient/micrometer/runtime/binder/vertx

I haven't added support for all MetricsOptions types, but I think we can both do this pretty easily.

[cescoffier]

That would be awesome @ebullient !

[jmartisk]

@ebullient very cool, thanks. I'll put this back on the TODO list and hopefully get round to this soon

[ebullient]

@ebullient very cool, thanks. I'll put this back on the TODO list and hopefully get round to this soon

let me know if you have questions about what I did...it ended up being pretty straightforward (I think?)

@cescoffier, I wanted to check with you at some point to see what else would be needed for reactive work. As mentioned, I only emit Http server metrics right now (in a format that integrates seamlessly with http dimensional data gathered by spring and micronaut). I suppose a conversation about what metrics are gathered and which tags should be used should happen sometime soon?

[cescoffier]

@ebullient We would need to expose the metrics of every Vert.x client (NetClient (TCP), Web Client, database client). I would also add event bus metrics.

[kenfinnigan]

I might be off base, but do we even want to add Vert.x metrics with SmallRye Metrics extension, or only have it active when using Micrometer extension?

[cescoffier]

It would be easier with micrometer. Otherwise, we would have to implement the vertx metrics SPI.

[jmartisk]

The Micrometer extension implements that SPI anyway (see https://github.com/quarkusio/quarkus/blob/1.8.0.Final/extensions/micrometer/runtime/src/main/java/io/quarkus/micrometer/runtime/binder/vertx/VertxHttpServerMetrics.java), so the Smallrye extension could basically copy that. If it is worth doing, I don't know, I have it on my radar, but with a low priority.

[cescoffier]

@jmartisk Should we close this issue as I would consider it done.

[jmartisk]

That depends if we're fine that it only works with the Micrometer extension. But as we're planning to push that one as the 'preferred' metrics extension, then I guess it shouldn't be considered a big problem if the SmallRye extension does not support it.
If someone is willing to implement it, I think it will be welcome, but to myself the priority of working on this is extemely low, given the circumstances.
Let's close it then.

[cescoffier]

I agree, @jmartisk. Micrometer support is good enough for now.

[himanshumps]

@cescoffier @jmartisk @ebullient

I am not able to get it to work with quarkus (I am using 2.5.0.Final release)

pom.xml

    <dependencies>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-resteasy</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-vertx-http-deployment</artifactId>
            <version>${quarkus.platform.version}</version>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-picocli</artifactId>
        </dependency>
        <dependency>
            <groupId>io.smallrye.reactive</groupId>
            <artifactId>smallrye-mutiny-vertx-web-client</artifactId>
        </dependency>
        <dependency>
            <groupId>io.smallrye.reactive</groupId>
            <artifactId>smallrye-mutiny-vertx-micrometer-metrics</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-micrometer-registry-prometheus</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-micrometer-deployment</artifactId>
        </dependency>
        <dependency>
            <groupId>io.quarkiverse.micrometer.registry</groupId>
            <artifactId>quarkus-micrometer-registry-azure-monitor</artifactId>
            <version>2.3.0</version>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-tcnative-boringssl-static</artifactId>
            <version>2.0.46.Final</version>
            <scope>compile</scope>
            <optional>false</optional>
        </dependency>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-transport-native-epoll</artifactId>
            <classifier>linux-x86_64</classifier>
            <scope>compile</scope>
            <optional>false</optional>
        </dependency>
    </dependencies>

application.properties

quarkus.micrometer.registry-enabled-default=true
quarkus.micrometer.binder-enabled-default=true
quarkus.micrometer.binder.vertx.enabled=true
quarkus.micrometer.binder.mp-metrics.enabled=true
quarkus.micrometer.binder.http-client.enabled=true

I need Webclient and other metrics. I am using the mutiny version of webclient.

[cescoffier]

I don't believe the web client is supported, but the vertx http client (used by the web client) should be there.

[ebullient]

What does smallrye-mutiny-vertx-micrometer-metrics do? Quarkus does not use the Vert.x Micrometer library/binding, as it has its own. If it is adding pure Micrometer API calls in the right place, all will be well. If it is expecting to do something with VertxMetrics / VertxOptions that would have an effect later, well.. it may not work like you're expecting.

(I can't understand what that entire project is doing.. holiday brain)

[himanshumps]

@cescoffier @ebullient @kenfinnigan @jmartisk @gsmet

I removed the dependency for smallrye-mutiny-vertx-micrometer-metrics but still no changes.

I am still not able to set the metrics. I tried doing this but looks like it is picked up late in the lifecycle

package com.test;

import io.micrometer.core.instrument.MeterRegistry;
import io.quarkus.arc.DefaultBean;
import io.vertx.core.VertxOptions;
import io.vertx.micrometer.MicrometerMetricsOptions;

import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.context.Dependent;
import javax.enterprise.inject.Produces;
import java.util.function.Consumer;

@Dependent
public class VertxOptionsCustomizerBean {

  MeterRegistry registry;

  public VertxOptionsCustomizerBean(MeterRegistry registry) {
    System.out.println("Inside the VertxOptionsCustomizerBean");
    this.registry = registry;
  }

  @Produces
  public Consumer<VertxOptions> vertxOptionsConsumer() {
    System.out.println("Inside the vertxOptionsConsumer");
    return new Consumer<VertxOptions>() {
      @Override
      public void accept(VertxOptions vertxOptions) {
        System.out.println("Applying vertx options");
        vertxOptions.setMetricsOptions(new MicrometerMetricsOptions().setMicrometerRegistry(registry).setEnabled(true));
      }
    };
  }
}

However if I create a new vertx instance, I am able to get the metrics. Is there any way I can enable micrometer metrics which is exposed by vertx.

This works

this.vertx = new Vertx(new VertxBuilder(new VertxOptions()
            .setPreferNativeTransport(true)
            .setMetricsOptions(new MicrometerMetricsOptions().setMicrometerRegistry(registry).setEnabled(true)))
            .threadFactory(VertxThreadFactory.INSTANCE)
            .executorServiceFactory(new QuarkusExecutorFactory(vertxConfiguration, LaunchMode.NORMAL))
            .init().vertx());

I don't wish to create new vertx instance though.

[ebullient]

If you intend to use the quarkus micrometer extension, you should not use the customizer bean. Either use the micrometer extension OR roll your own, not both.

Specifically, this is problematic:

    return new Consumer<VertxOptions>() {
      @Override
      public void accept(VertxOptions vertxOptions) {
        System.out.println("Applying vertx options");
        vertxOptions.setMetricsOptions(new MicrometerMetricsOptions().setMicrometerRegistry(registry).setEnabled(true));
      }
    };

I have an issue open for the quarkus micrometer extension itself to display more of the core vertx metrics. The problem I ran into is that the metrics as defined by Vert.x aren't what you think they are, because Quarkus uses its own thread/executor pools, so most of the Vert.x thread pool values are always 0.

[himanshumps]

I think if we add metrics capability to VertxConfiguration, it can propogate down to vertx. I do not see any capability in VertxRecorder to pass metrics details as VertxConfiguration doesn't have it. What do you think?

@ebullient

[ebullient]

The micrometer extension is already setting Vertx Metrics Options. We can not use the vanilla Vert.x Metrics support for a lot of reasons.

You have to choose. Roll your own (in which case, disable the micrometer extension), or use the Micrometer extension and either open new issues or add comments to the one I mentioned above about the metrics you want to see and now you expect to use them. As I said, the core Vert.x thread pool / executor metrics are not going to give you the information you think they will with Quarkus.

Right now, you're in a half-way house. You have the micrometer extension in place, but you've set/replaced Vert.x Metrics Options yourself (essentially removing the Micrometer connection in the process). I can't support that.