Missing core metrics from remote-storage-manager-metrics

[zhilingc]

What happened?

I'm running the current newest build (2024-10-23-1729694047) and configured my jmx exporter to pick up the metrics from the plugin, and I do see the metrics from aiven.kafka.server.tieredstorage.cache , aiven.kafka.server.tieredstorage.s3, and aiven.kafka.server.tieredstorage.thread-pool, but not aiven.kafka.server.tieredstorage (specifically type=remote-storage-manager-metrics)

I've verified this with ./kafka-run-class.sh kafka.tools.JmxTool to ensure that it's not an issue with the jmx exporter configuration.

Side note -- I wanted to run the demo locally to rule out any issues with my setup but the demo image is private, is there any way to make that available publicly?

What did you expect to happen?

I expect to see the core metrics (aiven.kafka.server.tieredstorage:type=remote-storage-manager-metrics)

What else do we need to know?

• version: 2024-10-23-1729694047
• kafka version: 2.7.1
• jmx exporter version: 0.16.1

[jeqo]

@zhilingc thanks for pointing out to this issues!

RSM metrics should be available as soon as tiered-storage starts to be used (i.e. segments start to be uploaded).
This happens as the meters are instantiated dynamically based on topic and partition tags. For the other metric registries that you mentioned, the meters are created when the plugin starts.
Could you double-check if this is the case on your environment?

I have also pushed both, the base image and the latest built, to properly test the demos; and created a PR to fix some tooling issues: #627