Service Bus queue metrics are not reported correctly for entities in same namespace #1050
Comments
3 tasks
|
I just tested: with the 1.5.0 version, approach 1) using the ServiceBusQueue Resource Type works fine |
|
Huh that's interesting 🤔 |
|
This seems to be working on my end but will dig deeper later |
|
I tried also with Another thing that gets reported differently between the two: The |
|
Thanks for the pointers, this is helpful! |
Created #1056 for that |
|
When using this config: version: v1
azureMetadata:
tenantId: c8819874-9e56-4e3f-b1a8-1c0325138f27
subscriptionId: 0f9d7fea-99e8-4768-8672-06a28514f77e
resourceGroupName: promitor
metricDefaults:
aggregation:
interval: 00:05:00
scraping:
# Every minute
schedule: "0 * * ? * *"
metrics:
- name: promitor_demo_servicebusqueue_queue_size
description: "Amount of active messages of the 'orders' queue (determined with ServiceBusQueue provider)"
resourceType: ServiceBusQueue
azureMetricConfiguration:
metricName: ActiveMessages
aggregation:
type: Average
# Optionally override the default aggregation interval (metricDefaults.aggregation.interval)
interval: 00:15:00
resources:
- namespace: promitor-messaging
queueName: orders
- namespace: promitor-messaging-other-subscription
queueName: orders
subscriptionId: 0329dd2a-59dc-4493-aa54-cb01cb027dc2
resourceGroupName: promitor-sourcesWith this server configuration: server:
httpPort: 88
metricSinks:
prometheusScrapingEndpoint:
metricUnavailableValue: NaN
enableMetricTimestamps: true # true by default
baseUriPath: /metrics
statsd:
host: graphite
port: 8125
metricPrefix: promitor.
metricsConfiguration:
absolutePath: /config/metrics-declaration.yaml
telemetry:
applicationInsights:
instrumentationKey: ABC
isEnabled: true
verbosity: warning
containerLogs:
isEnabled: true
verbosity: trace
defaultVerbosity: trace
azureMonitor:
logging:
informationLevel: Headers
isEnabled: trueEverything gets reported nicely: I used the same config against (command Can you share the logs of your instance please? I'll need to have some more info since I cannot reproduce it :/ |
|
Here you go, these are the logs with just sensitive information replaced, using I can see in the logs that 3 scraping Jobs for the same metric name get instantiated which looks like its expected, right? But then only 1 of these also logs that it scraped a value. I will also try with your config and see whats the difference |
|
And now I tried your configuration, with just the credentials, namespace and queue names replaced - with the same result, still only 1 queue gets reported. The only difference between our configurations that I can spot is, that you are using 2 queues from different namespaces, while my queues are both in the same namespace |
|
Was there any change to either the ServiceBusQueue resource or some base classes from 1.5.0 to now? Because with 1.5.0 even with my initial configuration, I get all the queues reported just fine. |
|
There was some refactoring which might have caused this so will try with multiple queues in same namespace to hopefully reproduce this! |
|
That seems to be the issue, when I use this: metrics:
- name: promitor_demo_servicebusqueue_queue_size
description: "Amount of active messages of the 'orders' queue (determined with ServiceBusQueue provider)"
resourceType: ServiceBusQueue
azureMetricConfiguration:
metricName: ActiveMessages
aggregation:
type: Average
# Optionally override the default aggregation interval (metricDefaults.aggregation.interval)
interval: 00:15:00
resources:
- namespace: promitor-messaging
queueName: orders
- namespace: promitor-messaging
queueName: sales
- namespace: promitor-messaging-other-subscription
queueName: orders
subscriptionId: 0329dd2a-59dc-4493-aa54-cb01cb027dc2
resourceGroupName: promitor-sourcesIt only returns this: |
tomkerkhove
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am trying to scrape active messages and deadlettered messages per queue for multiple queues from an Azure Service Bus Namespace. I tried 3 different approaches, leading to 2 different, both wrong, results:
Same as 2), but with only a single resource defined (the namespace), without a filter.
Expected Behavior
Per queue I get a metric reported with labels matching the queues name.
Actual Behavior
Approach 1) (via ServiceBusQueue resource type) reports only the metrics for the queue that is last in the list of defined resources.
Approach 2) and 3) (via Generic resource type) reports only a single metric, containing the sum for all queues in the namespace.
Steps to Reproduce the Problem
My instance returns the following metrics (I configured all 3 approaches described above in the same instance, using 3 different metric names):
A look into the Azure portal shows that indeed, the count of messages is 0 (as reported) for the 'queue3', but the values for the other 2 queues only show up as the sum (23), where in the Portal i can see one is 21, and the other is 2.
Portal Screenshot showing that the sum is actually 23:
Portal Screenshot showing that one of the queues has 21:
Specifications
The text was updated successfully, but these errors were encountered: