Batch Mode fails on Azure Firewall DataProcessed Metric

[hkfgo]

Report

[20:10:35 FTL] Failed to scrape resource for metric 'DataProcessed'
System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.ThrowHelper.ThrowArgumentNullException(String name)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at Promitor.Core.Scraping.AzureMonitorScraper`1.BatchScrapeResourceAsync(String subscriptionId, BatchScrapeDefinition`1 batchScrapeDefinition, PromitorMetricAggregationType aggregationType, TimeSpan aggregationInterval) in /src/Promitor.Core.Scraping/AzureMonitorScraper.cs:line 136
   at Promitor.Core.Scraping.Scraper`1.BatchScrapeAsync(BatchScrapeDefinition`1 batchScrapeDefinition) in /src/Promitor.Core.Scraping/Scraper.cs:line 125

Relevant code on that path:

var scrapeResults = new List<ScrapeResult>();
            // group based on resource, then do enrichment per group
            var groupedMeasuredMetrics = resourceIdTaggedMeasuredMetrics.GroupBy(measuredMetric => measuredMetric.ResourceId);
            foreach (IGrouping<string, ResourceAssociatedMeasuredMetric> resourceMetricsGroup in groupedMeasuredMetrics) 
            {
                var resourceId = resourceMetricsGroup.Key; 
                if (_resourceDefinitions.TryGetValue(resourceId, out Tuple<IAzureResourceDefinition, TResourceDefinition> resourceDefinitionTuple))

Somehow resourceID was not properly parsed out from metric results.

Expected Behavior

Scrape results processed and published.

Actual Behavior

Processing failed because resource ID was not attached to scrape results on this particular metric.

Steps to Reproduce the Problem

1. Target scraper against Firewall DataProcessed metric

Component

Scraper

Version

Scraper-v2.11.2

Configuration

Configuration:

# Add your scraping configuration here

Logs

example

Platform

None

Contact Details

No response

[github-actions]

Thank you for opening an issue! We rely on the community to maintain Promitor. (Learn more)

Is this something you want to contribute?

[locmai]

Seems like this is not particular for the Data Processed metric but more of a generic issue. Or could be to the Azure Firewall scraper scope, geting more log"

[20:10:35 WRN] No metric information found for metric ApplicationRuleHit with dimensions ["Status", "Reason", "Protocol"]. Details: No time series was found
[20:10:35 WRN] No metric information found for metric NetworkRuleHit with dimensions ["Status", "Reason", "Protocol"]. Details: No time series was found
[20:10:35 WRN] No metric information found for metric Throughput with dimensions []. Details: No time series was found
[20:10:35 FTL] Failed to scrape resource for metric 'ApplicationRuleHit'
System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.ThrowHelper.ThrowArgumentNullException(String name)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at Promitor.Core.Scraping.AzureMonitorScraper`1.BatchScrapeResourceAsync(String subscriptionId, BatchScrapeDefinition`1 batchScrapeDefinition, PromitorMetricAggregationType aggregationType, TimeSpan aggregationInterval) in /src/Promitor.Core.Scraping/AzureMonitorScraper.cs:line 136
   at Promitor.Core.Scraping.Scraper`1.BatchScrapeAsync(BatchScrapeDefinition`1 batchScrapeDefinition) in /src/Promitor.Core.Scraping/Scraper.cs:line 125
[20:10:35 FTL] Failed to scrape resource for metric 'Throughput'
System.ArgumentNullException: Value cannot be null. (Parameter 'key')
   at System.ThrowHelper.ThrowArgumentNullException(String name)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at Promitor.Core.Scraping.AzureMonitorScraper`1.BatchScrapeResourceAsync(String subscriptionId, BatchScrapeDefinition`1 batchScrapeDefinition, PromitorMetricAggregationType aggregationType, TimeSpan aggregationInterval) in /src/Promitor.Core.Scraping/AzureMonitorScraper.cs:line 136
   at Promitor.Core.Scraping.Scraper`1.BatchScrapeAsync(BatchScrapeDefinition`1 batchScrapeDefinition) in /src/Promitor.Core.Scraping/Scraper.cs:line 125

So that we jumped to this catch

promitor/src/Promitor.Core.Scraping/AzureMonitorScraper.cs

Lines 120 to 128 in 202e15b

	catch (MetricInformationNotFoundException metricsNotFoundException)
	{
	Logger.LogWarning("No metric information found for metric {MetricName} with dimensions {MetricDimensions}. Details: {Details}", metricsNotFoundException.Name, metricsNotFoundException.Dimensions, metricsNotFoundException.Details);
	var measuredMetric = dimensionNames.Count > 0
	? MeasuredMetric.CreateForDimensions(dimensionNames)
	: MeasuredMetric.CreateWithoutDimensions(null);
	resourceIdTaggedMeasuredMetrics.Add(measuredMetric.WithResourceIdAssociation(null));
	}

here so I traced back to the BatchQueryMetricAsync function and these 2 could throw the MetricInformationNotFoundException:

promitor/src/Promitor.Integrations.AzureMonitor/AzureMonitorQueryClient.cs

Lines 121 to 131 in 202e15b

	if (metricNamespace == null)
	{
	throw new MetricNotFoundException(metricName);
	}
	var metricsDefinitions = await _metricsQueryClient.GetAndCacheMetricDefinitionsAsync(resourceIds.First(), metricNamespace, _resourceMetricDefinitionMemoryCache, _metricDefinitionCacheDuration);
	var metricDefinition = metricsDefinitions.SingleOrDefault(definition => definition.Name.ToUpper() == metricName.ToUpper());
	if (metricDefinition == null)
	{
	throw new MetricNotFoundException(metricName);
	}