Couldnt aggregate cummilative metrics on time and then label

[SrinidhiK22]

Component(s)

processor/metricstransform

Describe the issue you're reporting

Background: I have a counter metric, custom_api_requests, with attributes endpoint (e.g., endpoint1, endpoint2) and region (e.g., east, west, south). When I scrape this metric every 10 seconds using the Prometheus receiver, I get 6 data points every 10 seconds. Over a 30-second period, this results in 18 data points.

Goal:
I want to aggregate these 18 data points into two (one for each endpoint), each with a same single timestamp (e.g., at the 30-second mark) and aggregated values based on label.

Attempted Solutions:

1. batch (timeout 30s) + metricstransform
   This gives 6 metric points for 30s ( 2metrics for 10s as they dont have same timestamp)
2. batch+ transform/truncateTime+ metricstransform
   This gives 6 metric points for 30s (all having 30s timestamp but aggregated like before)
3. batch + groupbyattrs + transform/truncateTime+ metricstransform
   This is giving two points but the value of counter is adding up

[github-actions]

Pinging code owners:

• processor/metricstransform: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[bacherfl]

Hi @SrinidhiK22 can you please provide some example input, ideally the payload of a prometheus endpoint containing the original metrics, and the expected output after applying the processors?

[SrinidhiK22]

@bacherfl Thanks for your reply.
So im using this receiver config for scraping the endpoint every 10s
prometheus:
config:
scrape_configs:
- job_name: 'metricsapp'
scrape_interval: 10s
static_configs:
- targets: ['host:port']
metrics_path: '/actuator/prometheus'

the endpoint has metrics like:

# HELP application_ready_time_seconds Time taken for the application to be ready to service requests
# TYPE application_ready_time_seconds gauge
application_ready_time_seconds{main_application_class="com.example.metricsapp.MetricsappApplication"} 4.25
# HELP application_started_time_seconds Time taken to start the application
# TYPE application_started_time_seconds gauge
application_started_time_seconds{main_application_class="com.example.metricsapp.MetricsappApplication"} 4.072
# HELP custom_api_requests_total Counts API requests for each endpoint and region
# TYPE custom_api_requests_total counter
custom_api_requests_total{endpoint="endpoint1",region="ap-south"} 1.0
custom_api_requests_total{endpoint="endpoint1",region="eu-west"} 1.0
custom_api_requests_total{endpoint="endpoint1",region="us-east"} 1.0
custom_api_requests_total{endpoint="endpoint2",region="ap-south"} 1.0
custom_api_requests_total{endpoint="endpoint2",region="eu-west"} 1.0
custom_api_requests_total{endpoint="endpoint2",region="us-east"} 1.0

and the collected metric is

11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 east
11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 west
11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 south
11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 east
11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 west
11:16:40 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 south

this thing repeats for every 10s like and increases if new requests come

11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 east
11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 west
11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint1 south
11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 east
11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 west
11:16:50 {"type":"cumulativeCount","count":0.0,"cumulative":1.0} endpoint2 south

I want to aggregate based on time interval consider 30s and also label : endpoint

Expected output : from 11:16:40 to 11:17:00:

11:17:00 {"type":"cumulativeCount","count":0.0,"cumulative":3.0} endpoint1
11:17:00 {"type":"cumulativeCount","count":0.0,"cumulative":3.0} endpoint2
after aggregation only the endpoint we aggregate on remains.

Requirement : this type of aggregation should work for all types of metrics like guages, histograms (only sum), updowncounters.