The best (possible) way to add custom attribute to the metric

[IvanKuzyshyn]

Is it possible to add a custom attribute to the standard metric? For example, I would like to extend http_server_duration_bucket by adding a custom tag. For example, is it possible to add a custom tag based on the request data, let's say I want to add a custom tag based on the query parameters coming in the request.

I have a basic setup using Node.js and Express server, like:

// app.js
require('./instrumentation')
const express = require('express');

const app = express();
const port = 4000;

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

app.use('/test', (req, res) => {
  console.log(`Handling ${req.method} ${req.url}`);

  const body = {
    message: `Reqeust: method ${req.method} to ${req.url} handled`,
  };

  res.status(200).end(JSON.stringify(body));
});

app.listen(port, () => {
  console.log(`Echo server is listening on port ${port}`);
});

// instrumentation.js
const opentelemetry = require('@opentelemetry/sdk-node');
const { PrometheusExporter } = require('@opentelemetry/exporter-prometheus');
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { ExpressInstrumentation, ExpressLayerType } = require('@opentelemetry/instrumentation-express');

const pPort = PrometheusExporter.DEFAULT_OPTIONS.port;
const pEndpoint = PrometheusExporter.DEFAULT_OPTIONS.endpoint;

const prometheusExporter = new PrometheusExporter(
    { preventServerStart: false },
    () => { console.log(`> Prometheus metrics available: http://localhost:${pPort}${pEndpoint}`); }
);

const httpInstrumentation = new HttpInstrumentation();
const expressInstrumentation = new ExpressInstrumentation();

const sdk = new opentelemetry.NodeSDK({
    metricReader: prometheusExporter,
    instrumentations: [ httpInstrumentation, expressInstrumentation ]
});

sdk.start();
console.log('> OpenTelemetry SDK started');

process.on("SIGTERM", () => {
    sdk
        .shutdown()
        .then(
            () => console.log('> SDK shut down successfully'),
            (err) => console.log('> Error shutting down SDK', err)
        )
        .finally(() => process.exit(0));
});

Imagine I want to execute this request:

curl -s -X POST "http://localhost:4000/test?appVersion=1.1.1"

and I would like to expect metrics like:

http_server_duration_bucket{http_scheme="http",http_method="POST",net_host_name="localhost",http_flavor="1.1",http_status_code="200",net_host_port="4000",http_route="/test",app_version="1.1.1",le="1000"}

Is there any way to do this?

I know that we need to carefully control custom attributes as this may drastically increase the amount of metrics. The request query parameter is just an example for simplification.

• This may affect other libraries, but I would like to get opinions here first

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[hertzsprung]

I've been looking for the same thing for opentelemetry-java. I wonder if this issue would be better off moving to a different opentelemetry project? (I'm not which, though!)

[IvanKuzyshyn]

I've found a solution to the problem I initially had. I was able to solve it in a bit different way than I intended from the very beginning.

Problem:

I am working on a project that is based on the Next.js framework. However, I'm using a custom Express server configuration. This configuration is based on the original Next.js custom server example and it sets "*" matcher for all the requests that come to Express to be handled by Next.js.
This works perfectly but it creates an issue that Express auto instrumentation sets the http_route="*" attribute to all the http_server_duration_bucket metric series that makes it impossible to properly query the data.

My solution is using this small piece of code to set the proper http_route attribute:

import { getRPCMetadata, RPCType } from '@opentelemetry/core';
import { context } from '@opentelemetry/api';

// ...

// Override "*" http_route attribute set by Express instrumentation
const route = matcher.match(parsedUrl); // Get proper route
const rpcMetadata = getRPCMetadata(context.active());

if (rpcMetadata?.type === RPCType.HTTP) {
  rpcMetadata.route = route;
}

Basically, getRPCMetadata(context.active()) gets RPC context for the active context, and rpcMetadata.route = route mutates the object by overriding attribute value.

For the full reference, please take a look at another discussion thread.

@hertzsprung, maybe this is something that can inspire you to find a similar solution for your problem.

[alexa-gt]

Maybe this shouldn't be closed? I would still very much like to see this implemented. I have a requirement to implement this but would only be possible with a custom attribute. Appending custom values to http.route seems like breaking semantic conventions and also an anitpattern.