Fix instrumentation of input/output token consumed to be recognized as numeric attributes

[Rindrics]

Task description

Fix instrumentation of input/output token consumed to be recognized as numeric attributes

Background

• related to Fix instrumentation on external service usage #168
• By Fix instrumentation on external service usage #169, we started measuring token consumed as a nested object tokenConsumed: {"input": 123, "output": 456}
• Although I was planning to parse it into two numeric fields with data pipeline function of our telemetry backend, but I found it a little difficult because tokenConsumed field was recognized as string attributes tokenConsumed: "{\"input\": 123, \"output\": 456}"
  • This seems to be related to the specification of OpenTelemetry that it does not support nested attributes

Additional context

Although we are sending business metrics (token consumption is a kind of business metrics) to our observability backend now, we are planning to send them to to BI platform in the future.

Ways to resolve current problem

• option 1: send input/output tokens as structured log using body field
  • pros: (slightly) better in data representation because two attributes related to token consumption can be represented as child attributes of tokenConsumed
  • cons:
    • bad in uniformity because "only" token consumption data is sent in body field although other metrics are
    • still requires preprocessing pipeline to get input/output token from body field
• option 2: send input/output tokens as two flat attributes
  • pros: no preprocessing parser required
  • cons: (slightly) bad in data representation

TODO (Optional)

• decide which option to choose
• fix current instrumentation

[Rindrics]

option 2 seems better for us because it requires no additional configuration

[Rindrics]

We have at least additional two options where to define data processor when we choose option 1

• option 1-a: OTEL collector
  • pros: no coupling with specific telemetry backend
  • cons:
    • requires infrastructure cost
    • (we do not have separated OTEL collector yet)
• option 1-b: telemetry backend
  • pros: shorter implementation time in current state
  • cons: processor resource is bound to telemetry backend

[Rindrics]

I suggest:

• sending tokenConsumedInput and tokenConsumedOutput as two flat attributes (option 2)
• using body to send nested attributes and deploying separate collector if we need many kinds of structured attributes in the future (option 1-a)