Metric integration

[nappa85]

Hello guys, I'm back after a long time :)
Actually I'm trying to integrate metrics (like opentelemetry) into the orm, to meter queries execution times.
What do you guys thing about that? Would it be helpful upstream?

To keep a cross-crate integration (opentelemetry is only one of the many metric crates around) I would simply add a global static OnceCell to store a callback function, if the user wants to use metrics has to call a set_metric_callback function that sets the OnceCell (and fails if called more than once), the callback receives some kind of Info struct that contains infos about the query.

The code would look like that (really high level idea):

static METRIC: OnceCell<Box<dyn fn(&Info<'_>)> = OnceCell::new();

pub fn set_metric_callback<F>(callback: F) -> Result<(), Error>
where
  F: fn(&Info<'_>),
{
  METRIC.set(Box::new(callback))
}

struct Info<'a> {
  elapsed: std::time::Duration,
  query: &'a Query,
}

impl Connection {
  async fn query<T>(query: &Query) -> Result<T, Error> {
    let _start = std::time::SystemTime::now();
    let res = sqlx::query(query).await;
    if let Some(callback) = METRIC.get() {
      let info = Info {
        elapsed: _start.elapsed().unwrap_or_default(),
        query,
      };
      callback(&info);
    }
    res
  }
}

At this point the usage would be really simple:

let meter =  opentelemetry::global::meter("ex.com/orm")
            .f64_value_recorder("query.duration")
            .with_description("Query latencies")
            .init();
sea_orm::set_meter_callback(|info| { meter.record(info.duration.as_secs_f64(), &[]); });

[billy1624]

Hey @nappa85, long time no see lolll

This https://github.com/open-telemetry/opentelemetry-rust?

I'm planning to benchmarking sea-orm & sea-query too. But didn't have time implementing it. I guess this would be a great tool to find the costly operations :P

[nappa85]

This https://github.com/open-telemetry/opentelemetry-rust?

Yes, this one

I'm planning to benchmarking sea-orm & sea-query too. But didn't have time implementing it. I guess this would be a great tool to find the costly operations :P

To benchmark the operations, another good pratice would be using the macro instrument from crate tracing https://docs.rs/tracing/latest/tracing/attr.instrument.html
Tracing is a drop-in replacement of log, it gives classic log macros like error! info! warn! etc...
Using the instrumentmacro you create a span where logs are aggregated, spans have sub-spans if nested call are instrumented too, creating a hierarchy of traces you see exactly where the application time has been spent.
More, tracing can be use with opentelemetry to send tracing data to aggregators like jaeger or zipking.

Tracing and metrics are two different things, opentelemetry covers them both

[billy1624]

Interesting :P
Will dig into it after sea-orm 0.5.0 lolll

[tyt2y3]

Hi Marco, welcome back! I hope that your experience with SeaORM had been good so far.
Sure, this seemed a good idea!

Two things:

I don't think it is the best strategy for it to be a global context, is there a specific reason for this?
We might be able to attach it to the DatabaseConnection
The method might as well be called set_meter_handler, or simply metered

PR is definitely welcomed!

[nappa85]

Sorry, I've done the PR before reading your reply.
Let me know what you think of it

[nappa85]

In facts putting the metric callback in the connection would open the door for different metric for different databases...
I'll work on it

[nappa85]

I've moved the metric as per-connection, let me know if you like it

[penso]

I'm looking at adding opentelemetry too, to send traces to a jaeger service. @nappa85 I can't find anything related to the PR you're talking about, was any work done in sea-orm & opentelemetry?

[nappa85]

Actually I've done this to use opentelemetry, I think the example is quite self-explanatory.
You've to attach a callback to every connection using https://docs.rs/sea-orm/latest/sea_orm/entity/prelude/enum.DatabaseConnection.html#method.set_metric_callback
Inside the callback you can populate as many metrics you want

[tyt2y3]

Thanks for the note!