-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(spdlog): impl JsonFormatter #69
Changes from 3 commits
20ba1c3
83d9da8
453757b
c9a5eb7
d70c801
f822ec6
c3c2636
e43aadf
7c7349a
929fe9d
0c83e4e
715d879
99b8736
11df475
10f17d3
8d7a1e1
3c4c229
d8f091e
18edfe4
e698a45
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
@@ -0,0 +1,184 @@ | ||||||||
use std::fmt::{self, Write}; | ||||||||
|
||||||||
use cfg_if::cfg_if; | ||||||||
use serde::{ser::SerializeStruct, Serialize}; | ||||||||
|
||||||||
use crate::{ | ||||||||
formatter::{FmtExtraInfo, Formatter, LOCAL_TIME_CACHER}, | ||||||||
Error, Record, StringBuf, __EOL, | ||||||||
}; | ||||||||
|
||||||||
struct JsonRecord<'a>(&'a Record<'a>); | ||||||||
|
||||||||
impl<'a> Serialize for JsonRecord<'a> { | ||||||||
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error> | ||||||||
where | ||||||||
S: serde::Serializer, | ||||||||
{ | ||||||||
let src_loc = self.0.source_location(); | ||||||||
|
||||||||
let mut record = | ||||||||
serializer.serialize_struct("JsonRecord", if src_loc.is_none() { 4 } else { 5 })?; | ||||||||
|
||||||||
record.serialize_field("level", &self.0.level())?; | ||||||||
{ | ||||||||
let mut local_time_cacher = LOCAL_TIME_CACHER.lock(); | ||||||||
record.serialize_field::<str>( | ||||||||
"time", | ||||||||
local_time_cacher | ||||||||
.get(self.0.time()) | ||||||||
.full_second_str() | ||||||||
.as_ref(), | ||||||||
)?; | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think for the machine format JSON, a timestamp number is more friendly. How about we rename the field to UPDATE: Seems that the cacher is not needed, just get the timestamp from field |
||||||||
} | ||||||||
record.serialize_field("tid", &self.0.tid())?; | ||||||||
record.serialize_field("payload", self.0.payload())?; | ||||||||
|
||||||||
if let Some(src_loc) = src_loc { | ||||||||
record.serialize_field("source_location", src_loc)?; | ||||||||
SpriteOvO marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
} | ||||||||
|
||||||||
record.end() | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
impl<'a> From<&'a Record<'a>> for JsonRecord<'a> { | ||||||||
fn from(value: &'a Record<'a>) -> Self { | ||||||||
JsonRecord(value) | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
enum JsonFormatterError { | ||||||||
Fmt(fmt::Error), | ||||||||
Serialization(serde_json::Error), | ||||||||
} | ||||||||
|
||||||||
impl From<fmt::Error> for JsonFormatterError { | ||||||||
fn from(value: fmt::Error) -> Self { | ||||||||
JsonFormatterError::Fmt(value) | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
impl From<serde_json::Error> for JsonFormatterError { | ||||||||
fn from(value: serde_json::Error) -> Self { | ||||||||
JsonFormatterError::Serialization(value) | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
impl From<JsonFormatterError> for crate::Error { | ||||||||
fn from(value: JsonFormatterError) -> Self { | ||||||||
match value { | ||||||||
JsonFormatterError::Fmt(e) => Error::FormatRecord(e), | ||||||||
JsonFormatterError::Serialization(e) => Error::Serialization(e.into()), | ||||||||
} | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
/// JSON logs formatter | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
The doc formatting seems to be broken by Rustfmt, try adding a |
||||||||
/// | ||||||||
/// Output format: | ||||||||
/// | ||||||||
/// ```json | ||||||||
/// {"level":"info","time":"2024-01-01 | ||||||||
/// 12:00:00","tid":123456,"payload":"test"} | ||||||||
/// | ||||||||
/// // with source location | ||||||||
/// {"level":"info","time":"2024-01-01 | ||||||||
/// 12:00:00","tid":123456,"payload":"test","source_location":{"module_path":" | ||||||||
/// module","file":"file.rs","line":42}} | ||||||||
/// ``` | ||||||||
#[derive(Clone)] | ||||||||
pub struct JsonFormatter; | ||||||||
SpriteOvO marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
|
||||||||
impl JsonFormatter { | ||||||||
/// Create a `JsonFormatter` | ||||||||
pub fn new() -> JsonFormatter { | ||||||||
SpriteOvO marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||
JsonFormatter | ||||||||
} | ||||||||
|
||||||||
fn format_impl( | ||||||||
&self, | ||||||||
record: &Record, | ||||||||
dest: &mut StringBuf, | ||||||||
) -> Result<FmtExtraInfo, JsonFormatterError> { | ||||||||
cfg_if! { | ||||||||
if #[cfg(not(feature = "flexible-string"))] { | ||||||||
dest.reserve(RESERVE_SIZE); | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
let json_record: JsonRecord = record.into(); | ||||||||
|
||||||||
dest.write_str(&serde_json::to_string(&json_record)?)?; | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Leaving this comment as a note.
I benchmarked Unfortunately, due to the |
||||||||
|
||||||||
dest.write_str(__EOL)?; | ||||||||
|
||||||||
Ok(FmtExtraInfo { style_range: None }) | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
impl Formatter for JsonFormatter { | ||||||||
fn format(&self, record: &Record, dest: &mut StringBuf) -> crate::Result<FmtExtraInfo> { | ||||||||
self.format_impl(record, dest).map_err(Into::into) | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
impl Default for JsonFormatter { | ||||||||
fn default() -> Self { | ||||||||
JsonFormatter::new() | ||||||||
} | ||||||||
} | ||||||||
|
||||||||
#[cfg(test)] | ||||||||
mod tests { | ||||||||
use chrono::prelude::*; | ||||||||
|
||||||||
use super::*; | ||||||||
use crate::{Level, SourceLocation, __EOL}; | ||||||||
|
||||||||
#[test] | ||||||||
fn should_format_json() { | ||||||||
let mut dest = StringBuf::new(); | ||||||||
let formatter = JsonFormatter::new(); | ||||||||
let record = Record::builder(Level::Info, "payload").build(); | ||||||||
let extra_info = formatter.format(&record, &mut dest).unwrap(); | ||||||||
|
||||||||
let local_time: DateTime<Local> = record.time().into(); | ||||||||
|
||||||||
assert_eq!(extra_info.style_range, None); | ||||||||
assert_eq!( | ||||||||
dest.to_string(), | ||||||||
format!( | ||||||||
r#"{{"level":"Info","time":"{}","tid":{},"payload":"{}"}}{}"#, | ||||||||
local_time.format("%Y-%m-%d %H:%M:%S"), | ||||||||
record.tid(), | ||||||||
"payload", | ||||||||
__EOL | ||||||||
) | ||||||||
); | ||||||||
} | ||||||||
|
||||||||
#[test] | ||||||||
fn should_format_json_with_src_loc() { | ||||||||
let mut dest = StringBuf::new(); | ||||||||
let formatter = JsonFormatter::new(); | ||||||||
let record = Record::builder(Level::Info, "payload") | ||||||||
.source_location(Some(SourceLocation::__new("module", "file.rs", 1, 2))) | ||||||||
.build(); | ||||||||
let extra_info = formatter.format(&record, &mut dest).unwrap(); | ||||||||
|
||||||||
let local_time: DateTime<Local> = record.time().into(); | ||||||||
|
||||||||
assert_eq!(extra_info.style_range, None); | ||||||||
assert_eq!( | ||||||||
dest.to_string(), | ||||||||
format!( | ||||||||
r#"{{"level":"Info","time":"{}","tid":{},"payload":"{}","source_location":{{"module_path":"module","file":"file.rs","line":1,"column":2}}}}{}"#, | ||||||||
local_time.format("%Y-%m-%d %H:%M:%S"), | ||||||||
record.tid(), | ||||||||
"payload", | ||||||||
__EOL | ||||||||
) | ||||||||
); | ||||||||
} | ||||||||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been thinking about this gate these days. Perhaps we can enable
JsonFormatter
when featureserde
is enabled and featureserde
implies dependenciesserde
andserde_json
, then featurejson-formatter
is no longer needed.The benefit of it is that we have ability to add other serialization crates in the future, so users can choose the "backend" of
JsonFormatter
by enabling just one feature.What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might add lots of dependencies for a single feature as new formatters are introduced.
Or we can just make a
SerdeFormatter
as @Lancern said to decouple the serializer from the formatter. So user can choose any serializer instead of theserde_json
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, makes sense. What about enable
JsonFormatter
when featureserde_json
is enabled, soserde_json
impliesserde
?SerdeFormatter
is fine, but we are in no hurry to implement it in this PR.What I meant by my comment was that
JsonFormatter
should not be limited to only usingserde
as a backend. Semantically,JSON
is a format andserde
is an implementation. It seems more correct that the implementation implies the format, rather than the format implying the implementation. So that if now or in the future, there are any json crates that aren'tserde
-based, we could easily add it without breaking anything.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
serde_json
impliesserde
sounds more reasonable.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this really implementable?
serde
is the de-facto standard and we already rely on the critical abstractions it provides. The "critical abstractions" I'm talking about here is theSerialize
trait which is provided byserde
. I cannot work out a way to bypass it and still implement a usableJsonFormatter
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is. For now, there are crates
rkyv
,miniserde
,nanoserde
, etc., they each have their own advantages and are notserde
-based. If we can harmlessly preserve the maximum possibilities, why not do so?If you are talking about deriving
serde::Serialize
macro for types ofspdlog-rs
, this can be unforced. Considering the following psuode-code:Then implement
JsonFormatter
:Also, we are not in a hurry to implement these other crate supports in this PR right now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@SpriteOvO OK this makes sense. I thought you mean we completely abandon any dependencies on serialization libraries and let the users choose what suits them. So we're still depending on a bunch of pre-selected serialization libraries (even if they're just optional) and let the user choose one among them.