Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON formatter #377

Merged
merged 30 commits into from
Oct 18, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
19b5314
WIP: JSON formatter
Oct 9, 2019
de8e931
Implement JSON formatting using SerdeMapVisitor
Oct 10, 2019
2857e3f
Better span handling
Oct 10, 2019
c695027
fmt
Oct 10, 2019
72615d3
fmt
Oct 10, 2019
f00b0c6
Add newline after json data
Oct 10, 2019
1272054
Disable ANSI feature if enabling JSON
Oct 10, 2019
4e1ef66
Add a note why we need the `WriteAdaptor`
Oct 10, 2019
cd51096
Add a note about the `json` feature flag
Oct 10, 2019
8a148ad
Revert "Disable ANSI feature if enabling JSON"
Oct 10, 2019
522b963
Add note how ansi doesn't work with json
Oct 10, 2019
f652a1b
AsSerde for Level, no extra trim, less allocations
Oct 11, 2019
b25bdaf
Use a shorter path
Oct 11, 2019
cb37cc6
Add json logging to the proxy_server example
Oct 11, 2019
abebff2
Fixes for @hawkw
Oct 11, 2019
7897c53
Update tracing-subscriber/src/lib.rs
Oct 12, 2019
d831828
Serialize span and fields into separate objects
Oct 14, 2019
3e8bd94
Fix compilation problems and tests
Oct 14, 2019
4117ad3
add the ability to serialize fields as a separate map
hawkw Oct 14, 2019
63e8cd6
format fields without requiring `tracing-core` change
hawkw Oct 14, 2019
f1fb716
undo tracing-core change
hawkw Oct 14, 2019
013819d
Almost correct span field recording
Oct 15, 2019
b1d4b8c
fmt
Oct 16, 2019
a1cf246
Set logger in a separate function
pimeys Oct 18, 2019
c4e9f7f
move WriteAdaptor out of serde crate
hawkw Oct 18, 2019
d72d5a7
rustfmt
hawkw Oct 18, 2019
b6e13ea
fix imports
hawkw Oct 18, 2019
62c6d5a
fix example not compiling
hawkw Oct 18, 2019
e53e392
fix warnings in example
hawkw Oct 18, 2019
fb42bd2
fix example again
hawkw Oct 18, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion nightly-examples/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ edition = "2018"

[dev-dependencies]
tracing = "0.1"
tracing-subscriber = { version = "0.1", path = "../tracing-subscriber" }
tracing-subscriber = { version = "0.1", path = "../tracing-subscriber", features = ["json"] }
tracing-futures = { path = "../tracing-futures", default-features = false, features = ["std-future"] }
tokio = { git = "https://github.com/tokio-rs/tokio.git" }
tracing-attributes = { path = "../tracing-attributes" }
futures-preview = "0.3.0-alpha.18"
clap = "2.33"
81 changes: 64 additions & 17 deletions nightly-examples/examples/proxy_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
//!
//! You can showcase this by running this in one terminal:
//!
//! cargo +nightly run --example proxy_server
//! cargo +nightly run --example proxy_server -- --log_format=(plain|json)
//!
//! This in another terminal
//!
Expand All @@ -29,12 +29,11 @@ use tokio::{
net::{TcpListener, TcpStream},
prelude::*,
};

use tracing::{debug, debug_span, info, warn};
use tracing_attributes::instrument;
use tracing_futures::Instrument;

use std::{env, net::SocketAddr};
use clap::{App, Arg, ArgMatches, arg_enum, value_t};
use std::net::SocketAddr;

#[instrument]
async fn transfer(
Expand Down Expand Up @@ -84,23 +83,50 @@ async fn transfer(
Ok(())
}

arg_enum! {
#[derive(PartialEq, Debug)]
pub enum LogFormat {
Plain,
Json,
}
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
use tracing_subscriber::{EnvFilter, FmtSubscriber};

let subscriber = FmtSubscriber::builder()
.with_env_filter(EnvFilter::from_default_env().add_directive("proxy_server=trace".parse()?))
.finish();
tracing::subscriber::set_global_default(subscriber)?;

let listen_addr = env::args()
.nth(1)
.unwrap_or_else(|| "127.0.0.1:8081".to_string());
let matches = App::new("Proxy Server Exemple")
.version("1.0")
.arg(
Arg::with_name("log_format")
.possible_values(&LogFormat::variants())
.case_insensitive(true)
.long("log_format")
.value_name("log_format")
.help("Formating of the logs")
.required(false)
.takes_value(true),
)
.arg(
Arg::with_name("listen_addr")
.long("listen_addr")
.help("Address to listen on")
.takes_value(true)
.required(false),
)
.arg(
Arg::with_name("server_addr")
.long("server_addr")
.help("Address to proxy to")
.takes_value(false)
.required(false),
)
.get_matches();

set_global_default(&matches)?;

let listen_addr = matches.value_of("listen_addr").unwrap_or("127.0.0.1:8081");
let listen_addr = listen_addr.parse::<SocketAddr>()?;

let server_addr = env::args()
.nth(2)
.unwrap_or_else(|| "127.0.0.1:3000".to_string());
let server_addr = matches.value_of("server_addr").unwrap_or("127.0.0.1:3000");
let server_addr = server_addr.parse::<SocketAddr>()?;

let mut listener = TcpListener::bind(&listen_addr).await?.incoming();
Expand Down Expand Up @@ -131,3 +157,24 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {

Ok(())
}

fn set_global_default(matches: &ArgMatches<'_>) -> Result<(), Box<dyn std::error::Error>> {
use tracing_subscriber::{FmtSubscriber, filter::EnvFilter};
match value_t!(matches, "log_format", LogFormat).unwrap_or(LogFormat::Plain) {
LogFormat::Json => {
let subscriber = FmtSubscriber::builder()
.json()
.with_env_filter(EnvFilter::from_default_env().add_directive("proxy_server=trace".parse()?))
.finish();
tracing::subscriber::set_global_default(subscriber)?;
}
LogFormat::Plain => {
let subscriber = FmtSubscriber::builder()
.with_env_filter(EnvFilter::from_default_env().add_directive("proxy_server=trace".parse()?))
.finish();
tracing::subscriber::set_global_default(subscriber)?;
}
}
Ok(())
}

55 changes: 55 additions & 0 deletions tracing-serde/src/fields.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
//! Support for serializing fields as `serde` structs or maps.
use super::*;

#[derive(Debug)]
pub struct SerializeFieldMap<'a, T>(&'a T);

pub trait AsMap: Sized + sealed::Sealed {
fn field_map(&self) -> SerializeFieldMap<Self> {
SerializeFieldMap(self)
}
}

impl<'a> AsMap for Event<'a> {}
impl<'a> AsMap for Attributes<'a> {}
impl<'a> AsMap for Record<'a> {}

// === impl SerializeFieldMap ===

impl<'a> Serialize for SerializeFieldMap<'a, Event<'_>> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let len = self.0.metadata().fields().len();
let serializer = serializer.serialize_map(Some(len))?;
let mut visitor = SerdeMapVisitor::new(serializer);
self.0.record(&mut visitor);
visitor.finish()
}
}

impl<'a> Serialize for SerializeFieldMap<'a, Attributes<'_>> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let len = self.0.metadata().fields().len();
let serializer = serializer.serialize_map(Some(len))?;
let mut visitor = SerdeMapVisitor::new(serializer);
self.0.record(&mut visitor);
visitor.finish()
}
}

impl<'a> Serialize for SerializeFieldMap<'a, Record<'_>> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let serializer = serializer.serialize_map(None)?;
let mut visitor = SerdeMapVisitor::new(serializer);
self.0.record(&mut visitor);
visitor.finish()
}
}
29 changes: 25 additions & 4 deletions tracing-serde/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ use tracing_core::{
span::{Attributes, Id, Record},
};

pub mod fields;

#[derive(Debug)]
pub struct SerializeField(Field);

Expand Down Expand Up @@ -156,10 +158,7 @@ impl<'a> Serialize for SerializeRecord<'a> {
S: Serializer,
{
let serializer = serializer.serialize_map(None)?;
let mut visitor = SerdeMapVisitor {
serializer,
state: Ok(()),
};
let mut visitor = SerdeMapVisitor::new(serializer);
self.0.record(&mut visitor);
visitor.finish()
}
Expand All @@ -170,6 +169,18 @@ struct SerdeMapVisitor<S: SerializeMap> {
state: Result<(), S::Error>,
}

impl<S> SerdeMapVisitor<S>
where
S: SerializeMap,
{
fn new(serializer: S) -> Self {
Self {
serializer,
state: Ok(()),
}
}
}

impl<S> Visit for SerdeMapVisitor<S>
where
S: SerializeMap,
Expand Down Expand Up @@ -320,12 +331,22 @@ impl<'a> AsSerde<'a> for tracing_core::span::Record<'a> {
}
}

impl<'a> AsSerde<'a> for Level {
type Serializable = SerializeLevel<'a>;

fn as_serde(&'a self) -> Self::Serializable {
SerializeLevel(self)
}
}

impl<'a> self::sealed::Sealed for Event<'a> {}

impl<'a> self::sealed::Sealed for Attributes<'a> {}

impl self::sealed::Sealed for Id {}

impl self::sealed::Sealed for Level {}

impl<'a> self::sealed::Sealed for Record<'a> {}

impl<'a> self::sealed::Sealed for Metadata<'a> {}
Expand Down
6 changes: 6 additions & 0 deletions tracing-subscriber/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ env-filter = ["matchers", "regex", "lazy_static"]
fmt = ["owning_ref"]
ansi = ["fmt", "ansi_term"]
registry_unstable = []
json = ["tracing-serde", "serde", "serde_json"]

# Alias for `env-filter`; renamed in version 0.1.2, and will be removed in 0.2.
filter = ["env-filter"]
Expand All @@ -44,6 +45,11 @@ ansi_term = { version = "0.11", optional = true }
owning_ref = { version = "0.4.0", optional = true }
chrono = { version = "0.4", optional = true }

# only required by the json feature
serde_json = { version = "1.0", optional = true }
pimeys marked this conversation as resolved.
Show resolved Hide resolved
serde = { version = "1.0", optional = true }
tracing-serde = { path = "../tracing-serde", optional = true }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll need to publish a tracing-serde release to crates.io before we can publish the changes in this branch — we can't publish a crate that depends on an unpublished crate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. Do you have a plan when we want this to happen?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When these changes land, I can take care of that.


# opt-in deps
parking_lot = { version = ">= 0.7, < 0.10", features = ["owning_ref"], optional = true }

Expand Down
Loading