Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for custom attributes for spans and logs #4102

Merged
merged 109 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
c3919d5
wip
bnjjj Oct 25, 2023
4b4f703
custom dynamic attributes is working for logs and spans
bnjjj Oct 26, 2023
2d7c51c
wip
bnjjj Oct 26, 2023
0903984
add support of custom attributes in json formatter
bnjjj Oct 26, 2023
e69c986
wip
bnjjj Oct 30, 2023
639c3e7
Attributes workaround
Oct 30, 2023
676e4e4
supergraph done, missing subgraph
bnjjj Oct 31, 2023
3e9a77b
add support of custom attributes for subgraph
bnjjj Oct 31, 2023
bbdc0e8
add better implementation of dynamic attributes with sampling
bnjjj Oct 31, 2023
c68a14f
fix lint
bnjjj Oct 31, 2023
d39f85c
Merge branch 'dev' of github.com:apollographql/router into bnjjj/cust…
bnjjj Oct 31, 2023
df02c43
fix tests
bnjjj Nov 2, 2023
448d8b8
comment dyn attribute in text formatter
bnjjj Nov 2, 2023
9def3e3
use otel constants
bnjjj Nov 2, 2023
cb1d523
add support of requirement level
bnjjj Nov 2, 2023
3f94291
lint
bnjjj Nov 2, 2023
3874345
address review comments
bnjjj Nov 2, 2023
d18beda
Merge dev
Nov 3, 2023
b3c2efb
address review comments
bnjjj Nov 3, 2023
f53e188
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 3, 2023
316f48d
remove chrono and fix attribute name for subgraph
bnjjj Nov 3, 2023
5eaa07a
Split attributes.rs
Nov 3, 2023
80e4c95
Move remaining selector only stuff from attributes.rs to selectors.rs
Nov 3, 2023
fc731a9
remove useless print and comment
bnjjj Nov 3, 2023
8c3bd0c
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 3, 2023
6528a26
Add some tests, fix baggage.
Nov 3, 2023
6479545
Add tests for trace id.
Nov 3, 2023
9fd343a
Add env test
Nov 3, 2023
616c821
Add more tests for env
Nov 6, 2023
a12330e
Fix clippy issues
Nov 6, 2023
882bab4
Add selectors test for operations
BrynCooke Nov 6, 2023
ffc5e9a
Fix operation name hash and add test
Nov 6, 2023
c06b887
Fix subgraph supergraph operation name hash and add test
Nov 6, 2023
2a8cf4c
Fix subgraph subgraph operation name hash and add tests
Nov 6, 2023
44c56fa
Add query selector and tests
Nov 6, 2023
1acae79
Add response_status and tests
Nov 6, 2023
ae5bf38
Add tests for query variable.
Nov 6, 2023
681d076
Add subgraph response status
Nov 6, 2023
a1a95bc
Skip new telemetry config that won't be supported in initial drop
Nov 6, 2023
e34dfad
Fix datadog trace IDs
Nov 7, 2023
76452d5
Fix snapshot
Nov 7, 2023
a46b1a7
Factor out baggage logic and add tests.
Nov 7, 2023
f7caed8
Remove AttributeValue::U128.
Nov 7, 2023
a9ef3c3
wip
bnjjj Nov 3, 2023
cc64b6d
delete useless code
bnjjj Nov 3, 2023
238b932
fix apollo telemetry
bnjjj Nov 6, 2023
a41c9c1
fix jaeger test
bnjjj Nov 7, 2023
bffe3e6
fix snapshot
bnjjj Nov 7, 2023
1e81827
Update schema snap
Nov 7, 2023
3df1cdd
Replace use of AttributeValue with opentelemetry::Value
Nov 7, 2023
2699e70
Lint
Nov 7, 2023
12d0098
Update schema snapshot
Nov 7, 2023
947a295
add more snapshot tests for legacy_request_span option
bnjjj Nov 7, 2023
755291d
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 7, 2023
7f55f46
Dry to_otel_value.
Nov 7, 2023
1c4c09a
add http.method attribute on request span to avoid breaking change
bnjjj Nov 7, 2023
86be403
fix rhai tests
bnjjj Nov 7, 2023
a8574a8
Merge branch 'bnjjj/custom_attributes_telemetry' into bnjjj/legacy_re…
Nov 7, 2023
6b89bea
Set legacy request span for integration tests.
Nov 7, 2023
76ca546
Add span factory to allow us to switch between legacy and new mode.
Nov 7, 2023
692ef02
Add span factory to allow us to switch between legacy and new mode. (…
bnjjj Nov 8, 2023
55de44b
fix tests and integrate span mode
bnjjj Nov 8, 2023
7d1867d
rename mode legacy to deprecated
bnjjj Nov 8, 2023
6163566
Add support for `telemetry.spans.mode` (#4139)
bnjjj Nov 8, 2023
46571d1
add usage metrics for spans configuration
bnjjj Nov 8, 2023
e6bb2df
Add config for new logging (#4165)
BrynCooke Nov 8, 2023
46abd1e
filter apollo_private when adding dyn attributes on unsampled spans
bnjjj Nov 8, 2023
670c588
fix text formatter with span attributes
bnjjj Nov 8, 2023
0603f84
fix config file for testdata
bnjjj Nov 9, 2023
f00384d
fix snapshot
bnjjj Nov 9, 2023
0b9c836
few improvements
bnjjj Nov 9, 2023
f911028
Allow log format as a string.
Nov 9, 2023
375fa8c
Fix lints
Nov 10, 2023
31bb81c
review comments
bnjjj Nov 10, 2023
0181124
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 10, 2023
71e3925
fix lint
bnjjj Nov 10, 2023
adbfb36
remove useless logging configuration
bnjjj Nov 10, 2023
6dd98e7
Merge branch 'dev' of github.com:apollographql/router into bnjjj/cust…
bnjjj Nov 10, 2023
f8aec5a
Move `experimental_logging.when_header` to `logging.experimental_when…
Nov 10, 2023
879b25a
Move `experimental_logging.when_header` to `logging.experimental_when…
Nov 10, 2023
f6d2686
Improve metrics a little and add license checks.
Nov 10, 2023
36c30ab
Rename logging_new to logging now the original logging field has gone.
Nov 10, 2023
a24679a
fix experimental notice
bnjjj Nov 10, 2023
a3074dc
Add the ability to set servicename and service namespace from logging
Nov 10, 2023
5b404ef
Make formatters use config directly rather than copying field by field.
Nov 10, 2023
746e1e7
fix test
bnjjj Nov 10, 2023
7d69ab9
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 10, 2023
1bcb533
Rename Trace to TracingCommon for consistency
Nov 10, 2023
32b1dbe
update snapshots
bnjjj Nov 10, 2023
1a4498e
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 10, 2023
5d17679
Rename `new` mode to `spec_compliant`
Nov 10, 2023
5af8456
Add resource support to logging.
Nov 10, 2023
fb00e16
Migrate telemetry.(logging|tracing|metrics) to telemetry.exporters.(l…
Nov 13, 2023
5832540
Change name of GetAttribute and GetAttributes to Selector and Selectors.
Nov 14, 2023
e7f0948
Add missing snapshot
Nov 15, 2023
8c5795b
temp
Nov 15, 2023
af7defd
Lint
Nov 15, 2023
1ad350b
Conditions tests
Nov 15, 2023
961a9d9
Fix tests
Nov 15, 2023
478c8af
Finish tests for conditions.
Nov 15, 2023
266db72
telemetry: get rid of the tracing_fmt layer and use our own optimized…
bnjjj Nov 15, 2023
cee76f4
Remove many of the dead_code statements.
Nov 15, 2023
d92a13f
Add tests for attributes, fix bugs found once tests were added.
Nov 17, 2023
4612e00
telemetry: add tests for the formatting layer (#4209)
bnjjj Nov 20, 2023
9572725
Move some code around for `defaults_for_level`.
Nov 20, 2023
4dab1b0
Add more tests for span attributes.
Nov 20, 2023
103f575
Merge branch 'dev' of github.com:apollographql/router into bnjjj/cust…
bnjjj Nov 20, 2023
9c222e2
Merge branch 'bnjjj/custom_attributes_telemetry' of github.com:apollo…
bnjjj Nov 20, 2023
18b4165
Fix dev mode
Nov 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 21 additions & 8 deletions Cargo.lock
Original file line number Diff line number Diff line change
Expand Up @@ -400,6 +400,7 @@ dependencies = [
"test-span",
"thiserror",
"tikv-jemallocator",
"time",
"tokio",
"tokio-rustls",
"tokio-stream",
Expand All @@ -416,6 +417,7 @@ dependencies = [
"tracing-core",
"tracing-futures",
"tracing-opentelemetry",
"tracing-serde",
"tracing-subscriber",
"tracing-test",
"trust-dns-resolver",
Expand Down Expand Up @@ -2220,9 +2222,13 @@ dependencies = [

[[package]]
name = "deranged"
version = "0.3.8"
version = "0.3.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2696e8a945f658fd14dc3b87242e6b80cd0f36ff04ea560fa39082368847946"
checksum = "0f32d04922c60427da6f9fef14d042d9edddef64cb9d4ce0d64d0685fbeb1fd3"
dependencies = [
"powerfmt",
"serde",
]

[[package]]
name = "derivative"
Expand Down Expand Up @@ -4906,6 +4912,12 @@ dependencies = [
"universal-hash",
]

[[package]]
name = "powerfmt"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"

[[package]]
name = "ppv-lite86"
version = "0.2.17"
Expand Down Expand Up @@ -6657,28 +6669,29 @@ dependencies = [

[[package]]
name = "time"
version = "0.3.25"
version = "0.3.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0fdd63d58b18d663fbdf70e049f00a22c8e42be082203be7f26589213cd75ea"
checksum = "c4a34ab300f2dee6e562c10a046fc05e358b29f9bf92277f30c3c8d82275f6f5"
dependencies = [
"deranged",
"itoa",
"powerfmt",
"serde",
"time-core",
"time-macros",
]

[[package]]
name = "time-core"
version = "0.1.1"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7300fbefb4dadc1af235a9cef3737cea692a9d97e1b9cbcd4ebdae6f8868e6fb"
checksum = "ef927ca75afb808a4d64dd374f00a2adf8d0fcff8e7b184af886c3c87ec4a3f3"

[[package]]
name = "time-macros"
version = "0.2.11"
version = "0.2.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb71511c991639bb078fd5bf97757e03914361c48100d52878b8e52b46fb92cd"
checksum = "4ad70d68dba9e1f8aceda7aa6711965dfec1cac869f311a51bd08b3a2ccbce20"
dependencies = [
"time-core",
]
Expand Down
2 changes: 2 additions & 0 deletions apollo-router/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,8 @@ aws-credential-types = "0.56.1"
aws-config = "0.56.1"
aws-types = "0.56.1"
sha1 = "0.10.6"
tracing-serde = "0.1.3"
time = { version = "0.3.30", features = ["serde"] }

[target.'cfg(macos)'.dependencies]
uname = "0.1.1"
Expand Down

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions apollo-router/src/context/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ pub(crate) mod extensions;

/// The key of the resolved operation name. This is subject to change and should not be relied on.
pub(crate) const OPERATION_NAME: &str = "operation_name";
pub(crate) const OPERATION_KIND: &str = "operation_kind";

/// Holds [`Context`] entries.
pub(crate) type Entries = Arc<DashMap<String, Value>>;
Expand Down
1 change: 1 addition & 0 deletions apollo-router/src/plugins/headers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -784,6 +784,7 @@ mod test {
.expect("expecting valid request"),
operation_kind: OperationKind::Query,
context: ctx,
subgraph_name: String::from("test").into(),
subscription_stream: None,
connection_closed_signal: None,
}
Expand Down
56 changes: 55 additions & 1 deletion apollo-router/src/plugins/telemetry/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,6 @@ pub(crate) struct Conf {
#[cfg(feature = "telemetry_next")]
/// Event configuration
pub(crate) events: config_new::events::Events,
#[cfg(feature = "telemetry_next")]
/// Span configuration
pub(crate) spans: config_new::spans::Spans,
#[cfg(feature = "telemetry_next")]
Expand Down Expand Up @@ -453,6 +452,8 @@ pub(crate) enum AttributeValue {
Bool(bool),
/// i64 values
I64(i64),
/// u128 values
U128(u128),
/// f64 values
F64(f64),
/// String values
Expand All @@ -461,6 +462,19 @@ pub(crate) enum AttributeValue {
Array(AttributeArray),
}

impl std::fmt::Display for AttributeValue {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
AttributeValue::Bool(val) => write!(f, "{val}"),
AttributeValue::I64(val) => write!(f, "{val}"),
AttributeValue::U128(val) => write!(f, "{val}"),
AttributeValue::F64(val) => write!(f, "{val}"),
AttributeValue::String(val) => write!(f, "{val}"),
AttributeValue::Array(val) => write!(f, "{val}"),
}
}
}

impl TryFrom<serde_json::Value> for AttributeValue {
type Error = ();
fn try_from(value: serde_json::Value) -> Result<Self, Self::Error> {
Expand All @@ -470,6 +484,9 @@ impl TryFrom<serde_json::Value> for AttributeValue {
serde_json::Value::Number(v) if v.is_i64() => {
Ok(AttributeValue::I64(v.as_i64().expect("i64 checked")))
}
serde_json::Value::Number(v) if v.is_u64() => Ok(AttributeValue::U128(
v.as_u64().expect("u64 checked") as u128,
)),
serde_json::Value::Number(v) if v.is_f64() => {
Ok(AttributeValue::F64(v.as_f64().expect("f64 checked")))
}
Expand Down Expand Up @@ -514,13 +531,26 @@ impl From<AttributeValue> for opentelemetry::Value {
match value {
AttributeValue::Bool(v) => Value::Bool(v),
AttributeValue::I64(v) => Value::I64(v),
AttributeValue::U128(v) => Value::I64(v as i64),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is odd 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, otel Value doesn't support u128 unfortunately

AttributeValue::F64(v) => Value::F64(v),
AttributeValue::String(v) => Value::String(v.into()),
AttributeValue::Array(v) => Value::Array(v.into()),
}
}
}

impl From<opentelemetry::Value> for AttributeValue {
fn from(value: opentelemetry::Value) -> Self {
match value {
opentelemetry::Value::Bool(v) => AttributeValue::Bool(v),
opentelemetry::Value::I64(v) => AttributeValue::I64(v),
opentelemetry::Value::F64(v) => AttributeValue::F64(v),
opentelemetry::Value::String(v) => AttributeValue::String(v.into()),
opentelemetry::Value::Array(v) => AttributeValue::Array(v.into()),
}
}
}

#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, PartialEq)]
#[serde(untagged, deny_unknown_fields)]
pub(crate) enum AttributeArray {
Expand All @@ -534,6 +564,17 @@ pub(crate) enum AttributeArray {
String(Vec<String>),
}

impl std::fmt::Display for AttributeArray {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
AttributeArray::Bool(val) => write!(f, "{val:?}"),
AttributeArray::I64(val) => write!(f, "{val:?}"),
AttributeArray::F64(val) => write!(f, "{val:?}"),
AttributeArray::String(val) => write!(f, "{val:?}"),
}
}
}

impl From<AttributeArray> for opentelemetry::Array {
fn from(array: AttributeArray) -> Self {
match array {
Expand All @@ -545,6 +586,19 @@ impl From<AttributeArray> for opentelemetry::Array {
}
}

impl From<opentelemetry::Array> for AttributeArray {
fn from(array: opentelemetry::Array) -> Self {
match array {
opentelemetry::Array::Bool(v) => AttributeArray::Bool(v),
opentelemetry::Array::I64(v) => AttributeArray::I64(v),
opentelemetry::Array::F64(v) => AttributeArray::F64(v),
opentelemetry::Array::String(v) => {
AttributeArray::String(v.into_iter().map(|v| v.into()).collect())
}
}
}
}

#[derive(Clone, Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields, untagged)]
pub(crate) enum SamplerOption {
Expand Down
Loading