Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(otel): spans hierarchy + sampled flags and propagation #10663

Merged
merged 3 commits into from
Apr 17, 2023

Conversation

samugi
Copy link
Member

@samugi samugi commented Apr 12, 2023

Summary

Continuation of #10583 and #10655 (otel plugin side).

Addresses the following issues:

Checklist

  • The Pull Request has tests
  • [not needed] There's an entry in the CHANGELOG (will add a unified changelog entry later including other fixes)
  • [not needed] There is a user-facing docs PR against https://github.com/Kong/docs.konghq.com - PUT DOCS PR HERE

Full changelog

  • [Implement ...]

Issue reference

KAG-1108

@samugi samugi marked this pull request as draft April 12, 2023 21:33
@samugi samugi changed the title Fix/otel span hierarchy sampled flags fix(otel): spans hierarchy and non-sampled spans propagation Apr 12, 2023
@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch from 0f736cd to c9e1ce3 Compare April 13, 2023 07:14
@samugi samugi changed the title fix(otel): spans hierarchy and non-sampled spans propagation fix(otel): spans hierarchy and sampled flags propagation Apr 13, 2023
@samugi samugi changed the title fix(otel): spans hierarchy and sampled flags propagation fix(otel): spans hierarchy + sampled flags and propagation Apr 13, 2023
@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch from c9e1ce3 to 516a6c1 Compare April 13, 2023 11:03
@pull-request-size pull-request-size bot added size/M and removed size/L labels Apr 13, 2023
@samugi samugi marked this pull request as ready for review April 13, 2023 11:04
addresses span hierarchy for otel, similarly to what
Kong/kong-ee#4904 did for dd
address propagation of non-sampled spans, similarly to what
Kong/kong-ee#5054 did for dd
@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch from 516a6c1 to 3719b95 Compare April 13, 2023 20:48
kong/plugins/opentelemetry/handler.lua Outdated Show resolved Hide resolved
kong/plugins/opentelemetry/handler.lua Outdated Show resolved Hide resolved
@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch 4 times, most recently from fe98a81 to 40faabc Compare April 14, 2023 14:29
@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch from 40faabc to 25e893b Compare April 14, 2023 14:37
@@ -500,6 +517,7 @@ local function set(conf_header_type, found_header_type, proxy_span, conf_default
end

if conf_header_type == "ot" or found_header_type == "ot" then
to_ot_trace_id = to_ot_trace_id or require "kong.plugins.opentelemetry.otlp".to_ot_trace_id
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved here otherwise requiring "propagation" from "instrumentation" was resulting in otlp to be initialized too soon

@samugi samugi force-pushed the fix/otel-span-hierarchy-sampled-flags branch from 25e893b to a09f69e Compare April 14, 2023 17:47
@@ -82,6 +83,15 @@ function _M.balancer(ctx)
local try_count = balancer_data.try_count
local upstream_connect_time = split(var.upstream_connect_time, ", ", "jo")

local last_try_balancer_span
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to have those decoupled from the OTEL plugin. Eventually, we will also decouple the creation of root span and "last_try_balancer_span` too.

@samugi samugi merged commit 0205797 into master Apr 17, 2023
@samugi samugi deleted the fix/otel-span-hierarchy-sampled-flags branch April 17, 2023 07:12
@kikito
Copy link
Member

kikito commented Apr 17, 2023

@samugi This will need to be cherry-picked into EE

ms2008 added a commit that referenced this pull request Jun 1, 2023
Zipkin manages its spans and calls `finish` on them, and the result is
that `finish` is called twice if instrumentations are enabled. Then an
assertion error is generated, resulting in the span not being sent
correctly. This was an issue introduced with
#10663

So my approach is not to store the propagation span if the reuse
parameter is not specified, in order to avoid the traces not being
generated correctly under that.
samugi pushed a commit that referenced this pull request Jun 1, 2023
* fix(zipkin): do not reuse `propagated_span` by default

Zipkin manages its spans and calls `finish` on them, and the result is
that `finish` is called twice if instrumentations are enabled. Then an
assertion error is generated, resulting in the span not being sent
correctly. This was an issue introduced with
#10663

So my approach is not to store the propagation span if the reuse
parameter is not specified, in order to avoid the traces not being
generated correctly under that.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants