AWS Lambda Instrumentation should use presence of context, not env var, to parent to Lambda

[anuraaga]

Currently, AWS Lambda instrumentation checks whether a Lambda-provided parent is sampled or not to determine whether to instead extract from headers.

https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/plugins/node/opentelemetry-instrumentation-aws-lambda/src/instrumentation.ts#L322

Actually Lambda always populates the environment variable, and it will be just a trace ID when env var propagation should be ignored. If we don't make sure to ignore this case, then it is never possible for the Lambda to start a sampled trace. This should be tweaked to not overwrite the parent variable

https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/plugins/node/opentelemetry-instrumentation-aws-lambda/src/instrumentation.ts#L312

[anuraaga]

/cc @wangzlei @willarmiros @NathanielRN

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[NathanielRN]

Just a quick question,

then it is never possible for the Lambda to start a sampled trace

Are you saying that Lambda cannot start a sampled trace if the parent was constructed by reading an _X_AMZN_TRACE_ID environment variable without a sampling decision? I that makes sense, because "no sampling decision" == "do not sample" IIRC. Actually it sounds a lot like open-telemetry/opentelemetry-python-contrib#649 (comment)

Which we solved by just updating the documentation to warn users that traces will be "not sampled" unless they explicitly set Sample=1; in the _X_AMZN_TRACE_ID.

[NathanielRN]

I was wondering if this is still an issue? I can see that the current implementation only overwrites the parent value if the _X_AMZN_TRACE_ID is ;Sampled=1. This should only be set if we have "active tracing" enabled on the AWS Lambda configuration, and is ;Sampled=0 if this is turned off (based on my tests on a Python Lambda).

opentelemetry-js-contrib/plugins/node/opentelemetry-instrumentation-aws-lambda/src/instrumentation.ts

Lines 352 to 355 in e5a65c6

	if (
	spanContext &&
	(spanContext.traceFlags & TraceFlags.SAMPLED) === TraceFlags.SAMPLED
	) {

There are also methods to disableAwsContextPropagation and provide a helper to extract a custom context from the Lambda event. So I would say the behavior is what we expect according to the OTel specifications for Determining a parents in Lambda Instrumentation? Please let me know what you think!

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stale for 14 days with no activity.