Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit trace ID propagation for SFN w/o Hashing #537

Merged
merged 26 commits into from
Nov 18, 2024

Conversation

avedmala
Copy link
Contributor

@avedmala avedmala commented Nov 12, 2024

What does this PR do?

Forked off of #526, this PR will use context objects to generate traceIDs rather than generating them form hashes

Add logic to extract trace context from _datadog for Step Functions cases where...

  1. Root of the overall execution is a Lambda. This means we have the x-datadog-trace-id but we need to compute the x-datadog-parent-id using parent SFN context object
  2. Root of the overall execution is another Step Function. This means we need to compute both x-datadog-trace-id and x-datadog-parent-id from RootExecutionId and the parent SFN context object
  3. Keeping legacy code where there's no _datadog and we instead figure out context from the Execution and State info

Also fixed the trigger tags parsing for our use case!

Motivation

Testing Guidelines

Additional Notes

Types of Changes

  • Bug fix
  • New feature
  • Breaking change
  • Misc (docs, refactoring, dependency upgrade, etc.)

Check all that apply

  • This PR's description is comprehensive
  • This PR contains breaking changes that are documented in the description
  • This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
  • This PR impacts documentation, and it has been updated (or a ticket has been logged)
  • This PR's changes are covered by the automated tests
  • This PR collects user input/sensitive content into Datadog
  • This PR passes the integration tests (ask a Datadog member to run the tests)

@@ -415,7 +452,9 @@ def is_legacy_lambda_step_function(event):
return False

event = event.get("Payload")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Quality Violation

variable name is the same as a function parameter (...read more)

A function parameter should only be read and not be modified. If your intent is to modify the value of the parameter, return the value in the function and handle the new value in the caller of the function.

View in Datadog  Leave us feedback  Documentation

@@ -254,6 +256,14 @@ def parse_event_source_arn(source: _EventSource, event: dict, context: Any) -> s
if source.event_type == EventTypes.CLOUDWATCH_EVENTS and event.get("resources"):
return event.get("resources")[0]

# todo: testme
# e.g. arn:aws:states:us-east-1:123456789012:stateMachine:stateMachineName
if source.event_type == EventTypes.STEPFUNCTIONS:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will parse the ARN so we we get trigger tags like this
Screenshot 2024-11-13 at 1 11 23 PM

@avedmala avedmala marked this pull request as ready for review November 13, 2024 20:40
@avedmala avedmala requested a review from a team as a code owner November 13, 2024 20:40
return ""


def _generate_sfn_parent_id(context: dict) -> int:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created these helpers to pull hashing and hex related details out of extract_context_from_step_functions as it was getting pretty verbose

meta = {}
dd_data = event.get("_datadog")

if dd_data and dd_data.get("serverless-version") == "v2":

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the difference between v1 and v2? Should we start from version v2 or v1?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no v1, I think we can start from v1 instead if we want

I think @kimi-p suggested v2, maybe because the "legacy" context is implied to be v1

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, "V2" may be somewhat confusing to customers because there is no explicit reference to "V1" at all.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sgtm!

Copy link
Contributor

@kimi-p kimi-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question regarding the legacy lambda check that I think we need to update, otherwise LGTM! Thank you!

datadog_lambda/trigger.py Outdated Show resolved Hide resolved
Copy link
Contributor

@kimi-p kimi-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joeyzhao2018 Could you take a look to confirm that we are using the right extraction pattern in this PR?

Copy link
Contributor

@joeyzhao2018 joeyzhao2018 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@avedmala avedmala merged commit c4da90e into main Nov 18, 2024
51 checks passed
@avedmala avedmala deleted the avedmala/sfn-span-link-no-hash branch November 18, 2024 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants