Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_segment_async doesn't get/pass the current Lambda segment #349

Open
abivolmv opened this issue Jul 27, 2022 · 4 comments
Open

in_segment_async doesn't get/pass the current Lambda segment #349

abivolmv opened this issue Jul 27, 2022 · 4 comments

Comments

@abivolmv
Copy link

Hello,
I am opening this issue on request of @NathanielRN after a discussion with him in #310 .

The problem is that when we have API GW -> Lambda invocation and in the Lambda we use async , then we get 2 separate traces. One from API GW to Lambda and another from in_segment_async . To tie them together I have to use the following workaround:

    # get current segment
    trace_id = xray_recorder.get_trace_entity().trace_id
    parent_id = xray_recorder.current_segment().id
    # enable xray to work with async
    # even though Lambda has its own segment we need to create one async segment
    xray_recorder.configure(context=AsyncContext())
    async with xray_recorder.in_segment_async('add_brochure', traceid=trace_id, parent_id=parent_id) as segment:
       ...

But even then most of the times (usually works first time after deployment only) it doesn't work because I get the error:

[ERROR] AttributeError: 'NoneType' object has no attribute 'trace_id'
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 19, in lambda_handler
    trace_id = xray_recorder.get_trace_entity().trace_id
@carolabadeer
Copy link
Contributor

Hello!
Can you clarify the reason why your workaround only works once and then throws this error? Have you tried any other workarounds that do not result in this error being thrown?

@abivolmv
Copy link
Author

I can only suspect that it is because of the either nature of Lambda : first run is a clean state and then it can get reused, or because not all traces are sampled and when they are not - there is no segment ? I have no working example so far.

@abivolmv
Copy link
Author

Could I spark some interest in this issue ? ^^

@carolabadeer
Copy link
Contributor

carolabadeer commented Oct 24, 2022

After looking into this issue further, there may be a few root causes that contribute to the issue you are seeing. As linked in the previous issue (#203), there is a known issue with using AsyncContext and LambdaContext together, and perhaps the current LambdaContext doesn't allow for the creation of a new segment (since there already exists a facade segment).

There are a few workarounds you can try in this case:

  • Removing the xray_recorder.configure(context=AsyncContext()) line and re-running the lambda function to see if the same error persists.
  • Keep AsyncContext but instead of doing xray_recorder.get_trace_entity().trace_id to fetch the trace ID, retrieve it from the lambda env variable (LAMBDA_TRACE_HEADER_KEY) instead. This method can be helpful in extracting the trace ID

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants