Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rearrange span processors to avoid repeating scrubbing and other tweaking #658

Merged
merged 6 commits into from
Dec 18, 2024

Conversation

alexmojaki
Copy link
Contributor

@alexmojaki alexmojaki commented Dec 10, 2024

Previously every span processor was wrapped in MainSpanProcessorWrapper which does various things including scrubbing which is quite slow. This meant that computation was inefficiently repeated for each processor. By default there are two such processors, one for the console and one for sending to logfire, so the computation was doubled. This PR bundles each processor into a SynchronousMultiSpanProcessor which simply delegates to each of them, and that is wrapped in a MainSpanProcessorWrapper to prevent the computation duplication. The same is done for PendingSpanProcessor. All this means that this code:

import logfire

logfire.configure()

with logfire.span('outer'):
    for _ in range(1000):
        with logfire.span('inner'):
            pass

runs about 30-40% faster.

TailSamplingProcessor has also been moved up in the tree of processors and is only created once instead of multiple times, which saves on computation and memory and prevents creating pending spans for spans that have already finished.

Copy link

cloudflare-workers-and-pages bot commented Dec 10, 2024

Deploying logfire-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 2d0b0c0
Status: ✅  Deploy successful!
Preview URL: https://cd090ce7.logfire-docs.pages.dev
Branch Preview URL: https://alex-scrub-once.logfire-docs.pages.dev

View logs

Copy link

codecov bot commented Dec 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (7e6b140) to head (2d0b0c0).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #658   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files          137       137           
  Lines        10920     10929    +9     
  Branches      1520      1524    +4     
=========================================
+ Hits         10920     10929    +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alexmojaki alexmojaki requested a review from Kludex December 10, 2024 13:38
@alexmojaki alexmojaki marked this pull request as ready for review December 10, 2024 13:38
@alexmojaki alexmojaki requested review from Kludex and removed request for Kludex December 18, 2024 10:24
@alexmojaki alexmojaki enabled auto-merge (squash) December 18, 2024 11:02
@alexmojaki alexmojaki merged commit 9e629da into main Dec 18, 2024
12 checks passed
@alexmojaki alexmojaki deleted the alex/scrub-once branch December 18, 2024 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants