Rearrange span processors to avoid repeating scrubbing and other tweaking #658
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously every span processor was wrapped in
MainSpanProcessorWrapper
which does various things including scrubbing which is quite slow. This meant that computation was inefficiently repeated for each processor. By default there are two such processors, one for the console and one for sending to logfire, so the computation was doubled. This PR bundles each processor into aSynchronousMultiSpanProcessor
which simply delegates to each of them, and that is wrapped in aMainSpanProcessorWrapper
to prevent the computation duplication. The same is done forPendingSpanProcessor
. All this means that this code:runs about 30-40% faster.
TailSamplingProcessor
has also been moved up in the tree of processors and is only created once instead of multiple times, which saves on computation and memory and prevents creating pending spans for spans that have already finished.