Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] contrib/valyala/fasthttp.v1 memory leak #2961

Closed
0angelic0 opened this issue Nov 4, 2024 · 1 comment · Fixed by #2962
Closed

[BUG] contrib/valyala/fasthttp.v1 memory leak #2961

0angelic0 opened this issue Nov 4, 2024 · 1 comment · Fixed by #2962
Labels
apm:ecosystem contrib/* related feature requests or bugs bug unintended behavior that has to be fixed needs-triage New issues that have not yet been triaged

Comments

@0angelic0
Copy link
Contributor

The implementation of contrib/valyala/fasthttp.v1 WrapHandler has a memory leak issue. We use this in our production system and it cause memory leak. After profiling and review the code we found the issue is in the WrapHandler function.

Version of dd-trace-go

v1.69.0

Describe what happened:

The issue is the spanOpts variable in WrapHandler function. Now the code initialize the slice of tracer.StartSpanOption as WrapHandler function scope and use it in a closure anonymous function that will return as a fasthttp.RequestHandler. The usage is to be appended with the defaultSpanOptions and tracer context (from tracer.Extract and tracer.ChildOf). After that the spanOpts is used to create a Span.

The problem is now spanOpts is the same slice for every http request that will come in and be handled. So. the slice will growth indefinitely since every request will append several tracer.StartSpanOption into the same slice. And eventually causes the memory leak problem.

Describe what you expected:

Expect the spanOpts to not growing every time the request came in. So it doesn't cause a memory leak.

It should be initialized in the closure anonymous function that will return as a fasthttp.RequestHandler instead of WrapHandler function. So it will be garbage collected every time the request handler end.

Steps to reproduce the issue:

Just write a small hello world fasthttp program and load test it to see the memory leak.

This is the memory usage for 7 times of 10,000 requests each.

Image

Additional environment details (Version of Go, Operating System, etc.):

@0angelic0 0angelic0 added the bug unintended behavior that has to be fixed label Nov 4, 2024
@github-actions github-actions bot added apm:ecosystem contrib/* related feature requests or bugs needs-triage New issues that have not yet been triaged labels Nov 4, 2024
@0angelic0
Copy link
Contributor Author

I'm proposing the fix as to move the initialization of spanOpts into the closure anonymous function that will return as a fasthttp.RequestHandler. Already tested in my fork and the memory leak was fixed.

This is the memory usage for 7 times of 10,000 requests each. No memory leak.

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
apm:ecosystem contrib/* related feature requests or bugs bug unintended behavior that has to be fixed needs-triage New issues that have not yet been triaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant