Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

redirecting >11 times causes "MaxListenersExceededWarning" #1711

Closed
KhafraDev opened this issue Oct 17, 2022 · 7 comments · Fixed by #1715 or #2823
Closed

redirecting >11 times causes "MaxListenersExceededWarning" #1711

KhafraDev opened this issue Oct 17, 2022 · 7 comments · Fixed by #1715 or #2823
Labels
bug Something isn't working

Comments

@KhafraDev
Copy link
Member

KhafraDev commented Oct 17, 2022

Bug Description

Each redirect adds one terminated listener to fetchParams and it's never removed.

Reproducible By

import { once } from 'events'
import { createServer } from 'http'
import { fetch } from 'undici'

let redirects = 0

const server = createServer((req, res) => {
  if (redirects === 15) {
    res.end('Okay goodbye')
    return
  }

  res.writeHead(302, {
    Location: `/${redirects++}`
  })
  res.end()
}).listen(0)

await once(server, 'listening')

const url = `http://localhost:${server.address().port}`

fetch(url, { redirect: 'follow' })

Expected Behavior

Logs & Screenshots

MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 terminated listeners added to [Fetch]. Use emitter.setMaxListeners() to increase limit

Environment

Additional context

@jonseymour
Copy link

jonseymour commented Feb 23, 2024

Right, but if you have to lift the warning so high that it masks actual leaks you cease to get any benefit from the warning since if there is an actual leak in one of these event sources the warning never fires or if does fires once due a spurious condition then, when an actual leak occurs later, the code fails to emit a second warning because it has already emitted a spurious warning.

Maybe the most pragmatic thing would be to place a rate limit on the number of warnings, but ensure they are issued periodically along with the current number of listeners. That way, an actual leak could be observed (by observing a steady increase in the watermark) without necessarily polluting the logs with too many spurious warnings.

edit: of course, this would likely an upstream change.

@Uzlopak
Copy link
Contributor

Uzlopak commented Feb 23, 2024

The problem is solved by @KhafraDev. You can open an issue in node core and talk there about more strategies to improve this.

From the undici perspective the bug will be gone when #2823 is merged.

@jonseymour
Copy link

Ok, good to see the root cause is being addressed now, thanks.

@daveyarwood
Copy link

I've been running into what I think is the same or similar issue to what was reported in this issue - lots of MaxListenersExceededWarnings being logged, and my service eventually runs out of memory and dies. When I run with --trace-warnings, the stacktrace points to Undici.

It looks like the fix (#2823) was released in Undici 6.7.0, and I was hopeful that upgrading the dependency in my project would fix my issue. But after updating Undici to 6.13.0, I'm still getting a lot of MaxListenersExceededWarnings:

(node:14480) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 101 abort listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
    at _addListener (node:events:588:17)
    at EventEmitter.addListener (node:events:606:10)
    at addAbortListener (/Users/daveyarwood/code/spark/node_modules/undici/lib/core/util.js:443:10)
    at addSignal (/Users/daveyarwood/code/spark/node_modules/undici/lib/api/abort-signal.js:36:3)
    at new RequestHandler (/Users/daveyarwood/code/spark/node_modules/undici/lib/api/api-request.js:66:5)
    at Pool.request (/Users/daveyarwood/code/spark/node_modules/undici/lib/api/api-request.js:171:25)
    at /Users/daveyarwood/code/spark/node_modules/undici/lib/api/api-request.js:164:15
    at new Promise (<anonymous>)
    at Pool.request (/Users/daveyarwood/code/spark/node_modules/undici/lib/api/api-request.js:163:12)

@daveyarwood
Copy link

Another note: I did try increasing the default max event listeners to a high number:

EventEmitter.defaultMaxListeners = 5000

And I'm still seeing the warnings come up. Granted, my service is doing a lot of work in parallel (many promises being fired off), so I'm not sure if the problem might be user error. Are there best practices about using this library in many promises concurrently?

@daveyarwood
Copy link

For context, I am using undici transitively via Elasticsearch.

@KhafraDev
Copy link
Member Author

It doesn't look like elasticsearch is using fetch, which is what #2823 fixed the issue for. I would recommend opening up a new issue for this and, if possible, provide a minimal reproducible example.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
4 participants