MaxListenersExceededWarning: Possible EventEmitter memory leak detected

[gohmc]

When I ran load test against lb4 backed application, i got flooded with the above warnings and observed the lb4 application slow down significantly (4x slower than normal with very poor throughput). However, the application didn't crash.

It looks like too many interceptors need to watch the event loop? It can be reproduce with one of the lb4 example:

lb4 example greeting-app
cd loopback4-example-greeting-app
npm start // with --trace-warnings
Use a load test application against http://127.0.0.1:3000/greet/world. Example:
autocannon -c 10 -d 40 -p 10 localhost:3000/greet/world

(node:30929) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 bind listeners added to [RestServer]. Use emitter.setMaxListeners() to increase limit
at _addListener (events.js:268:17)
at RestServer.addListener (events.js:284:10)
at RequestContext.addParentEventListener (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/context.js:152:22)
at RequestContext.setupEventHandlersIfNeeded (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/context.js:113:14)
at RequestContext.subscribe (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/context.js:414:14)
at ContextView.open (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/context-view.js:46:43)
at Object.createViewGetter (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/context-view.js:171:10)
at Object.resolve (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/core/dist/extension-point.js:58:26)
at /home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolver.js:102:30
at /home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolution-session.js:92:53
at Object.tryWithFinally (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/value-promise.js:157:18)
at Function.runWithInjection (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolution-session.js:92:32)
at resolve (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolver.js:99:61)
at /home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolver.js:176:16
at Object.resolveList (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/value-promise.js:130:32)
at resolveInjectedArguments (/home/ec2-user/projects/loopback/loopback4-example-greeting-app/node_modules/@loopback/context/dist/resolver.js:159:28)

Thanks for looking into this.

[raymondfeng]

The extension point uses listeners to watch events from the request -> rest server -> app context hierarchy so that newly bound extensions can be picked up. The following up will remove the listener:

https://github.com/strongloop/loopback-next/blob/master/packages/rest/src/request-context.ts#L105

In your case, if you have more than 10 concurrent requests, you may set the warning message. We probably should increase the limit of listeners for Context class.

The performance issue might be related to #4356.

[raymondfeng]

@gohmc Do you use https://github.com/strongloop/loopback-next/tree/master/examples/greeting-app for benchmarking?

[gohmc]

@raymondfeng, the example was used to reproduce the issue only, it was another production application that encounter performance issue. there was a lot of MaxListenersExceededWarning in the log file.

latest development is we used another example (hello-world) and run the same test and did not see neither the warning or performance issue. so far this issue is related to interceptor.

it will be difficult if not impossible to determine max listeners in real world (eg. spike). maybe rather than pub/sub pattern, the observer mechanism can adopt call chain pattern to disassociate from the event loop.

will look deeper to #4356, thanks!