Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: EventEmitter memory leak detected. Seems occurs on file rotation. #2065

Open
ArtemijArtemev opened this issue Feb 9, 2022 · 7 comments

Comments

@ArtemijArtemev
Copy link

ArtemijArtemev commented Feb 9, 2022

The problem

(node:17202) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
(Use `node --trace-warnings ...` to show where the warning was created)
MaxListenersExceededWarning
Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 error listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
    at _addListener (events.js:390:17)
    at WriteStream.prependListener (events.js:413:14)
    at onFinished (internal/streams/writable.js:692:10)
    at WriteStream.Writable.end (internal/streams/writable.js:594:7)
    at File._endStream (/node_modules/winston/lib/winston/transports/file.js:526:18)
    at File.logged (/node_modules/winston/lib/winston/transports/file.js:172:12)
    at afterWrite (/node_modules/winston/node_modules/readable-stream/lib/_stream_writable.js:469:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)
(node:17202) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
MaxListenersExceededWarning
Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 finish listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
    at _addListener (events.js:390:17)
    at WriteStream.addListener (events.js:406:10)
    at onFinished (internal/streams/writable.js:691:10)
    at WriteStream.Writable.end (internal/streams/writable.js:594:7)
    at File._endStream (/node_modules/winston/lib/winston/transports/file.js:526:18)
    at File.logged (/node_modules/winston/lib/winston/transports/file.js:172:12)
    at afterWrite (/node_modules/winston/node_modules/readable-stream/lib/_stream_writable.js:469:3)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)

What version of Winston presents the issue?

v3.5.1

What version of Node are you using?

v14.16.0

If this worked in a previous version of Winston, which was it?

No response

Minimum Working Example

const winston = require('winston');
const winston_graylog2 = require('winston-graylog2');
const winston_cluster = require('winston-cluster');
const cluster = require('cluster');
const config = require('./config');
const mkdirp = require('mkdirp');
const {createLogDirs, makeLogsPath} = require('../../commons/utils');

const serviceName = config.get('serviceName');
const transports = [];
const settings = {
	"silent": config.get("log:silent"),
	"exitOnError": config.get("log:exitOnError"),
	"level": config.get("log:level")
};

let logger;

if (cluster.isMaster) {

	const default_format = winston.format.printf((info) => {
		return `[${info.level.toUpperCase()}] ${info.metadata.timestamp} [${info.metadata.service}]: ${info.message} `;
	});

	const printf = winston.format.printf(info =>  {
		return `${info.timestamp} - ${info.metadata.service} - ${info.level}: ${info.message}`;
	});

	const baseFormatter = winston.format.combine(
		winston.format.errors({ stack: true }), // Handle errors (was automagic in winston@2)
		winston.format.splat(),                 // Handle splat (was automagic in winston@2)
		winston.format.timestamp(),             // { timestamp: true }
	);

	const defaultFormatter = winston.format.combine(
		baseFormatter, printf
	);

	const consoleFormatter = winston.format.combine(
		baseFormatter, winston.format.colorize(), printf
	);

	transports.push(
		new (winston.transports.Console)({
			format: consoleFormatter
		})
	);

	if (config.get('log:log_filesystem')) {

		const Logs_path = makeLogsPath(__dirname + '/..', config);
		createLogDirs(mkdirp, Logs_path);

		transports.push(
			new winston.transports.File(
				{
					name: 'info',
					json: false,
					format: defaultFormatter,
					filename: Logs_path.info_log['full_path'],
					level: 'info',
					maxFiles: config.get('log:info_max_files'),
					maxsize: config.get('log:info_max_size'),
					tailable: true
				})
		);

		transports.push(
			new winston.transports.File(
				{
					name: 'error',
					json: false,
					format: defaultFormatter,
					filename: Logs_path.error_log['full_path'],
					level: 'error',
					maxFiles: config.get('log:error_max_files'),
					maxsize: config.get('log:error_max_size'),
					tailable: true
				})
		);

	}

	if (config.get('log:log_graylog')) {

		transports.push(
			new winston_graylog2(config.get('log:graylog'))
		);

	}

	settings.transports = transports;


	logger = winston.createLogger({
		level: settings.level ? settings.level : 'info',
		defaultMeta: { service: serviceName },
		format: settings.format ? settings.format : winston.format.combine(
			winston.format.timestamp(),
			winston.format.metadata(),
			default_format
		),
		transports: settings.transports ? settings.transports : new winston.transports.Console(),
		silent: settings.silent,
		exitOnError: settings.exitOnError
	});

} else {
	logger = winston.createLogger({transports: [
        new winston_cluster({
            level: 'info',
        }),
    ]});
}

module.exports = {
	logger: logger
};

Additional information

No response

@ArtemijArtemev ArtemijArtemev changed the title [Bug]: EventEmitter memory leak detected [Bug]: EventEmitter memory leak detected. Seems occurs on rotation file. Feb 9, 2022
@ArtemijArtemev ArtemijArtemev changed the title [Bug]: EventEmitter memory leak detected. Seems occurs on rotation file. [Bug]: EventEmitter memory leak detected. Seems occurs on file rotation. Feb 9, 2022
@wbt
Copy link
Contributor

wbt commented Feb 11, 2022

Is that really a minimal reproducible example?
Often, the process of cutting things out, seeing if the issue is still observed, and if so cutting some more on repeat, is very helpful in determining exactly which cut step makes the error go away.

@mohanrajanna15
Copy link

We are seeing this on our production systems, can we please expedite this?

@wbt
Copy link
Contributor

wbt commented Mar 17, 2022

@mohanrajanna15 Absolutely; you can dive in and try to figure out what is happening and why, and even better if you can, how to fix it.
Winston as a project has no funding, no dedicated developers' time, and no warranty. It has a decent open-source code base and a few volunteers who try to be helpful in the few minutes of time we can find here and there between other tasks.
For the most part, issues only get fixed when people affected by them are both annoyed and skilled enough to actually spend the time to figure out exactly why the issue is happening and how to correct that. If you've got a dev shop where this presents a real issue in production systems, you might have the motivation to be that party for this one!

@max-mayorov
Copy link

We are having similar warning in our application (Nextjs)
MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 uncaughtException listeners added to [process]. Use emitter.setMaxListeners() to increase limit

@DABH
Copy link
Contributor

DABH commented Apr 10, 2024

If you have high log throughput (legitimately), you might see this happen (I.e. not necessarily a bug). See other thread you posted on as well as #1334

@max-mayorov
Copy link

Why will this warning be generated on high log throughput? Still seems like a bug - is one single event listener not enough? Why more has to be added?

@DABH
Copy link
Contributor

DABH commented Apr 11, 2024

See #1791 (comment) , we can discuss over there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants