Log files grow without limit when loggingEnabled is turned off

[markcowl]

With loggingEnabled turned off, there is no limit to the size of log files in the LogFiles directory, unless a new iisnode instance is spun up

[tjanczuk]

I would like to use this bug as an opportunity to simplify and reconcile the logging story in iisnode.

Currently there are two logging mechanisms we support: the one driven by loggingEnabled, and the one using the interceptor approach and nodeProcessCommandLine extensibility point. After this fix, only the latter will remain, but it will be wired up by default.

Here is the proposal:

1. All the native code within iisnode responsible for logging and log size quota enforcement is going away. iisnode always wires up the NUL device as stdout and stderr of node.exe processes it creates.
2. Configuration values that remain unchanged: loggingEnabled, maxLogFileSizeInKB.
3. (BREAKING CHANGE) Configuration values removed: logDirectoryNameSuffix, appendToExistingLog, logFileFlushInterval.
4. Configuration values added: logDirectory - this is the full directory name where log files will be stored relative to the entry point of the node.js application; the default is "iisnode"; maxTotalLogFileSizeInKB - the maximum total size of all log files; interceptor - fully qualified file name of the node.js application that will be launched instead of the actual application entry point targeted by the request (the fully qualified file name of the actual application entry point is provided as a first parameter to the node.js application identified by interceptor); default value of interceptor is %programfiles%\iisnode\interceptor.js; maxLogFiles - maximum number of log files in the logDirectory - defaults to 20
5. Interceptor.js ships with all SKUs of iisnode.
6. nodeProcessCommandLine default remains unchanged
7. If loggingEnabled is false, interceptor.js is a no-op and simply invokes application code passed as a second parameter to node.exe.
8. If loggingEnabled is true, intercepts.js intercepts calls to process.stdout.write and process.stderr.write to perform logging, then delegates to application code passed as a second parameter to node.exe.
9. Every time a new node.exe process starts, interceptor.js generates unique files name for stderr and stdout, and starts logging to them. The log files are created lazely on first write to stdout or stderr, respectively.
10. If the number of bytes written to a particular log file exceeds maxLogFileSizeInKB, interceptor.js closes the file, generates a new unique file name, and continues logging to the new file.
11. Interceptor.js contains logic to purge the log file directory if the total size of log files exceeds maxTotalLogFileSizeInKB or the number of log files exceeds maxLogFiles. The logic is invoked every time inteceptor.js creates a new log file (which is 9 or 10 above). Files are removed in increasing order of last modified date until the total size is reduced below the maxTotalMaxFileSizeInKB limit and the number is reduced below maxLogFiles.
12. Interceptor.js maintains an index.html file in the log directory that contains links to all individual log files. This is meant to allow HTTP access to log files in cases where the log directory is in scope of IIS static handler, but directory browsing is disabled in IIS. The index.html file is updated every time a new log file is created (i.e. 9 and 10 above).
13. The URL rewrite rules we provide by default must be modified to have the "iisnode" folder mapped to the IIS static file handler.

So where does this design leave us? By default, logs will be stored in the "iisnode" directory next to the entry point of the application. If the application is accessible at http://foo.com/server.js, logs would be accessible at http://foo.com/iisnode.

In hosting environments like Azure, the default configuration will change to specify something like logDirectory=....\LogFiles\iisnode.

[markcowl]

I think unifying to one logging system makes sense, and the node logging system gives maximum flexibility.

The only ask I would have is that all configuration settings that control logs are surfaced as simple settings in iisnode.yml (and web.config). [maxToatalLogFileSizeInKB, maxLogFileSize, logDirectory, etc.]

[tjanczuk]

Whatever we do with configuration settings there will be parity between web.config and iisnode.yml. And yes, all the settings mentioned above will be actual configuration settings (as opposed to env variables).

[yavorg]

So to summarize the breaking changes, we're removing some knobs and changing the default location where logs go if they turn on loggingEnabled, correct?

[tjanczuk]

Yes, the default location of logs will change. Now, iisnode still has loggingEnabled==true by default as of v0.1.19.

---

From: Yavor Georgiev [[email protected]]
Sent: Thursday, May 17, 2012 12:30 PM
To: Tomasz Janczuk
Subject: Re: [iisnode] Log files grow without limit when loggingEnabled is turned off (#168)

So to summarize the breaking changes, we're removing some knobs and changing the default location where logs go if they turn on loggingEnabled, correct?

---

Reply to this email directly or view it on GitHub:
#168 (comment)

[tjanczuk]

Few changes to the design outlined previously were made inline (changes in bold)

[tjanczuk]

In case of exceptions on initialization the exception is logged twice - it should be once.

[tjanczuk]

The latter issue is addressed with c4f9aee