Logging to disk by default

[montanaflynn]

Shouldn't Kong not be logging to files by default? As of now installations are writing to access.log and error.log. Writing to disk for every request will slow Kong down plus we have a plugin for file logging.

If you're using Kong and enable the file logging plugin it will write to the same errors.log file that is also being written to for nginx errors, which would cause problems for any existing nginx log parsers.

Since the file logging plugin does more then just log errors, maybe it should be split into two logs, one for access and the other for errors with a choice to enable one or the other or both. It would be nice if the log paths are customizable as well.

Here are my suggestions:

1. Turn off nginx access.log & errors.log logging by default, make them an option in kong.yml
2. Make the file logging plugin write to it's own kong-access.log and kong-errors.log

[subnetmarco]

Yes we should disable access_log by default.

A little overview on how nginx logs work. Nginx supports two built-in logs:

• access_logs, for nginx access logs
• error_log, for everything else including errors and info messages (note: not just errors).

info or debug messages are being logged into the error_log handler as well, and while I do agree is misleading, there is nothing we can do since that is part of the nginx core (I think error_log should be renamed to system_log or something like that).

The "File Log" logs info messages containing data in a different format compared to a regular access log (for example including lots of different additional properties like consumer_id and api_id) using the built-in nginx log handler.

We can't turn off error.log because that would prevent the errors from being logged (error logs don't slow down requests, they are printed only when an error happens), and it would prevent the "File Log" to work since the "File Log" plugin is leveraging the built-in nginx logging handler which happens to log everything, including info messages, into the error_log handler.

[montanaflynn]

So we're basically stuck putting Kong's custom access logs into nginx's own error.log? That's rather disappointing as it's confusing, breaks existing nginx log parsers and makes checking the Kong-specific logs harder than it should be. There's no way to create a separate log file for Kong?

[montanaflynn]

Also when looking at the File Log plugin format I can't help but think it needs some attention. It starts with Lua specific stuff, turns into stringified JSON and then ends in a weird sentence.

2015/05/07 20:52:10 [info] 69#0: *11 [lua] log.lua:8: execute(): {"started_at":1431031929000,"url":"\/","response":{"size":"10763","headers":{"x-powered-by":"Express","Connection":"close","Content-Type":"text\/html; charset=utf-8","cf-ray":"1e2fce984cfd11e9-SJC","x-ratelimit-limit":"1000","x-ratelimit-remaining":"999","Transfer-Encoding":"chunked","via":"kong\/0.2.0-2","set-cookie":"__cfduid=db41de3c132ee96d74a67c587e3ff19891431031929; expires=Fri, 06-May-16 20:52:09 GMT; path=\/; domain=.mockbin.com; HttpOnly","server":"cloudflare-nginx","vary":"Accept-Encoding","etag":"W\/\"WjyUny1hiU0eFTCRGSBgnQ==\""}},"api":{"public_dns":"mockbin.com","id":"2fdce05d-4f19-4f16-cfbe-ae2550dae250","target_url":"http:\/\/mockbin.com\/","name":"mockbin","created_at":1431031831000},"request":{"size":"74","headers":{"accept":"*\/*","host":"mockbin.com","user-agent":"curl\/7.29.0"}},"status":200,"ip":"127.0.0.1"} while logging request, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://104.24.103.220:80/", host: "mockbin.com"

[subnetmarco]

@montanaflynn you see that format because that's the format that nginx internally logs messages, so the "File Log" plugin doesn't break any nginx log parser, and the value section of the log is the JSON representation.

We could create our own file handler and write to a custom file as long as it's non-blocking.

[montanaflynn]

Maybe "break" was the wrong word but it will affect any existing tools that monitor nginx error logs since by default they don't put access logs or JSON in the error.log. I think the plugin should output to it's own files for all the reasons stated above.

[subnetmarco]

access_logs have been disabled by default

[subnetmarco]

Now the file log plugin supports a file path in the path property where it will log requests.