parser: document ltsv and logfmt parsers (#179)

ltsv and logfmt parsers are already supported, but there are no document.

- Add new parser: ltsv fluent/fluent-bit#646
- Add new parser: logfmt fluent/fluent-bit#871

I added the document about them.

Signed-off-by: Ichinose Shogo <[email protected]>

SUMMARY.md

@@ -67,6 +67,8 @@
 * [Parsers](parser/README.md)
   * [JSON Parser](parser/json.md)
   * [Regular Expression Parser](parser/regular_expression.md)
+  * [LTSV Parser](parser/ltsv.md)
+  * [Logfmt Parser](parser/logfmt.md)
   * [Decoders](parser/decoder.md)
 * [Filter Plugins](filter/README.md)
   * [Grep](filter/grep.md)

parser/README.md

@@ -26,7 +26,7 @@ Multiple parsers can be defined and each section have it own properties. The fol
 | Key | Description |
 | :--- | :--- |
 | Name | Set an unique name for the parser in question. |
-| Format | Specify the format of the parser, the available options here are: [json](json.md) or [regex](regular_expression.md). |
+| Format | Specify the format of the parser, the available options here are: [json](json.md), [regex](regular_expression.md), [ltsv](ltsv.md) or [logfmt][logfmt.md]. |
 | Regex | If format is _regex_, this option _must_ be set specifying the Ruby Regular Expression that will be used to parse and compose the structured message. |
 | Time\_Key | If the log entry provides a field with a timestamp, this option specify the name of that field. |
 | Time\_Format | Specify the format of the time field so it can be recognized and analyzed properly. Fluent-bit uses `strptime(3)` to parse time so you can ferer to [strptime documentation](https://linux.die.net/man/3/strptime) for available modifiers. |

parser/logfmt.md

@@ -0,0 +1,25 @@
+# Logfmt Parser
+
+The **logfmt** parser allows to parse the logfmt format described in https://brandur.org/logfmt .
+A more formal description is in https://godoc.org/github.com/kr/logfmt .
+
+Here is an example configuration:
+
+```python
+[PARSER]
+    Name        logfmt
+    Format      logfmt
+```
+
+The following log entry is a valid content for the parser defined above:
+
+```text
+key1=val1 key2=val2
+```
+
+After processing, it internal representation will be:
+
+```text
+[1540936693, {"key1"=>"val1",
+              "key2"=>"val2"}]
+```

parser/ltsv.md

@@ -0,0 +1,45 @@
+# LTSV Parser
+
+The **ltsv** parser allows to parse [LTSV](http://ltsv.org/) formatted texts. 
+
+Labeled Tab-separated Values (LTSV format is a variant of Tab-separated Values (TSV). Each record in a LTSV file is represented as a single line. Each field is separated by TAB and has a label and a value. The label and the value have been separated by ':'.
+
+Here is an example how to use this format in the apache access log.
+
+Config this in httpd.conf:
+
+```text
+LogFormat "host:%h\tident:%l\tuser:%u\ttime:%t\treq:%r\tstatus:%>s\tsize:%b\treferer:%{Referer}i\tua:%{User-Agent}i" combined_ltsv
+CustomLog "logs/access_log" combined_ltsv
+```
+
+The parser.conf:
+
+```python
+[PARSER]
+    Name        access_log_ltsv
+    Format      ltsv
+    Time_Key    time
+    Time_Format [%d/%b/%Y:%H:%M:%S %z]
+    Types       status:integer size:integer
+```
+
+The following log entry is a valid content for the parser defined above:
+
+```text
+host:127.0.0.1  ident:- user:-  time:[10/Jul/2018:13:27:05 +0200]       req:GET / HTTP/1.1      status:200      size:16218      referer:http://127.0.0.1/       ua:Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
+host:127.0.0.1  ident:- user:-  time:[10/Jul/2018:13:27:05 +0200]       req:GET /assets/plugins/bootstrap/css/bootstrap.min.css HTTP/1.1        status:200      size:121200     referer:http://127.0.0.1/       ua:Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
+host:127.0.0.1  ident:- user:-  time:[10/Jul/2018:13:27:05 +0200]       req:GET /assets/css/headers/header-v6.css HTTP/1.1      status:200      size:37706      referer:http://127.0.0.1/       ua:Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
+host:127.0.0.1  ident:- user:-  time:[10/Jul/2018:13:27:05 +0200]       req:GET /assets/css/style.css HTTP/1.1  status:200      size:1279       referer:http://127.0.0.1/       ua:Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0
+```
+
+After processing, it internal representation will be:
+
+```text
+[1531222025.000000000, {"host"=>"127.0.0.1", "ident"=>"-", "user"=>"-", "req"=>"GET / HTTP/1.1", "status"=>200, "size"=>16218, "referer"=>"http://127.0.0.1/", "ua"=>"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0"}]
+[1531222025.000000000, {"host"=>"127.0.0.1", "ident"=>"-", "user"=>"-", "req"=>"GET /assets/plugins/bootstrap/css/bootstrap.min.css HTTP/1.1", "status"=>200, "size"=>121200, "referer"=>"http://127.0.0.1/", "ua"=>"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0"}]
+[1531222025.000000000, {"host"=>"127.0.0.1", "ident"=>"-", "user"=>"-", "req"=>"GET /assets/css/headers/header-v6.css HTTP/1.1", "status"=>200, "size"=>37706, "referer"=>"http://127.0.0.1/", "ua"=>"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0"}]
+[1531222025.000000000, {"host"=>"127.0.0.1", "ident"=>"-", "user"=>"-", "req"=>"GET /assets/css/style.css HTTP/1.1", "status"=>200, "size"=>1279, "referer"=>"http://127.0.0.1/", "ua"=>"Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0"}]
+```
+
+The time has been converted to Unix timestamp \(UTC\).