Skip to content

Commit

Permalink
Handle IPv6 zone id in IIS filebeat ingest pipeline (elastic#9869)
Browse files Browse the repository at this point in the history
IIS logs can include zone ids when using IPv6, this is correctly parsed
but geoip processor doesn't accept these addresses. Create a temporary
field without the zone id to be used by geoip processor.

(cherry picked from commit d59ae8c)
  • Loading branch information
jsoriano committed Jan 7, 2019
1 parent 1ca42f0 commit 3c7c650
Show file tree
Hide file tree
Showing 4 changed files with 38 additions and 1 deletion.
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ https://github.com/elastic/beats/compare/v6.6.0...6.x[Check the HEAD diff]
*Auditbeat*

*Filebeat*

- Correctly parse `December` or `Dec` in the Syslog input. {pull}9349[9349]
- Fix improperly set config for CRI Flag in Docker Input {pull}8899[8899]
- Just enabling the `elasticsearch` fileset and starting Filebeat no longer causes an error. {pull}8891[8891]
Expand All @@ -66,6 +67,7 @@ https://github.com/elastic/beats/compare/v6.6.0...6.x[Check the HEAD diff]
- Stop runners disabled by hints after previously being started. {pull}9305[9305]
- Fix saved objects in filebeat haproxy dashboard. {pull}9417[9417]
- Fixed a memory leak when harvesters are closed. {pull}7820[7820]
- Support IPv6 addresses with zone id in IIS ingest pipeline. {issue}9836[9836] {pull}9869[9869]

*Heartbeat*

Expand Down
16 changes: 15 additions & 1 deletion filebeat/module/iis/error/ingest/default.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,24 @@
"field": "iis.error.time"
}
}, {
"geoip": {
"grok": {
"field": "iis.error.remote_ip",
"patterns": [
"%{NOZONEIP:iis.error.remote_ip_geoip}"
],
"pattern_definitions": {
"NOZONEIP": "[^%]*"
}
}
}, {
"geoip": {
"field": "iis.error.remote_ip_geoip",
"target_field": "iis.error.geoip"
}
}, {
"remove": {
"field": "iis.error.remote_ip_geoip"
}
}],
"on_failure" : [{
"set" : {
Expand Down
5 changes: 5 additions & 0 deletions filebeat/module/iis/error/test/ipv6_zone_id.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2018-12-30 13:48:36
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri streamid sc-status s-siteid s-reason s-queuename
2018-12-30 14:22:07 ::1%0 49958 ::1%0 80 - - - - - - Timer_ConnectionIdle -
16 changes: 16 additions & 0 deletions filebeat/module/iis/error/test/ipv6_zone_id.log-expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[
{
"@timestamp": "2018-12-30T14:22:07.000Z",
"ecs.version": "1.0.0-beta2",
"event.dataset": "error",
"event.module": "iis",
"iis.error.queue_name": "-",
"iis.error.reason_phrase": "Timer_ConnectionIdle",
"iis.error.remote_ip": "::1%0",
"iis.error.remote_port": "49958",
"iis.error.server_ip": "::1%0",
"iis.error.server_port": "80",
"input.type": "log",
"log.offset": 195
}
]

0 comments on commit 3c7c650

Please sign in to comment.