[Packetbeat] New SIP protocol (elastic#21221)

* new protocol:sip:Make a directory and Readme file.

new protocol:sip:update include list

new protocol:sip:initial blank file

DNSをベースとしてまずはDNSでやっている内容を解析開始・・・コメント付け中。

読み進めた分更新

パーサの実装を開始

パーサをのデコーダをsipPlugin内に移設もろもろ

リクエスト、レスポンス判定を追加

SIPのパース, SDPのパース追加、パーサをいろいろばらばらに。醜い・・・

ヘッダパース系メソッドをsipMessageのメンバに変更

バッファリングの仕組みのプロトタイプ作成

ファイル分割、オブジェクト毎にファイルを分割。その他実装を進めているところ。とちゅう

publish methodの実装

一部エラーハンドリングを追加

TODO更新とデータ構造を追加

ヘッダ処理諸々追加,途中

README TODO更新
'

実働確認用に追加

フィールド名にsip.を付与、unixtimenanoをフィールドに追加(デフォルトのtimestampだとSIP信号を並び替えるのに精度不足なため。)

フィールド命名規則を更新

Added english description

change field name

align the indents

fields.yml update about timestamp

テストケース追加

added testcase

add testcases

add testcases and fixed some bug cases

add test cases

add testcase parseSIPHeaders

fixed bug cases.

comment and refactoring

update testcases

add monitoring element named 'sip.message_ignored'

move publish function call from expireBuffer to callback function when buffer expired.

add testcase, bufferExpire

remove unnecessary pkg

add testcase at publish method

add, edit and migrate test cases

modify time duration

change timer code

remove fragmneted process

translate comments

add linux amd64 binary

Comments translated

update informations

add windows bin

update TODO list

add no mandantory header parse check

Add compact-form test case

Add compact-form test case

Add compact-form test case

Add compact-form test case

support compact form

TODO list update

add sip uri parser

add detail mode

add binary

remove unnecessary file

bug fix:broken when response parse in detail mode

bug fix:detail mode

modify detail mode

modify detail mode

Update Readme about compact form and parse detail sip header and request-uri

Update Readme about compact form and parse detail sip header and request-uri

Update Readme about compact form and parse detail sip header and request-uri

Update Readme about compact form and parse detail sip header and request-uri

expand config parsing options

edit variable names

arrangement and add text README file

refine Readme message

update Readme text

update Readme on Configuration

Go coding style was checked with golint

Erase duplicate field in field.yml, move src and dst fields into sip filed

Update docs, fields.asciidoc

Update docs, fields.asciidoc

* Fixes and style changes

* Refactor to be more similar to http parser and add system tests

* Add event action

* Add related fields

* Update fields and docs

* Add sip to docs

* Add beta warning

* Parse SDP, Contact, Via and auth

* Add suggestions

Co-authored-by: tj8000rpm <[email protected]>

CHANGELOG.next.asciidoc

@@ -746,6 +746,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
   port. {pull}19209[19209]
 - Add ECS fields for x509 certs, event categorization, and related IP info. {pull}19167[19167]
 - Add 100-continue support {issue}15830[15830] {pull}19349[19349]
+- Add initial SIP protocol support {pull}21221[21221]
 
 
 *Functionbeat*

packetbeat/_meta/config/beat.docker.yml.tmpl

@@ -36,3 +36,6 @@ packetbeat.protocols.cassandra:
 
 packetbeat.protocols.tls:
   ports: [443, 993, 995, 5223, 8443, 8883, 9243]
+
+packetbeat.protocols.sip:
+  ports: [5060]

packetbeat/_meta/config/beat.reference.yml.tmpl

@@ -531,6 +531,19 @@ packetbeat.protocols:
   # Set to true to publish fields with null values in events.
   #keep_null: false
 
+- type: sip
+  # Configure the ports where to listen for SIP traffic. You can disable the SIP protocol by commenting out the list of ports.
+  ports: [5060]
+
+  # Parse the authorization headers
+  parse_authorization: true
+
+  # Parse body contents (only when body is SDP)
+  parse_body: true
+
+  # Preserve original contents in event.original
+  keep_original: true
+
 {{header "Monitored processes"}}
 
 # Packetbeat can enrich events with information about the process associated

packetbeat/_meta/config/beat.yml.tmpl

@@ -101,6 +101,10 @@ packetbeat.protocols:
     - 8883  # Secure MQTT
     - 9243  # Elasticsearch
 
+- type: sip
+  # Configure the ports where to listen for SIP traffic. You can disable the SIP protocol by commenting out the list of ports.
+  ports: [5060]
+
 {{header "Elasticsearch template setting"}}
 
 setup.template.settings:

packetbeat/_meta/sample_outputs/sip.json

@@ -0,0 +1,77 @@
+{
+    "@metadata.beat": "packetbeat",
+    "@metadata.type": "_doc",
+    "client.ip": "192.168.1.2",
+    "client.port": 5060,
+    "destination.ip": "212.242.33.35",
+    "destination.port": 5060,
+    "event.action": "sip_register",
+    "event.category": [
+        "network",
+        "protocol",
+        "authentication"
+    ],
+    "event.dataset": "sip",
+    "event.duration": 0,
+    "event.kind": "event",
+    "event.original": "REGISTER sip:sip.cybercity.dk SIP/2.0\r\nVia: SIP/2.0/UDP 192.168.1.2;branch=z9hG4bKnp112903503-43a64480192.168.1.2;rport\r\nFrom: <sip:[email protected]>;tag=6bac55c\r\nTo: <sip:[email protected]>\r\nCall-ID: 578222729-4665d775@578222732-4665d772\r\nContact:  <sip:[email protected]:5060;line=aca6b97ca3f5e51a>;expires=1200;q=0.500\r\nExpires: 1200\r\nCSeq: 75 REGISTER\r\nContent-Length: 0\r\nAuthorization: Digest username=\"voi18062\",realm=\"sip.cybercity.dk\",uri=\"sip:192.168.1.2\",nonce=\"1701b22972b90f440c3e4eb250842bb\",opaque=\"1701a1351f70795\",nc=\"00000001\",response=\"79a0543188495d288c9ebbe0c881abdc\"\r\nMax-Forwards: 70\r\nUser-Agent: Nero SIPPS IP Phone Version 2.0.51.16\r\n\r\n",
+    "event.sequence": 75,
+    "event.type": [
+        "info"
+    ],
+    "network.application": "sip",
+    "network.community_id": "1:dOa61R2NaaJsJlcFAiMIiyXX+Kk=",
+    "network.iana_number": "17",
+    "network.protocol": "sip",
+    "network.transport": "udp",
+    "network.type": "ipv4",
+    "related.hosts": [
+        "sip.cybercity.dk"
+    ],
+    "related.ip": [
+        "192.168.1.2",
+        "212.242.33.35"
+    ],
+    "related.user": [
+        "voi18062"
+    ],
+    "server.ip": "212.242.33.35",
+    "server.port": 5060,
+    "sip.auth.realm": "sip.cybercity.dk",
+    "sip.auth.scheme": "Digest",
+    "sip.auth.uri.host": "192.168.1.2",
+    "sip.auth.uri.original": "sip:192.168.1.2",
+    "sip.auth.uri.scheme": "sip",
+    "sip.call_id": "578222729-4665d775@578222732-4665d772",
+    "sip.contact.uri.host": "sip.cybercity.dk",
+    "sip.contact.uri.original": "sip:[email protected]",
+    "sip.contact.uri.scheme": "sip",
+    "sip.contact.uri.username": "voi18062",
+    "sip.cseq.code": 75,
+    "sip.cseq.method": "REGISTER",
+    "sip.from.tag": "6bac55c",
+    "sip.from.uri.host": "sip.cybercity.dk",
+    "sip.from.uri.original": "sip:[email protected]",
+    "sip.from.uri.scheme": "sip",
+    "sip.from.uri.username": "voi18062",
+    "sip.max_forwards": 70,
+    "sip.method": "REGISTER",
+    "sip.to.uri.host": "sip.cybercity.dk",
+    "sip.to.uri.original": "sip:[email protected]",
+    "sip.to.uri.scheme": "sip",
+    "sip.to.uri.username": "voi18062",
+    "sip.type": "request",
+    "sip.uri.host": "sip.cybercity.dk",
+    "sip.uri.original": "sip:sip.cybercity.dk",
+    "sip.uri.scheme": "sip",
+    "sip.user_agent.original": "Nero SIPPS IP Phone Version 2.0.51.16",
+    "sip.version": "2.0",
+    "sip.via.original": [
+        "SIP/2.0/UDP 192.168.1.2;branch=z9hG4bKnp112903503-43a64480192.168.1.2;rport"
+    ],
+    "source.ip": "192.168.1.2",
+    "source.port": 5060,
+    "status": "OK",
+    "type": "sip",
+    "user.name": "voi18062"
+}