Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New protocol support: SIP #7181

Closed
wants to merge 32 commits into from
Closed

Conversation

tj8000rpm
Copy link
Contributor

@tj8000rpm tj8000rpm commented May 26, 2018

Hi all. I implemented a new protocol SIP to packetbeat. #152
The SIP(Session Initiation Protocol) is a communications protocol for signaling and controlling multimedia communication sessions. SIP is used many VoIP applications at not only enterprise uses but also telecom careers.

SIP is text-base protocol like HTTP. But SIP has various unique features like :

  • SIP is server-client model, but it roles may changes call by call.
  • SIP is request-response model, but server may (usualy) reply many responses for one request.
  • There many requests and responses in one call.
  • It is not know when the call will end.

Therefore, I implemented with following plans.

  • Published for each SIP message(request or response)
  • Additional timestamp to make it possible to keep message order
  • Parse and store each header field and SDP body(option)
  • Received raw message is stored as text value(option)

More detail is writen in README.md

@elasticmachine
Copy link
Collaborator

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

func TestParseDetailNameAddr(t *testing.T) {
sip := sipPlugin{}
name_addr:="\"display_name\"<sip:[email protected]:5060;user=phone>;hogehoge"
display_name, user_info, host, port, uri_params, params:=sip.parseDetailNameAddr(name_addr)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var display_name should be displayName
don't use underscores in Go names; var user_info should be userInfo
don't use underscores in Go names; var uri_params should be uriParams


func TestParseDetailNameAddr(t *testing.T) {
sip := sipPlugin{}
name_addr:="\"display_name\"<sip:[email protected]:5060;user=phone>;hogehoge"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var name_addr should be nameAddr


func TestPaseDetailURI(t *testing.T){
var uri string
var user_info string; var host string; var port string; var uri_params []string

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var user_info should be userInfo
don't use underscores in Go names; var uri_params should be uriParams


raw_text :="test raw string"
method_text:="INVITE"
phrase_text:="OK"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var phrase_text should be phraseText

sip.setFromConfig(&cfg)

raw_text :="test raw string"
method_text:="INVITE"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var method_text should be methodText

from:=`"0311112222"<sip:[email protected]:5060>;tag=FromTag`
to:=`<sip:[email protected]>;tag=ToTag`
cseq_num:=6789
cseq_method:="INVITE"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var cseq_method should be cseqMethod

status_text:="Ringing"
from:=`"0311112222"<sip:[email protected]:5060>;tag=FromTag`
to:=`<sip:[email protected]>;tag=ToTag`
cseq_num:=6789

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var cseq_num should be cseqNum

sip.initDetailOption()
raw_text :="test raw string"
status_number:=(uint16)(180)
status_text:="Ringing"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var status_text should be statusText

sip.parseDetail=true
sip.initDetailOption()
raw_text :="test raw string"
status_number:=(uint16)(180)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var status_number should be statusNumber


sip.parseDetail=true
sip.initDetailOption()
raw_text :="test raw string"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var raw_text should be rawText

func TestParseDetailNameAddr(t *testing.T) {
sip := sipPlugin{}
name_addr:="\"display_name\"<sip:[email protected]:5060;user=phone>;hogehoge"
display_name, user_info, host, port, uri_params, params:=sip.parseDetailNameAddr(name_addr)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var display_name should be displayName
don't use underscores in Go names; var user_info should be userInfo
don't use underscores in Go names; var uri_params should be uriParams


func TestParseDetailNameAddr(t *testing.T) {
sip := sipPlugin{}
name_addr:="\"display_name\"<sip:[email protected]:5060;user=phone>;hogehoge"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var name_addr should be nameAddr


func TestPaseDetailURI(t *testing.T){
var uri string
var user_info string; var host string; var port string; var uri_params []string

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var user_info should be userInfo
don't use underscores in Go names; var uri_params should be uriParams


raw_text :="test raw string"
method_text:="INVITE"
phrase_text:="OK"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var phrase_text should be phraseText

sip.setFromConfig(&cfg)

raw_text :="test raw string"
method_text:="INVITE"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var method_text should be methodText

from:=`"0311112222"<sip:[email protected]:5060>;tag=FromTag`
to:=`<sip:[email protected]>;tag=ToTag`
cseq_num:=6789
cseq_method:="INVITE"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var cseq_method should be cseqMethod

status_text:="Ringing"
from:=`"0311112222"<sip:[email protected]:5060>;tag=FromTag`
to:=`<sip:[email protected]>;tag=ToTag`
cseq_num:=6789

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var cseq_num should be cseqNum

sip.initDetailOption()
raw_text :="test raw string"
status_number:=(uint16)(180)
status_text:="Ringing"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var status_text should be statusText

sip.parseDetail=true
sip.initDetailOption()
raw_text :="test raw string"
status_number:=(uint16)(180)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var status_number should be statusNumber


sip.parseDetail=true
sip.initDetailOption()
raw_text :="test raw string"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var raw_text should be rawText


raw_text :="test raw string"
method_text:="INVITE"
phrase_text:="OK"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var phrase_text should be phraseText

headers := map[string][]common.NetString{}
to_h:=[]common.NetString{}
to_h=append(to_h,common.NetString(to))
from_h:=[]common.NetString{}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var from_h should be fromH


sip.parseDetail=true
sip.initDetailOption()
raw_text :="test raw string"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var raw_text should be rawText

methodText := "INVITE"
from := `"0311112222"<sip:[email protected]:5060>;tag=FromTag`
to := `<sip:[email protected]>`
requestUri := "sip:+8137890123;npdi;[email protected]:5060;user=phone;transport=udp"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

var requestUri should be requestURI

prevChar = ' '
pos := -1
_ = port
in_addr := false

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var in_addr should be inAddr

return user_info, host, port, params
}

func (sip *sipPlugin) parseDetailNameAddr(addr string) (display_name string, user_info string, host string, port string, addrparams []string, params []string) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; method result display_name should be displayName
don't use underscores in Go names; method result user_info should be userInfo

pos := -1
ppos := -1
epos := len(addr)
in_v6 := false

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var in_v6 should be inV6

sip.publishMessage(sipMsg)
}

func (sip *sipPlugin) parseDetailURI(addr string) (user_info string, host string, port string, params []string) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; method result user_info should be userInfo

sdp_lines := strings.Split(string(rawData), "\r\n")
for i := 0; i < len(sdp_lines); i++ {

key_val := strings.SplitN(sdp_lines[i], "=", 2)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var key_val should be keyVal


func (msg *sipMessage) parseBody_SDP(rawData []byte) (body *map[string][]common.NetString, err error) {
body = &map[string][]common.NetString{}
sdp_lines := strings.Split(string(rawData), "\r\n")

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var sdp_lines should be sdpLines

return nil
}

func (msg *sipMessage) parseBody_SDP(rawData []byte) (body *map[string][]common.NetString, err error) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; method parseBody_SDP should be parseBodySDP

_ = err
if err != nil {
debugf("%s : parseError", lower_case_content_type)
return fmt.Errorf("invalid %s format.", lower_case_content_type)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error strings should not be capitalized or end with punctuation or a newline

// Switch the function with body content type
// TODO: Now, it is supported only SDP,
// more SIP body application support (I planning to support SIP-I(ISUP)/Multi-part)
lower_case_content_type := strings.ToLower(string(getLastElementStrArray(contenttype_array)))

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't use underscores in Go names; var lower_case_content_type should be lowerCaseContentType

@st1t
Copy link
Contributor

st1t commented Dec 27, 2018

Hi @tj8000rpm .
Do you have still intention that merge to this pull request.
If you have it, please read follow document.
and, I think that It settle Travis's problem to execute 'make update'.

https://www.elastic.co/guide/en/beats/devguide/current/beats-contributing.html#update-scripts

@tj8000rpm tj8000rpm requested a review from a team as a code owner December 30, 2018 04:49
@tj8000rpm
Copy link
Contributor Author

Hi @st1t ,
Thank you for your advice. I read the docs and try it.
Thanks

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for submitting this!

I have a few suggestions to improve, but overall looks good.

@@ -0,0 +1,375 @@
# SIP(Session Initiation Protocol) for packetbeat
The SIP(Session Initiation Protocol) is a communications protocol for signaling and controlling multimedia communication sessions. SIP is used many VoIP applications at not only enterprise uses but also telecom careers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and in some other places you use careers to mean carriers.

- name: status-phrase
type: keyword
description: >
SIP Response , status phrase
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra space before comma


// MessegeStatus
const (
SipStatusReceived = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use iota initialization for these and the other constants in this file
https://github.com/golang/go/wiki/Iota


// Convert compact form to full form
if len(key) == 1 {
switch key {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you replace this switch with a map[byte]string ?

@tj8000rpm tj8000rpm requested a review from a team as a code owner January 4, 2019 15:25
@elasticmachine
Copy link
Collaborator

Pinging @elastic/siem (Team:SIEM)

@khou
Copy link

khou commented Mar 6, 2020

Out of curiosity, is this no longer being considered?

@tj8000rpm
Copy link
Contributor Author

hi all.
there are people who are interested this feature, so I'm going to try resolve conflicts and apply to new version.

@rinderbe
Copy link

Hi @tj8000rpm, i've waited such a long time for someone to take on this task. Looks like you're almost on the finish line here :)
Any plans for taking it to the line and beyond?

Appreciate your work!

/Robin

@vnil1994
Copy link

I've also been waiting for SIP support for a very long time now. Do you have any ETA on when it might be available for version 7?

Thanks!

@cla-checker-service
Copy link

cla-checker-service bot commented May 5, 2020

💚 CLA has been signed

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 5, 2020

❕ Build Aborted

Either there was a build timeout or someone aborted the build.'}

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Branch indexing]

  • Start Time: 2020-05-13T15:47:06.792+0000

  • Duration: 125 min 36 sec (7476256)

Log output

Expand to view the last 100 lines of log output

[2020-05-13T16:51:16.348Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-05-13T16:51:16.348Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-05-13T16:51:16.348Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-05-13T16:51:16.348Z] + FILE=journalbeat/build/coverage/full.cov
[2020-05-13T16:51:16.348Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-05-13T16:51:19.449Z] Stage "Elastic Agent x-pack" skipped due to when conditional
[2020-05-13T16:51:19.452Z] Stage "Elastic Agent x-pack Windows" skipped due to when conditional
[2020-05-13T16:51:19.453Z] Stage "Elastic Agent Mac OS X" skipped due to when conditional
[2020-05-13T16:51:19.454Z] Stage "Filebeat oss" skipped due to when conditional
[2020-05-13T16:51:19.455Z] Stage "Filebeat x-pack" skipped due to when conditional
[2020-05-13T16:51:19.456Z] Stage "Filebeat Mac OS X" skipped due to when conditional
[2020-05-13T16:51:19.457Z] Stage "Filebeat Windows" skipped due to when conditional
[2020-05-13T16:51:19.458Z] Stage "Heartbeat" skipped due to when conditional
[2020-05-13T16:51:19.459Z] Stage "Auditbeat oss" skipped due to when conditional
[2020-05-13T16:51:19.460Z] Stage "Auditbeat x-pack" skipped due to when conditional
[2020-05-13T16:51:19.463Z] Stage "Libbeat" skipped due to when conditional
[2020-05-13T16:51:19.465Z] Stage "Libbeat x-pack" skipped due to when conditional
[2020-05-13T16:51:19.466Z] Stage "Metricbeat OSS Unit tests" skipped due to when conditional
[2020-05-13T16:51:19.473Z] Stage "Metricbeat OSS Integration tests" skipped due to when conditional
[2020-05-13T16:51:19.474Z] Stage "Metricbeat Python integration tests" skipped due to when conditional
[2020-05-13T16:51:19.475Z] Stage "Metricbeat x-pack" skipped due to when conditional
[2020-05-13T16:51:19.476Z] Stage "Metricbeat crosscompile" skipped due to when conditional
[2020-05-13T16:51:19.477Z] Stage "Metricbeat Mac OS X" skipped due to when conditional
[2020-05-13T16:51:19.479Z] Stage "Metricbeat Windows" skipped due to when conditional
[2020-05-13T16:51:19.483Z] Stage "dockerlogbeat" skipped due to when conditional
[2020-05-13T16:51:19.485Z] Stage "Winlogbeat" skipped due to when conditional
[2020-05-13T16:51:19.486Z] Stage "Winlogbeat Windows x-pack" skipped due to when conditional
[2020-05-13T16:51:19.488Z] Stage "Functionbeat" skipped due to when conditional
[2020-05-13T16:51:19.489Z] Stage "Journalbeat" skipped due to when conditional
[2020-05-13T16:51:19.489Z] Stage "Generators" skipped due to when conditional
[2020-05-13T16:51:19.490Z] Stage "Kubernetes" skipped due to when conditional
[2020-05-13T16:51:19.805Z] Stage "Heartbeat" skipped due to when conditional
[2020-05-13T16:51:19.807Z] Stage "Auditbeat oss" skipped due to when conditional
[2020-05-13T16:51:19.808Z] Stage "Libbeat" skipped due to when conditional
[2020-05-13T16:51:19.809Z] Stage "Metricbeat x-pack" skipped due to when conditional
[2020-05-13T16:51:19.811Z] Stage "dockerlogbeat" skipped due to when conditional
[2020-05-13T16:51:19.813Z] Stage "Winlogbeat" skipped due to when conditional
[2020-05-13T16:51:19.814Z] Stage "Functionbeat" skipped due to when conditional
[2020-05-13T16:51:19.815Z] Stage "Journalbeat" skipped due to when conditional
[2020-05-13T16:51:19.817Z] Stage "Generators" skipped due to when conditional
[2020-05-13T16:51:21.709Z] Stage "Heartbeat" skipped due to when conditional
[2020-05-13T16:51:21.713Z] Stage "Auditbeat oss" skipped due to when conditional
[2020-05-13T16:51:21.715Z] Stage "Libbeat" skipped due to when conditional
[2020-05-13T16:51:21.716Z] Stage "Metricbeat x-pack" skipped due to when conditional
[2020-05-13T16:51:21.717Z] Stage "Winlogbeat" skipped due to when conditional
[2020-05-13T16:51:21.718Z] Stage "Functionbeat" skipped due to when conditional
[2020-05-13T16:51:21.721Z] Stage "Generators" skipped due to when conditional
[2020-05-13T16:51:22.400Z] Stage "Heartbeat" skipped due to when conditional
[2020-05-13T16:51:22.411Z] Stage "Auditbeat oss" skipped due to when conditional
[2020-05-13T16:51:22.412Z] Stage "Libbeat" skipped due to when conditional
[2020-05-13T16:51:22.414Z] Stage "Functionbeat" skipped due to when conditional
[2020-05-13T16:51:22.415Z] Stage "Generators" skipped due to when conditional
[2020-05-13T16:51:23.257Z] Stage "Auditbeat oss" skipped due to when conditional
[2020-05-13T16:51:23.262Z] Stage "Generators" skipped due to when conditional
[2020-05-13T16:51:34.586Z] Still waiting to schedule task
[2020-05-13T16:51:34.586Z] All nodes of label ‘ubuntu&&immutable’ are offline
[2020-05-13T17:37:44.861Z] Running on beats-ci-immutable-ubuntu-1604-1589391372931288742 in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181
[2020-05-13T17:51:18.794Z] Cancelling nested steps due to timeout
[2020-05-13T17:51:20.361Z] Failed in branch Packetbeat
[2020-05-13T17:51:20.678Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-05-13T17:51:21.129Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-05-13T17:51:21.169Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats/Lint
[2020-05-13T17:51:21.913Z] + cat
[2020-05-13T17:51:21.913Z] + /usr/local/bin/runbld ./runbld-script
[2020-05-13T17:51:21.913Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-05-13T17:51:28.878Z] runbld>>> runbld started
[2020-05-13T17:51:28.878Z] runbld>>> 1.6.11/a66728ff8f4356963772e6e6d2069392fa06acbe
[2020-05-13T17:51:29.889Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-7181' in order of occurrence in the config (last value wins).
[2020-05-13T17:51:30.943Z] runbld>>> Debug logging enabled.
[2020-05-13T17:51:30.943Z] runbld>>> Storing result
[2020-05-13T17:51:31.237Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-05-13T17:51:31.237Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200513175130-2C85567D
[2020-05-13T17:51:31.237Z] runbld>>> Adding system facts.
[2020-05-13T17:51:31.856Z] runbld>>> Adding vcs info for the latest commit:  bb2b8318aed8c76e970bd781056c6f3d9a4bfd3e
[2020-05-13T17:51:32.143Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-05-13T17:51:32.143Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-05-13T17:51:32.143Z] Processing JUnit reports with runbld...
[2020-05-13T17:51:32.143Z] + echo 'Processing JUnit reports with runbld...'
[2020-05-13T17:51:32.441Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-05-13T17:51:32.441Z] runbld>>> DURATION: 11ms
[2020-05-13T17:51:32.441Z] runbld>>> STDOUT: 40 bytes
[2020-05-13T17:51:32.441Z] runbld>>> STDERR: 49 bytes
[2020-05-13T17:51:32.441Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-05-13T17:51:32.441Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-05-13T17:51:33.963Z] runbld>>> Storing build metadata: 
[2020-05-13T17:51:33.963Z] runbld>>> Adding test report.
[2020-05-13T17:51:33.963Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-05-13T17:51:34.975Z] runbld>>> Found 0 test output files
[2020-05-13T17:51:34.975Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 0 Skipped: 0
[2020-05-13T17:51:34.975Z] runbld>>> Storing result
[2020-05-13T17:51:35.256Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-05-13T17:51:35.256Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200513175130-2C85567D
[2020-05-13T17:51:35.256Z] runbld>>> Email notification disabled by environment variable.
[2020-05-13T17:51:35.256Z] runbld>>> Slack notification disabled by environment variable.
[2020-05-13T17:51:41.435Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181
[2020-05-13T17:51:41.803Z] [INFO] getVaultSecret: Getting secrets
[2020-05-13T17:51:41.920Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-05-13T17:51:43.336Z] + chmod 755 generate-build-data.sh
[2020-05-13T17:51:43.336Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/117 ABORTED 7476256
[2020-05-13T17:51:43.887Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/117/steps/?limit=10000 -o steps-info.json

@tj8000rpm
Copy link
Contributor Author

@vnil1994 and @rinderbe,
Thank you for following this feature. I adapted this feature to Beats new version.
It works in my local environment and almost tests passed my local test. but it seems to that my CLA was expired.

@adriansr
Copy link
Contributor

adriansr commented May 8, 2020

cla/check

@adriansr
Copy link
Contributor

adriansr commented May 8, 2020

Hi @tj8000rpm

I'm trying to get this merged (finally!).

It seems there is a problem with the CLA. I see that the last few commits are not attributed to your user, but to Hiroshi Tsuji <ho-tsuji@....> (no github username). This might be caused by you accidentally pushing from an email address that is not associated to your github account. Can you fix this, either by changing the author on those commits or by adding that email to your github profile?

Also, this needs a changelog entry, in CHANGELOG.next.asciidoc under Added > Packetbeat, something in the lines of:

Added support for decoding SIP protocol. {pull}7181[7181]

But feel free to use your own wording.

@tj8000rpm
Copy link
Contributor Author

@adriansr
Thank you for your advice. I accidentally worked with the settings for the company.
It is difficult to me all historical email addr change to private one, so I added a work email addr to github setting.

And I also added a CHANGELOG entry.

@elasticmachine
Copy link
Collaborator

elasticmachine commented May 29, 2020

❕ Build Aborted

Either there was a build timeout or someone aborted the build.'}

Pipeline View Test View Changes Artifacts

Expand to view the summary

Build stats

  • Build Cause: [Branch indexing]

  • Start Time: 2020-06-18T17:17:30.965+0000

  • Duration: 122 min 53 sec

Test stats 🧪

Test Results
Failed 0
Passed 180
Skipped 1
Total 181

Log output

Expand to view the last 100 lines of log output

[2020-06-18T18:04:20.190Z] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2020-06-18T18:04:20.190Z] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2020-06-18T18:04:20.190Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2020-06-18T18:04:20.190Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2020-06-18T18:04:20.190Z] 	at java.lang.Thread.run(Thread.java:748)
[2020-06-18T18:04:20.190Z] No artifacts found that match the file pattern "**/build/TEST*.out". Configuration error?
[2020-06-18T18:04:20.563Z] + curl -sSLo codecov https://codecov.io/bash
[2020-06-18T18:04:20.824Z] + FILE=auditbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f auditbeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + FILE=filebeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f filebeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + FILE=heartbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f heartbeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + FILE=libbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f libbeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + FILE=metricbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f metricbeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + FILE=packetbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] + [ -f packetbeat/build/coverage/full.cov ]
[2020-06-18T18:04:20.824Z] + bash codecov -f packetbeat/build/coverage/full.cov
[2020-06-18T18:04:20.824Z] 
[2020-06-18T18:04:20.824Z]   _____          _
[2020-06-18T18:04:20.824Z]  / ____|        | |
[2020-06-18T18:04:20.824Z] | |     ___   __| | ___  ___ _____   __
[2020-06-18T18:04:20.824Z] | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
[2020-06-18T18:04:20.824Z] | |___| (_) | (_| |  __/ (_| (_) \ V /
[2020-06-18T18:04:20.824Z]  \_____\___/ \__,_|\___|\___\___/ \_/
[2020-06-18T18:04:20.824Z]                               Bash-20200602-f809a24
[2020-06-18T18:04:20.824Z] 
[2020-06-18T18:04:20.824Z] 
[2020-06-18T18:04:20.824Z] ==> Jenkins CI detected.
[2020-06-18T18:04:20.825Z]     project root: .
[2020-06-18T18:04:20.825Z]     Fixing merge commit SHA
[2020-06-18T18:04:20.825Z]     Yaml found at: codecov.yml
[2020-06-18T18:04:20.825Z]     -> Found 1 reports
[2020-06-18T18:04:20.825Z] ==> Detecting git/mercurial file structure
[2020-06-18T18:04:21.087Z] ==> Reading reports
[2020-06-18T18:04:21.087Z]     + packetbeat/build/coverage/full.cov bytes=429582
[2020-06-18T18:04:21.087Z] ==> Appending adjustments
[2020-06-18T18:04:21.087Z]     https://docs.codecov.io/docs/fixing-reports
[2020-06-18T18:04:29.235Z]     + Found adjustments
[2020-06-18T18:04:29.235Z] ==> Gzipping contents
[2020-06-18T18:04:29.496Z] ==> Uploading reports
[2020-06-18T18:04:29.496Z]     url: https://codecov.io
[2020-06-18T18:04:29.496Z]     query: branch=PR-7181&commit=cdbfe48b15181e9d9bd9be4cdfc3039f5ecee2d1&build=6&build_url=https%3A%2F%2Fbeats-ci.elastic.co%2Fjob%2FBeats%2Fjob%2Fbeats-beats-mbp%2Fjob%2FPR-7181%2F6%2F&name=&tag=&slug=elastic%2Fbeats&service=jenkins&flags=&pr=7181&job=
[2020-06-18T18:04:29.496Z]     -> Pinging Codecov
[2020-06-18T18:04:29.496Z] https://codecov.io/upload/v4?package=bash-20200602-f809a24&token=secret&branch=PR-7181&commit=cdbfe48b15181e9d9bd9be4cdfc3039f5ecee2d1&build=6&build_url=https%3A%2F%2Fbeats-ci.elastic.co%2Fjob%2FBeats%2Fjob%2Fbeats-beats-mbp%2Fjob%2FPR-7181%2F6%2F&name=&tag=&slug=elastic%2Fbeats&service=jenkins&flags=&pr=7181&job=
[2020-06-18T18:04:29.756Z] HTTP 400
[2020-06-18T18:04:29.756Z] Please provide the repository token to upload reports via `-t :repository-token`
[2020-06-18T18:04:29.756Z] + FILE=winlogbeat/build/coverage/full.cov
[2020-06-18T18:04:29.756Z] + [ -f winlogbeat/build/coverage/full.cov ]
[2020-06-18T18:04:29.756Z] + FILE=journalbeat/build/coverage/full.cov
[2020-06-18T18:04:29.756Z] + [ -f journalbeat/build/coverage/full.cov ]
[2020-06-18T19:19:02.966Z] Cancelling nested steps due to timeout
[2020-06-18T19:19:03.024Z] Failed in branch Filebeat x-pack Mac OS X
[2020-06-18T19:19:03.161Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-06-18T19:19:03.472Z] + find . -type f -name TEST*.xml -path */build/* -delete
[2020-06-18T19:19:03.483Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats/Lint
[2020-06-18T19:19:03.570Z] Running in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats/Packetbeat-oss
[2020-06-18T19:19:03.925Z] + cat
[2020-06-18T19:19:03.925Z] + /usr/local/bin/runbld ./runbld-script
[2020-06-18T19:19:03.925Z] Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
[2020-06-18T19:19:10.511Z] runbld>>> runbld started
[2020-06-18T19:19:10.511Z] runbld>>> 1.6.12/f45d832f2ba0aa2722ab4ec1fda8ad140f027f8b
[2020-06-18T19:19:11.900Z] runbld>>> The following profiles matched the job 'Beats/beats-beats-mbp/PR-7181' in order of occurrence in the config (last value wins).
[2020-06-18T19:19:13.283Z] runbld>>> Debug logging enabled.
[2020-06-18T19:19:13.283Z] runbld>>> Storing result
[2020-06-18T19:19:13.283Z] runbld>>> Store result: created {:total 2, :successful 2, :failed 0} 1
[2020-06-18T19:19:13.283Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200618191913-649F9537
[2020-06-18T19:19:13.283Z] runbld>>> Adding system facts.
[2020-06-18T19:19:14.227Z] runbld>>> Adding vcs info for the latest commit:  8decdc0532b55e99fb468b70a6a59d015e3474a8
[2020-06-18T19:19:14.228Z] runbld>>> >>>>>>>>>>>> SCRIPT EXECUTION BEGIN >>>>>>>>>>>>
[2020-06-18T19:19:14.228Z] runbld>>> Adding /usr/lib/jvm/java-8-openjdk-amd64/bin to the path.
[2020-06-18T19:19:14.228Z] Processing JUnit reports with runbld...
[2020-06-18T19:19:14.228Z] + echo 'Processing JUnit reports with runbld...'
[2020-06-18T19:19:14.800Z] runbld>>> <<<<<<<<<<<< SCRIPT EXECUTION END <<<<<<<<<<<<
[2020-06-18T19:19:14.800Z] runbld>>> DURATION: 32ms
[2020-06-18T19:19:14.800Z] runbld>>> STDOUT: 40 bytes
[2020-06-18T19:19:14.800Z] runbld>>> STDERR: 49 bytes
[2020-06-18T19:19:14.800Z] runbld>>> WRAPPED PROCESS: SUCCESS (0)
[2020-06-18T19:19:14.800Z] runbld>>> Searching for build metadata in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-06-18T19:19:16.182Z] runbld>>> Storing build metadata: 
[2020-06-18T19:19:16.182Z] runbld>>> Adding test report.
[2020-06-18T19:19:16.182Z] runbld>>> Searching for junit test output files with the pattern: TEST-.*\.xml$ in: /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181/src/github.com/elastic/beats
[2020-06-18T19:19:17.125Z] runbld>>> Found 1 test output files
[2020-06-18T19:19:17.386Z] runbld>>> Test output logs contained: Errors: 0 Failures: 0 Tests: 181 Skipped: 1
[2020-06-18T19:19:17.386Z] runbld>>> Storing result
[2020-06-18T19:19:17.386Z] runbld>>> FAILURES: 0
[2020-06-18T19:19:17.647Z] runbld>>> Store result: updated {:total 2, :successful 2, :failed 0} 2
[2020-06-18T19:19:17.647Z] runbld>>> BUILD: https://c150076387b5421f9154dfbf536e5c60.us-west1.gcp.cloud.es.io:9243/build-1587637540455/t/20200618191913-649F9537
[2020-06-18T19:19:17.907Z] runbld>>> Email notification disabled by environment variable.
[2020-06-18T19:19:17.907Z] runbld>>> Slack notification disabled by environment variable.
[2020-06-18T19:19:23.439Z] Running on Jenkins in /var/lib/jenkins/workspace/Beats_beats-beats-mbp_PR-7181
[2020-06-18T19:19:23.546Z] [INFO] getVaultSecret: Getting secrets
[2020-06-18T19:19:23.602Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2020-06-18T19:19:24.467Z] + chmod 755 generate-build-data.sh
[2020-06-18T19:19:24.467Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/6 ABORTED 7313240
[2020-06-18T19:19:24.467Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/6/steps/?limit=10000 -o steps-info.json
[2020-06-18T19:19:24.718Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/6/tests/?status=FAILED -o tests-errors.json
[2020-06-18T19:19:24.968Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/Beats/beats-beats-mbp/PR-7181/runs/6/log/ -o pipeline-log.txt

@marc-gr
Copy link
Contributor

marc-gr commented Jun 10, 2020

jenkins run tests

@marc-gr
Copy link
Contributor

marc-gr commented Aug 27, 2020

@tj8000rpm Let's try to merge this once again 💪

It would be nice to update the branch, since CHANGELOG is conflicted and probably would be a good idea to run make fmt update one more time. Since it is quite behind, LMK if you want to sort it out yourself or want us to take over from this point.

In general it looks fine, if you can would be nice to rename the files to snake_case i.e.: sip_message instead of sipMessage, to follow general conventions.

Thanks for your patience one more time ❤️ hopefully we'll get this through sooner than later 🎉

@marc-gr
Copy link
Contributor

marc-gr commented Sep 15, 2020

Hello @tj8000rpm!

Please let us know if you need any help or want us to do the changes ourselves :)

Thanks! ❤️

@marc-gr marc-gr self-assigned this Sep 16, 2020
@ebeahan
Copy link
Member

ebeahan commented Sep 22, 2020

Linking an open issue with some possible SIP mappings that align well with ECS: elastic/ecs#420

@marc-gr Would you mind working with us to adjust the fields from this PR with the ones from the linked issue? We still need to review them a bit, but feel they are pretty good already.

cc @jamiehynds @webmat

@marc-gr
Copy link
Contributor

marc-gr commented Sep 22, 2020

Linking an open issue with some possible SIP mappings that align well with ECS: elastic/ecs#420

@marc-gr Would you mind working with us to adjust the fields from this PR with the ones from the linked issue? We still need to review them a bit, but feel they are pretty good already.

cc @jamiehynds @webmat

That is awesome!

I have been taking a look at this branch and the merge/rebase of it is quite tricky since is quite old. @tj8000rpm since it seems that this is something we want to move forward soon, I'll be opening a new PR based on this one to solve the issues and add the new field mappings so you do not lose authorship since you did all the work ❤️ 🙇‍♂️

@marc-gr marc-gr mentioned this pull request Sep 22, 2020
4 tasks
@marc-gr
Copy link
Contributor

marc-gr commented Sep 22, 2020

@ebeahan @tj8000rpm I opened #21221 to follow with the changes and ECS mappings. I am closing this to avoid confusion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.