Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Cisco support for the Syslog parser #10760

Merged
merged 8 commits into from
Feb 18, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- The `elasticsearch/deprecation` fileset now indexes the `component` field under `elasticsearch` instead of `elasticsearch.server`. {pull}10445[10445]
- Remove field `kafka.log.trace.full` from kafka.log fielset. {pull}10398[10398]
- Change field `kafka.log.class` for kafka.log fileset from text to keyword. {pull}10398[10398]
- Address add_kubernetes_metadata processor issue where old source field is
- Address add_kubernetes_metadata processor issue where old source field is
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this change intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I presume I wont be the only one removing the additional space, I propose we make the change :)

still used for matcher. {issue}10505[10505] {pull}10506[10506]
- Change type of haproxy.source from text to keyword. {pull}10506[10506]
- Rename `event.type` to `suricata.eve.event_type` in Suricata module because event.type is reserved for future use by ECS. {pull}10575[10575]
Expand Down Expand Up @@ -185,6 +185,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix bad bytes count in `docker` input when filtering by stream. {pull}10211[10211]
- Fixed data types for roles and indices fields in `elasticsearch/audit` fileset {pull}10307[10307]
- Ensure `source.address` is always populated by the nginx module (ECS). {pull}10418[10418]
- Add support for Cisco syslog format used by their switch. {pull}10760[10760]
- Cover empty request data, url and version in Apache2 module{pull}10730[10730]

*Heartbeat*
Expand Down
6 changes: 6 additions & 0 deletions filebeat/_meta/fields.common.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,12 @@
The input type from which the event was generated. This field is set to the value specified
for the `type` option in the input section of the Filebeat config file.

- name: event.sequence
type: long
required: false
description: >
The sequence number of this event.

- name: syslog.facility
type: long
required: false
Expand Down
12 changes: 12 additions & 0 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6585,6 +6585,18 @@ required: True
The input type from which the event was generated. This field is set to the value specified for the `type` option in the input section of the Filebeat config file.


--

*`event.sequence`*::
+
--
type: long

required: False

The sequence number of this event.


--

*`syslog.facility`*::
Expand Down
2 changes: 1 addition & 1 deletion filebeat/include/fields.go

Large diffs are not rendered by default.

13 changes: 13 additions & 0 deletions filebeat/input/syslog/event.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ type event struct {
nanosecond int
year int
loc *time.Location
sequence int
}

// newEvent() return a new event.
Expand All @@ -84,6 +85,7 @@ func newEvent() *event {
minute: -1,
second: -1,
year: time.Now().Year(),
sequence: -1,
}
}

Expand Down Expand Up @@ -269,6 +271,17 @@ func (s *event) HasPid() bool {
return s.pid > 0
}

// SetSequence set the sequence number for this event.
func (s *event) SetSequence(b []byte) {
s.sequence = bytesToInt(b)
}

// Sequence returns the sequence number of the event when defined,
// otherwise return -1.
func (s *event) Sequence() int {
return s.sequence
}

// SetNanoSecond sets the nanosecond.
func (s *event) SetNanosecond(b []byte) {
// We assume that we receive a byte array representing a nanosecond, this might not be
Expand Down
5 changes: 5 additions & 0 deletions filebeat/input/syslog/input.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ import (

// Parser is generated from a ragel state machine using the following command:
//go:generate ragel -Z -G2 parser.rl -o parser.go
//go:generate go fmt parser.go

// Severity and Facility are derived from the priority, theses are the human readable terms
// defined in https://tools.ietf.org/html/rfc3164#section-4.1.1.
Expand Down Expand Up @@ -250,6 +251,10 @@ func createEvent(ev *event, metadata inputsource.NetworkMetadata, timezone *time
f["event"] = event
f["process"] = process

if ev.Sequence() != -1 {
f["event.sequence"] = ev.Sequence()
}

return &beat.Event{
Timestamp: ev.Timestamp(timezone),
Meta: common.MapStr{
Expand Down
26 changes: 26 additions & 0 deletions filebeat/input/syslog/input_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,32 @@ func TestProgram(t *testing.T) {
})
}

func TestSequence(t *testing.T) {
t.Run("is set", func(t *testing.T) {
e := newEvent()
e.SetMessage([]byte("hello world"))
e.SetProgram([]byte("sudo"))
e.SetSequence([]byte("123"))
m := dummyMetadata()
event := createEvent(e, m, time.Local, logp.NewLogger("syslog"))
v, err := event.GetValue("event.sequence")
if !assert.NoError(t, err) {
return
}
assert.Equal(t, v, 123)
})

t.Run("is not set", func(t *testing.T) {
e := newEvent()
e.SetMessage([]byte("hello world"))
m := dummyMetadata()
event := createEvent(e, m, time.Local, logp.NewLogger("syslog"))

_, err := event.GetValue("event.sequence")
assert.Error(t, err)
})
}

func dummyMetadata() inputsource.NetworkMetadata {
ip := "127.0.0.1"
parsedIP := net.ParseIP(ip)
Expand Down
Loading