Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add syslog receiver support #2482

Merged
merged 28 commits into from
Mar 16, 2021
Merged

add syslog receiver support #2482

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
e13b62b
add syslog receiver support
wph95 Feb 28, 2021
d9e2106
fix test data race
wph95 Mar 1, 2021
b9ca227
fix lint
wph95 Mar 1, 2021
90703ce
fix ci
wph95 Mar 1, 2021
62f2059
fix lint ci run "make generate"
wph95 Mar 1, 2021
cf15de8
fix license header
wph95 Mar 2, 2021
4b0ecb2
- upgrade log-collector to 0.15.1
wph95 Mar 2, 2021
b33c5f2
Merge branch 'main' into add-syslog-receiver
wph95 Mar 2, 2021
6eeff49
remove unused err handle
wph95 Mar 3, 2021
9fe945d
update log collection to 0.15.2@fb7443
wph95 Mar 3, 2021
f466443
update README
wph95 Mar 3, 2021
b4ef12d
Merge branch 'main' into add-syslog-receiver
wph95 Mar 3, 2021
fea3081
fix spell error
wph95 Mar 3, 2021
7034f32
make generate
wph95 Mar 8, 2021
8db0f32
add a unittest to cover 100%
wph95 Mar 8, 2021
8d6c2e7
Merge branch 'main' into add-syslog-receiver
wph95 Mar 8, 2021
14e3d49
update go sum
wph95 Mar 8, 2021
fdb9ced
fix merge code error
wph95 Mar 8, 2021
ff10a9b
Merge branch 'main' into add-syslog-receiver
wph95 Mar 9, 2021
8223468
update readme and update go mod
wph95 Mar 9, 2021
063970c
clean up code, remove unused code and Dependencies
wph95 Mar 10, 2021
a83c7cc
Merge branch 'main' into add-syslog-receiver
wph95 Mar 10, 2021
97640a3
update syslog receiver mod
wph95 Mar 10, 2021
101a5dd
upgrade opentelemetry-log-collection to support standard TLS config
wph95 Mar 16, 2021
fcff6bd
restart ci
wph95 Mar 16, 2021
abac718
Merge branch 'main' into add-syslog-receiver
djaglowski Mar 16, 2021
1868e5c
fix merge failed
wph95 Mar 16, 2021
dfa5708
use NopFactories() instead of ExampleComponents()
wph95 Mar 16, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cmd/otelcontribcol/unstable_components_enabled.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,12 @@ import (
"go.opentelemetry.io/collector/component"

"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver"
"github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver"
)

func extraReceivers() []component.ReceiverFactory {
return []component.ReceiverFactory{
filelogreceiver.NewFactory(),
syslogreceiver.NewFactory(),
}
}
4 changes: 4 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.0.0-00010101000000-000000000000
Expand Down Expand Up @@ -176,6 +177,9 @@ replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zooke

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver => ./receiver/filelogreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver => ./receiver/syslogreceiver


replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/memcachedreceiver => ./receiver/memcachedreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbytraceprocessor => ./processor/groupbytraceprocessor
Expand Down
1 change: 1 addition & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1028,6 +1028,7 @@ github.com/onsi/gomega v1.10.2/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1y
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0 h1:hnf4rn2NxvlXKdTCXeq7zV0Jdx2mIlX2aUMTViq6+ks=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/open-telemetry/opentelemetry-log-collection v0.15.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
Expand Down
2 changes: 1 addition & 1 deletion internal/stanza/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza
go 1.14

require (
github.com/open-telemetry/opentelemetry-log-collection v0.14.0
github.com/open-telemetry/opentelemetry-log-collection v0.15.0
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.21.1-0.20210225192722-e6319ac4c6fc
go.uber.org/zap v1.16.0
Expand Down
4 changes: 2 additions & 2 deletions internal/stanza/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -650,8 +650,8 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
github.com/onsi/gomega v1.10.1/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/onsi/gomega v1.10.2/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1ybHNo=
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0 h1:hnf4rn2NxvlXKdTCXeq7zV0Jdx2mIlX2aUMTViq6+ks=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/open-telemetry/opentelemetry-log-collection v0.15.0 h1:umwOBwFSNcrKuHT/RHxkLPJmDgpkCi2JhIpJw2qCXxU=
github.com/open-telemetry/opentelemetry-log-collection v0.15.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opentracing-contrib/go-grpc v0.0.0-20191001143057-db30781987df/go.mod h1:DYR5Eij8rJl8h7gblRrOZ8g0kW1umSpKqYIBTgeDtLo=
Expand Down
2 changes: 1 addition & 1 deletion receiver/filelogreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.14
require (
github.com/observiq/nanojack v0.0.0-20201106172433-343928847ebc
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.14.0
github.com/open-telemetry/opentelemetry-log-collection v0.15.0
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.21.1-0.20210225192722-e6319ac4c6fc
go.uber.org/zap v1.16.0
Expand Down
2 changes: 2 additions & 0 deletions receiver/filelogreceiver/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -757,6 +757,8 @@ github.com/onsi/gomega v1.10.2/go.mod h1:iN09h71vgCQne3DLsj+A5owkum+a2tYe+TOCB1y
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0 h1:hnf4rn2NxvlXKdTCXeq7zV0Jdx2mIlX2aUMTViq6+ks=
github.com/open-telemetry/opentelemetry-log-collection v0.14.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/open-telemetry/opentelemetry-log-collection v0.15.0 h1:umwOBwFSNcrKuHT/RHxkLPJmDgpkCi2JhIpJw2qCXxU=
github.com/open-telemetry/opentelemetry-log-collection v0.15.0/go.mod h1:DwA3MWfRL0iRdhnjn+c/Ihhr+6Fu11K+8C7NYQmEL/w=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
Expand Down
1 change: 1 addition & 0 deletions receiver/syslogreceiver/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
32 changes: 32 additions & 0 deletions receiver/syslogreceiver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Syslog Receiver

Parses Syslogs from tcp/udp using the [opentelemetry-log-collection](https://github.com/open-telemetry/opentelemetry-log-collection) library.

Supported pipeline types: logs

> :construction: This receiver is in alpha and configuration fields are subject to change.

## Required Parameters

- `operators` is an array of [operators](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/operators/README.md#what-operators-are-available). Each operator performs a simple responsibility, such as parsing a timestamp or JSON. Chain together operators to process logs into a desired format.
wph95 marked this conversation as resolved.
Show resolved Hide resolved

## Optional Parameters

- `plugin_dir` is the path to a directory which contains `stanza` [plugins](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/plugins.md#using-plugins). Plugins are parameterized pipelines that are designed for specific use cases.
- `offsets_file` is the path to a file that `stanza` will use to remember where it left off when reading from files or other persistent input sources. If specified, `stanza` will create and manage this file.

## Operator Basics

- Every operator has a `type`.
- Every operator can be given a unique `id`. If you use the same type of operator more than once in a pipeline, you must specify an `id`. Otherwise, the `id` defaults to the value of `type`.
- Operators will output to the next operator in the pipeline. The last operator in the pipeline will emit from the receiver. Optionally, the `output` parameter can be used to specify the `id` of another operator to which logs will be passed directly.

Receiver Configuration
```yaml
receivers:
syslog:
tcp:
listen_address: "0.0.0.0:54526"
syslog:
wph95 marked this conversation as resolved.
Show resolved Hide resolved
protocol: rfc5424
```
15 changes: 15 additions & 0 deletions receiver/syslogreceiver/doc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Copyright -c Google LLC
wph95 marked this conversation as resolved.
Show resolved Hide resolved
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package syslogreceiver
14 changes: 14 additions & 0 deletions receiver/syslogreceiver/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver

go 1.14

require (
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.15.0
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.21.1-0.20210225192722-e6319ac4c6fc
go.uber.org/zap v1.16.0
gopkg.in/yaml.v2 v2.4.0
)

replace github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza => ../../internal/stanza
Loading