Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add syslog receiver support #2482

Merged
merged 28 commits into from
Mar 16, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
e13b62b
add syslog receiver support
wph95 Feb 28, 2021
d9e2106
fix test data race
wph95 Mar 1, 2021
b9ca227
fix lint
wph95 Mar 1, 2021
90703ce
fix ci
wph95 Mar 1, 2021
62f2059
fix lint ci run "make generate"
wph95 Mar 1, 2021
cf15de8
fix license header
wph95 Mar 2, 2021
4b0ecb2
- upgrade log-collector to 0.15.1
wph95 Mar 2, 2021
b33c5f2
Merge branch 'main' into add-syslog-receiver
wph95 Mar 2, 2021
6eeff49
remove unused err handle
wph95 Mar 3, 2021
9fe945d
update log collection to 0.15.2@fb7443
wph95 Mar 3, 2021
f466443
update README
wph95 Mar 3, 2021
b4ef12d
Merge branch 'main' into add-syslog-receiver
wph95 Mar 3, 2021
fea3081
fix spell error
wph95 Mar 3, 2021
7034f32
make generate
wph95 Mar 8, 2021
8db0f32
add a unittest to cover 100%
wph95 Mar 8, 2021
8d6c2e7
Merge branch 'main' into add-syslog-receiver
wph95 Mar 8, 2021
14e3d49
update go sum
wph95 Mar 8, 2021
fdb9ced
fix merge code error
wph95 Mar 8, 2021
ff10a9b
Merge branch 'main' into add-syslog-receiver
wph95 Mar 9, 2021
8223468
update readme and update go mod
wph95 Mar 9, 2021
063970c
clean up code, remove unused code and Dependencies
wph95 Mar 10, 2021
a83c7cc
Merge branch 'main' into add-syslog-receiver
wph95 Mar 10, 2021
97640a3
update syslog receiver mod
wph95 Mar 10, 2021
101a5dd
upgrade opentelemetry-log-collection to support standard TLS config
wph95 Mar 16, 2021
fcff6bd
restart ci
wph95 Mar 16, 2021
abac718
Merge branch 'main' into add-syslog-receiver
djaglowski Mar 16, 2021
1868e5c
fix merge failed
wph95 Mar 16, 2021
dfa5708
use NopFactories() instead of ExampleComponents()
wph95 Mar 16, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/collectdreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/dockerstatsreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver v0.0.0-00010101000000-000000000000
Expand Down Expand Up @@ -180,6 +181,8 @@ replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/zooke

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/filelogreceiver => ./receiver/filelogreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver => ./receiver/syslogreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/receiver/memcachedreceiver => ./receiver/memcachedreceiver

replace github.com/open-telemetry/opentelemetry-collector-contrib/processor/groupbyattrsprocessor => ./processor/groupbyattrsprocessor
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1110,6 +1110,8 @@ github.com/onsi/gomega v1.10.4/go.mod h1:g/HbgYopi++010VEqkFgJHKC09uJiW9UkXvMUuK
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0-rc1/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
Expand All @@ -1133,6 +1135,7 @@ github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnh
github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/openzipkin/zipkin-go v0.2.5 h1:UwtQQx2pyPIgWYHRg+epgdx1/HnBQTgN3/oIYEJTQzU=
github.com/openzipkin/zipkin-go v0.2.5/go.mod h1:KpXfKdgRDnnhsxw4pNIH9Md5lyFqKUa4YDFlwRYAMyE=
github.com/orijtech/prometheus-go-metrics-exporter v0.0.6/go.mod h1:BiTx/ugZex8LheBk3j53tktWaRdFjV5FCfT2o0P7msE=
github.com/ory/go-acc v0.2.6/go.mod h1:4Kb/UnPcT8qRAk3IAxta+hvVapdxTLWtrr7bFLlEgpw=
github.com/ory/viper v1.7.5/go.mod h1:ypOuyJmEUb3oENywQZRgeAMwqgOyDqwboO1tj3DjTaM=
github.com/pact-foundation/pact-go v1.0.4/go.mod h1:uExwJY4kCzNPcHRj+hCR/HBbOOIwwtUjcrb0b5/5kLM=
Expand Down Expand Up @@ -1189,6 +1192,7 @@ github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDf
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.4.1/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.5.1/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU=
github.com/prometheus/client_golang v1.6.0/go.mod h1:ZLOG9ck3JLRdB5MgO8f+lLTe83AXG6ro35rLTxvnIl4=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
Expand Down
3 changes: 1 addition & 2 deletions internal/stanza/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@ module github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza
go 1.14

require (
github.com/open-telemetry/opentelemetry-log-collection v0.16.0
github.com/pelletier/go-toml v1.8.0 // indirect
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
Expand Down
2 changes: 2 additions & 0 deletions internal/stanza/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -692,6 +692,8 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opentracing-contrib/go-grpc v0.0.0-20191001143057-db30781987df/go.mod h1:DYR5Eij8rJl8h7gblRrOZ8g0kW1umSpKqYIBTgeDtLo=
Expand Down
2 changes: 1 addition & 1 deletion receiver/filelogreceiver/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.14
require (
github.com/observiq/nanojack v0.0.0-20201106172433-343928847ebc
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.16.0
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
Expand Down
2 changes: 2 additions & 0 deletions receiver/filelogreceiver/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -799,6 +799,8 @@ github.com/onsi/gomega v1.9.0/go.mod h1:Ho0h+IUsWyvy1OpqCwxlQ/21gkhVunqlU8fDGcoT
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0 h1:5DCDDrx95Hg4GPl2trng/CtYoGD0W0fjKQ3v6NQWmL4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.0/go.mod h1:J1tOFFS2olDMZyiU1mtnCLnAv+uElKeYDbZ2ZAgjMU4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31 h1:FruNezloXnyrkbsQhprOfk0i1M5ftEOZ3sG0+lpRJP4=
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31/go.mod h1:9E7qy5/82Mr4FysOjfbxcHxk2+AXyOukDhdmezEjNdo=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVojFA6h/TRcI=
Expand Down
1 change: 1 addition & 0 deletions receiver/syslogreceiver/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
69 changes: 69 additions & 0 deletions receiver/syslogreceiver/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Syslog Receiver

Parses Syslogs from tcp/udp using
the [opentelemetry-log-collection](https://github.com/open-telemetry/opentelemetry-log-collection) library.

Supported pipeline types: logs

> :construction: This receiver is in alpha and configuration fields are subject to change.

## Configuration

| Field | Default | Description |
| ---------- | ---------------- | ------------------------------------------------------------ |
| `id` | `syslog_input` | A unique identifier for the operator |
| `udp` |`nil` | Defined udp_input operator. (see the UDP configuration section) |
| `tcp` | `nil` | Defined tcp_input operator. (see the TCP configuration section) |
| `protocol` | required | The protocol to parse the syslog messages as. Options are `rfc3164` and `rfc5424` |
| `location` | `UTC` | The geographic location (timezone) to use when parsing the timestamp (Syslog RFC 3164 only). The available locations depend on the local IANA Time Zone database. [This page](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) contains many examples, such as `America/New_York`. |
| `timestamp` | `nil` | An optional [timestamp](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/types/timestamp.md) block which will parse a timestamp field before passing the entry to the output operator |
| `severity` | `nil` | An optional [severity](https://github.com/open-telemetry/opentelemetry-log-collection/blob/main/docs/docs/types/severity.md) block which will parse a severity field before passing the entry to the output operator
| `attributes` | {} | A map of `key: value` labels to add to the entry's attributes |
| `resource` | {} | A map of `key: value` labels to add to the entry's resource |

### UDP Configuration

| Field | Default | Description |
| --- | --- | --- |
| `listen_address` | required | A listen address of the form `<ip>:<port>` |

### TCP Configuration

| Field | Default | Description |
| --- | --- | --- |
| `max_buffer_size` | `1024kib` | Maximum size of buffer that may be allocated while reading TCP input |
| `listen_address` | required | A listen address of the form `<ip>:<port>` |
| `tls` | | An optional `TLS` configuration (see the TLS configuration section) |

#### TLS Configuration

wph95 marked this conversation as resolved.
Show resolved Hide resolved
The `tcp_input` operator supports TLS, disabled by default.

| Field | Default | Description |
| --- | --- | --- |
| `cert_file` | | Path to the TLS cert to use for TLS required connections. |
| `key_file` | | Path to the TLS key to use for TLS required connections.|
| `ca_file` | | Path to the CA cert. For a client this verifies the server certificate. For a server this verifies client certificates. If empty uses system root CA. |
| `client_ca_file` | | (optional) Path to the TLS cert to use by the server to verify a client certificate. This sets the ClientCAs and ClientAuth to RequireAndVerifyClientCert in the TLSConfig. Please refer to godoc.org/crypto/tls#Config for more information.
|

## Example Configurations

TCP Configuration:

```yaml
- type: syslog_input
protocol: rfc5424
tcp:
listen_address: "0.0.0.0:54526"
```

UDP Configuration:

```yaml
- type: syslog_input
udp:
listen_address: "0.0.0.0:54526"
protocol: rfc3164
location: UTC
```
15 changes: 15 additions & 0 deletions receiver/syslogreceiver/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
// Copyright 2021 OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package syslogreceiver
14 changes: 14 additions & 0 deletions receiver/syslogreceiver/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
module github.com/open-telemetry/opentelemetry-collector-contrib/receiver/syslogreceiver

go 1.14

require (
github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza v0.0.0-00010101000000-000000000000
github.com/open-telemetry/opentelemetry-log-collection v0.16.1-0.20210315170618-aa8d82509c31
github.com/stretchr/testify v1.7.0
go.opentelemetry.io/collector v0.22.1-0.20210313012550-03904de3dd61
go.uber.org/zap v1.16.0
gopkg.in/yaml.v2 v2.4.0
)

replace github.com/open-telemetry/opentelemetry-collector-contrib/internal/stanza => ../../internal/stanza
Loading