Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[otel] add geoip processor component #6134

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
+1,102 −331
Draft

[otel] add geoip processor component #6134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a50dcd8
[otel] add geoip processor component
rogercoll Nov 25, 2024
d6cedc7
chore: add changelog fragment
rogercoll Nov 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,382 changes: 1,052 additions & 330 deletions NOTICE.txt
View file
Open in desktop

Large diffs are not rendered by default.

32 changes: 32 additions & 0 deletions changelog/fragments/1732809895-add-GeoIP-processor-to-EDOT-collector.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Kind can be one of:
# - breaking-change: a change to previously-documented behavior
# - deprecation: functionality that is being removed in a later release
# - bug-fix: fixes a problem in a previous version
# - enhancement: extends functionality but does not break or fix existing behavior
# - feature: new functionality
# - known-issue: problems that we are aware of in a given version
# - security: impacts on the security of a product or a user’s deployment.
# - upgrade: important information for someone upgrading from a prior version
# - other: does not fit into any of the other categories
kind: feature

# Change summary; a 80ish characters long description of the change.
summary: add GeoIP processor to EDOT collector

# Long description; in case the summary is not enough to describe the change
# this field accommodate a description without length limits.
# NOTE: This field will be rendered only for breaking-change and known-issue kinds at the moment.
#description:

# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
component: "elastic-agent"

# PR URL; optional; the PR number that added the changeset.
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
# Please provide it if you are adding a fragment for a different PR.
#pr: https://github.com/owner/repo/1234

# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
# If not present is automatically filled by the tooling with the issue linked to the PR number.
#issue: https://github.com/owner/repo/1234
3 changes: 3 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/extension/storage/filestorage v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/geoipprocessor v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.113.0
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor v0.113.0
Expand Down Expand Up @@ -438,6 +439,8 @@ require (
github.com/openshift/api v3.9.0+incompatible // indirect
github.com/openshift/client-go v0.0.0-20210521082421-73d9475a9142 // indirect
github.com/openzipkin/zipkin-go v0.4.3 // indirect
github.com/oschwald/geoip2-golang v1.11.0 // indirect
github.com/oschwald/maxminddb-golang v1.13.0 // indirect
github.com/ovh/go-ovh v1.6.0 // indirect
github.com/peterbourgon/diskv v2.0.1+incompatible // indirect
github.com/pierrec/lz4 v2.6.1+incompatible // indirect
Expand Down
12 changes: 12 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1005,6 +1005,10 @@ github.com/mattn/go-sqlite3 v1.14.15/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/maxatome/go-testdeep v1.12.0 h1:Ql7Go8Tg0C1D/uMMX59LAoYK7LffeJQ6X2T04nTH68g=
github.com/maxatome/go-testdeep v1.12.0/go.mod h1:lPZc/HAcJMP92l7yI6TRz1aZN5URwUBUAfUNvrclaNM=
github.com/maxmind/MaxMind-DB v0.0.0-20240605211347-880f6b4b5eb6 h1:q3FNO1NzvowWXqFEam6FwbsgGeTXBL5FOxjMT45x2Ls=
github.com/maxmind/MaxMind-DB v0.0.0-20240605211347-880f6b4b5eb6/go.mod h1:8uyvr1CxDqu0x6NrZZ7CAs6nR0G/wSY8K+Q7uQlNwK4=
github.com/maxmind/mmdbwriter v1.0.0 h1:bieL4P6yaYaHvbtLSwnKtEvScUKKD6jcKaLiTM3WSMw=
github.com/maxmind/mmdbwriter v1.0.0/go.mod h1:noBMCUtyN5PUQ4H8ikkOvGSHhzhLok51fON2hcrpKj8=
github.com/miekg/dns v1.1.26/go.mod h1:bPDLeHnStXmXAq1m/Ch/hvfNHr14JKNPMBo3VZKjuso=
github.com/miekg/dns v1.1.41/go.mod h1:p6aan82bvRIyn+zDIv9xYNUpwa73JcSh9BKwknJysuI=
github.com/miekg/dns v1.1.62 h1:cN8OuEF1/x5Rq6Np+h1epln8OiyPWV+lROx9LxcGgIQ=
Expand Down Expand Up @@ -1167,6 +1171,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributespr
github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor v0.113.0/go.mod h1:U0wBuG6Jz+DBzcPNCmRVZaZTXqaKC+RYo4eJiSKJwwk=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.113.0 h1:+eYxV9vp6u8fKM+9acEJYGUa3SD1vJF776c/haougNQ=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor v0.113.0/go.mod h1:xSVeb2A5wmIuJ9Vak9UwPCP/yN1SDd+pBKfYHROW6YE=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/geoipprocessor v0.113.0 h1:v2rwRhXr6vJMxqiYbeeccWLYluo+uyF5/Nm8N5JR8nM=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/geoipprocessor v0.113.0/go.mod h1:bs85IioUbQ5JbkRcHuFLJY+v8jfZ9otsYy9OO7m/YBY=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.113.0 h1:bloEe7hK+H95zQ5WusQwQCILjudl6ljyR4kO95+Ocuo=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor v0.113.0/go.mod h1:/vYbT5YZ/SyKGtbBtKCI00sGUk3Xd90A2hT5iSWP8Dk=
github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.113.0 h1:5cEQNFwYAn8PJ66l88lGLMSz9TYWiIUFayDwAtpJumw=
Expand Down Expand Up @@ -1205,6 +1211,10 @@ github.com/openshift/client-go v0.0.0-20210521082421-73d9475a9142 h1:ZHRIMCFIJN1
github.com/openshift/client-go v0.0.0-20210521082421-73d9475a9142/go.mod h1:fjS8r9mqDVsPb5td3NehsNOAWa4uiFkYEfVZioQ2gH0=
github.com/openzipkin/zipkin-go v0.4.3 h1:9EGwpqkgnwdEIJ+Od7QVSEIH+ocmm5nPat0G7sjsSdg=
github.com/openzipkin/zipkin-go v0.4.3/go.mod h1:M9wCJZFWCo2RiY+o1eBCEMe0Dp2S5LDHcMZmk3RmK7c=
github.com/oschwald/geoip2-golang v1.11.0 h1:hNENhCn1Uyzhf9PTmquXENiWS6AlxAEnBII6r8krA3w=
github.com/oschwald/geoip2-golang v1.11.0/go.mod h1:P9zG+54KPEFOliZ29i7SeYZ/GM6tfEL+rgSn03hYuUo=
github.com/oschwald/maxminddb-golang v1.13.0 h1:R8xBorY71s84yO06NgTmQvqvTvlS/bnYZrrWX1MElnU=
github.com/oschwald/maxminddb-golang v1.13.0/go.mod h1:BU0z8BfFVhi1LQaonTwwGQlsHUEu9pWNdMfmq4ztm0o=
github.com/otiai10/copy v1.14.0 h1:dCI/t1iTdYGtkvCuBG2BgR6KZa83PTclw4U5n2wAllU=
github.com/otiai10/copy v1.14.0/go.mod h1:ECfuL02W+/FkTWZWgQqXPWZgW9oeKCSQ5qVfSc4qc4w=
github.com/otiai10/mint v1.5.1 h1:XaPLeE+9vGbuyEHem1JNk3bYc7KKqyI/na0/mLd/Kks=
Expand Down Expand Up @@ -1681,6 +1691,8 @@ go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN8
go.uber.org/zap v1.24.0/go.mod h1:2kMP+WWQ8aoFoedH3T2sq6iJ2yDWpHbP0f6MQbS9Gkg=
go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8=
go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go4.org/netipx v0.0.0-20230824141953-6213f710f925 h1:eeQDDVKFkx0g4Hyy8pHgmZaK0EqB4SD6rvKbUdN3ziQ=
go4.org/netipx v0.0.0-20230824141953-6213f710f925/go.mod h1:PLyyIXexvUFg3Owu6p/WfdlivPbZJsZdgWZlrGope/Y=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
Expand Down
1 change: 1 addition & 0 deletions internal/pkg/otel/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ This section provides a summary of components included in the Elastic Distributi
| [memorylimiterprocessor](https://github.com/open-telemetry/opentelemetry-collector/blob/processor/memorylimiterprocessor/v0.113.0/processor/memorylimiterprocessor/README.md) | v0.113.0 |
| [attributesprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/attributesprocessor/v0.113.0/processor/attributesprocessor/README.md) | v0.113.0 |
| [filterprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/filterprocessor/v0.113.0/processor/filterprocessor/README.md) | v0.113.0 |
| [geoipprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/geoipprocessor/v0.113.0/processor/geoipprocessor/README.md) | v0.113.0 |
| [k8sattributesprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/k8sattributesprocessor/v0.113.0/processor/k8sattributesprocessor/README.md) | v0.113.0 |
| [resourcedetectionprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/resourcedetectionprocessor/v0.113.0/processor/resourcedetectionprocessor/README.md) | v0.113.0 |
| [resourceprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/resourceprocessor/v0.113.0/processor/resourceprocessor/README.md) | v0.113.0 |
Expand Down
3 changes: 2 additions & 1 deletion internal/pkg/otel/components.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import (
// Processors:
attributesprocessor "github.com/open-telemetry/opentelemetry-collector-contrib/processor/attributesprocessor" // for modifying signal attributes
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/filterprocessor"
geoipprocessor "github.com/open-telemetry/opentelemetry-collector-contrib/processor/geoipprocessor" // for adding geographical metadata associated to an IP address
k8sattributesprocessor "github.com/open-telemetry/opentelemetry-collector-contrib/processor/k8sattributesprocessor" // for adding k8s metadata
"github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor"
resourceprocessor "github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourceprocessor" // for modifying resource attributes
Expand Down Expand Up @@ -62,7 +63,6 @@ import (

func components(extensionFactories ...extension.Factory) func() (otelcol.Factories, error) {
return func() (otelcol.Factories, error) {

var err error
factories := otelcol.Factories{}

Expand Down Expand Up @@ -91,6 +91,7 @@ func components(extensionFactories ...extension.Factory) func() (otelcol.Factori
attributesprocessor.NewFactory(),
transformprocessor.NewFactory(),
filterprocessor.NewFactory(),
geoipprocessor.NewFactory(),
k8sattributesprocessor.NewFactory(),
elasticinframetricsprocessor.NewFactory(),
resourcedetectionprocessor.NewFactory(),
Expand Down