Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x-pack/filebeat/input/etw: New input #36915

Merged
merged 33 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c76d494
First version of ETW input
chemamartinez Oct 19, 2023
0f46bd9
Minor fixes for ETW input
chemamartinez Oct 26, 2023
d363515
More fixes and requested changes for ETW input
chemamartinez Nov 20, 2023
619492c
Include ETW in the default input list for Windows
chemamartinez Nov 28, 2023
2fdddf0
Tests for config input
chemamartinez Nov 28, 2023
df644f4
Sync input close calls
chemamartinez Nov 29, 2023
7ba80fd
Update config file and docs
chemamartinez Nov 29, 2023
e67262e
Fix some tabs in reference file
chemamartinez Nov 30, 2023
18fa380
Add metadata to ETW events
chemamartinez Nov 30, 2023
78e37c2
Fix PR checks (docs and tests)
chemamartinez Dec 4, 2023
a860810
Fix lint error in input
chemamartinez Dec 4, 2023
87779ac
Improve docs with supported providers and platforms
chemamartinez Dec 20, 2023
e73c27e
Fix requested changes for ETW input
chemamartinez Feb 5, 2024
aea3059
Add ETW input to changelog
chemamartinez Feb 6, 2024
6d518d4
Rename GetHandler to AttachToExistingSession in ETW input
chemamartinez Feb 6, 2024
81ce171
Fix NewSession unit test
chemamartinez Feb 7, 2024
4b52f54
Add tests for input helpers
chemamartinez Feb 7, 2024
1a8b635
Fix linting error in input_test.go
chemamartinez Feb 7, 2024
7557179
Fix some unit tests
chemamartinez Feb 7, 2024
3692a2a
Unit tests for ETW input
chemamartinez Feb 8, 2024
a2c138f
Fix CloseSession call in tests
chemamartinez Feb 8, 2024
e4df38f
Fix building of event and some refactors
chemamartinez Feb 8, 2024
0fe3b02
Add field mapping to ETW input
chemamartinez Feb 8, 2024
29d2896
Added files after make update
chemamartinez Feb 8, 2024
027e5a4
Export fields mapping to docs
chemamartinez Feb 8, 2024
83d60c6
Fix timestamp and GUID for buildEvent tests
chemamartinez Feb 8, 2024
c0349cc
Adjust ETW mapping to fit ECS
chemamartinez Feb 10, 2024
2aa117e
Update fields built files
chemamartinez Feb 10, 2024
c8df9cb
Address review comments
andrewkroh Feb 12, 2024
e036719
Merge remote-tracking branch 'elastic/main' into dev-etw-filebeat-input
andrewkroh Feb 12, 2024
4940400
filebeat/docs - rebuild with field changes
andrewkroh Feb 12, 2024
c04f88a
Merge branch 'main' into dev-etw-filebeat-input
narph Feb 13, 2024
30b3f3a
fix tests
narph Feb 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,7 @@ CHANGELOG*
/x-pack/filebeat/input/cel/ @elastic/security-service-integrations
/x-pack/filebeat/input/cometd/ @elastic/obs-infraobs-integrations
/x-pack/filebeat/input/entityanalytics/ @elastic/security-service-integrations
/x-pack/filebeat/input/etw/ @elastic/sec-windows-platform
/x-pack/filebeat/input/gcppubsub/ @elastic/security-service-integrations
/x-pack/filebeat/input/gcs/ @elastic/security-service-integrations
/x-pack/filebeat/input/http_endpoint/ @elastic/security-service-integrations
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,7 @@ Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will d
- Add support for PEM-based Okta auth in HTTPJSON. {pull}37772[37772]
- Prevent complete loss of long request trace data. {issue}37826[37826] {pull}37836[37836]
- Add support for PEM-based Okta auth in CEL. {pull}37813[37813]
- Add ETW input. {pull}36915[36915]

*Auditbeat*

Expand Down
195 changes: 195 additions & 0 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ grouped in the following categories:
* <<exported-fields-threatintel>>
* <<exported-fields-tomcat>>
* <<exported-fields-traefik>>
* <<exported-fields-winlog>>
* <<exported-fields-zeek>>
* <<exported-fields-zookeeper>>
* <<exported-fields-zoom>>
Expand Down Expand Up @@ -158960,6 +158961,200 @@ alias to: source.geo.region_iso_code

--

[[exported-fields-winlog]]
== Windows ETW fields

Fields from the ETW input (Event Tracing for Windows).



[float]
=== winlog

All fields specific to the Windows Event Tracing are defined here.



*`winlog.activity_id`*::
+
--
A globally unique identifier that identifies the current activity. The events that are published with this identifier are part of the same activity.


type: keyword

required: False

--

*`winlog.channel`*::
+
--
Used to enable special event processing. Channel values below 16 are reserved for use by Microsoft to enable special treatment by the ETW runtime. Channel values 16 and above will be ignored by the ETW runtime (treated the same as channel 0) and can be given user-defined semantics.


type: keyword

required: False

--

*`winlog.event_data`*::
+
--
The event-specific data. The content of this object is specific to any provider and event.


type: object

required: False

--

*`winlog.flags`*::
+
--
Flags that provide information about the event such as the type of session it was logged to and if the event contains extended data.


type: keyword

required: False

--

*`winlog.keywords`*::
+
--
The keywords are used to indicate an event's membership in a set of event categories.


type: keyword

required: False

--

*`winlog.level`*::
+
--
Level of severity. Level values 0 through 5 are defined by Microsoft. Level values 6 through 15 are reserved. Level values 16 through 255 can be defined by the event provider.


type: keyword

required: False

--

*`winlog.opcode`*::
+
--
The opcode defined in the event. Task and opcode are typically used to identify the location in the application from where the event was logged.


type: keyword

required: False

--

*`winlog.process_id`*::
+
--
Identifies the process that generated the event.


type: keyword

required: False

--

*`winlog.provider_guid`*::
+
--
A globally unique identifier that identifies the provider that logged the event.


type: keyword

required: False

--

*`winlog.provider_name`*::
+
--
The source of the event log record (the application or service that logged the record).


type: keyword

required: False

--

*`winlog.session`*::
+
--
Configured session to forward ETW events from providers to consumers.


type: keyword

required: False

--

*`winlog.severity`*::
+
--
Human-readable level of severity.


type: keyword

required: False

--

*`winlog.task`*::
+
--
The task defined in the event. Task and opcode are typically used to identify the location in the application from where the event was logged.


type: keyword

required: False

--

*`winlog.thread_id`*::
+
--
Identifies the thread that generated the event.


type: keyword

required: False

--

*`winlog.version`*::
+
--
Specify the version of a manifest-based event.


type: keyword

required: False

--

[[exported-fields-zeek]]
== Zeek fields

Expand Down
9 changes: 6 additions & 3 deletions filebeat/docs/filebeat-options.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,10 @@ You can configure {beatname_uc} to use the following inputs:
* <<{beatname_lc}-input-cometd>>
* <<{beatname_lc}-input-container>>
* <<{beatname_lc}-input-entity-analytics>>
* <<{beatname_lc}-input-etw>>
* <<{beatname_lc}-input-filestream>>
* <<{beatname_lc}-input-gcp-pubsub>>
* <<{beatname_lc}-input-gcs>>
* <<{beatname_lc}-input-http_endpoint>>
* <<{beatname_lc}-input-httpjson>>
* <<{beatname_lc}-input-journald>>
Expand All @@ -90,7 +92,6 @@ You can configure {beatname_uc} to use the following inputs:
* <<{beatname_lc}-input-syslog>>
* <<{beatname_lc}-input-tcp>>
* <<{beatname_lc}-input-udp>>
* <<{beatname_lc}-input-gcs>>

include::multiline.asciidoc[]

Expand All @@ -112,10 +113,14 @@ include::inputs/input-container.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-entity-analytics.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-etw.asciidoc[]

include::inputs/input-filestream.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-gcp-pubsub.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-gcs.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-http-endpoint.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-httpjson.asciidoc[]
Expand Down Expand Up @@ -143,5 +148,3 @@ include::inputs/input-tcp.asciidoc[]
include::inputs/input-udp.asciidoc[]

include::inputs/input-unix.asciidoc[]

include::../../x-pack/filebeat/docs/inputs/input-gcs.asciidoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -180,3 +180,44 @@
# This is used to shift collection start time and end time back in order to
# collect logs when there is a delay in CloudWatch.
#latency: 1m

#------------------------------ ETW input --------------------------------
# Beta: Config options for ETW (Event Trace for Windows) input (Only available for Windows)
#- type: etw
#enabled: false
#id: etw-dnsserver

# Path to an .etl file to read from.
#file: "C:\Windows\System32\Winevt\Logs\Logfile.etl"

# GUID of an ETW provider.
# Run 'logman query providers' to list the available providers.
#provider.guid: {EB79061A-A566-4698-9119-3ED2807060E7}

# Name of an ETW provider.
# Run 'logman query providers' to list the available providers.
#provider.name: Microsoft-Windows-DNSServer

# Tag to identify created sessions.
# If missing, its default value is the provider ID prefixed by 'Elastic-'.
#session_name: DNSServer-Analytical-Trace

# Filter collected events with a level value that is less than or equal to this level.
# Allowed values are critical, error, warning, informational, and verbose.
#trace_level: verbose

# 8-byte bitmask that enables the filtering of events from specific provider subcomponents.
# The provider will write a particular event if the event's keyword bits match any of the bits
# in this bitmask.
# Run 'logman query providers "<provider.name>"' to list available keywords.
#match_any_keyword: 0x8000000000000000

# 8-byte bitmask that enables the filtering of events from
# specific provider subcomponents. The provider will write a particular
# event if the event's keyword bits match all of the bits in this bitmask.
# Run 'logman query providers "<provider.name>"' to list available keywords.
#match_all_keyword: 0

# An existing session to read from.
# Run 'logman query -ets' to list existing sessions.
#session: UAL_Usermode_Provider
Loading
Loading