Affecting all Beats
-
The document id fields has been renamed from @metadata.id to @metadata._id 15859
-
Update to Golang 1.12.1. 11330
-
Disable Alibaba Cloud and Tencent Cloud metadata providers by default. 12812
-
Libbeat: Do not overwrite agent.*, ecs.version, and host.name. 14407
-
Libbeat: Cleanup the x-pack licenser code to use the new license endpoint and the new format. 15091
-
Refactor metadata generator to support adding metadata across resources 14875
-
Remove
AddDockerMetadata
andAddKubernetesMetadata
processors from thescript
processor. They can still be used as normal processors in the configuration. 16349 16514 -
Introduce APM libbeat instrumentation, active when running the beat with ELASTIC_APM_ACTIVE=true. 17938
-
Remove the non-ECS
agent.hostname
field. Use theagent.name
oragent.id
fields for an identifier. 16377 18328 -
Make error message about locked data path actionable. 18667
-
Ensure dynamic template names are unique for the same field. 18849
-
Remove the deprecated
xpack.monitoring.
settings. Going forward onlymonitoring.
settings may be used. 9424 18608 -
Added
certificate
TLS verification mode to ignore server name mismatch. 12283 20293 -
Autodiscover doesn’t generate any configuration when a variable is missing. Previously it generated an incomplete configuration. 20898
-
Remove redundant
cloudfoundry.*.timestamp
fields. This value is set in@timestamp
. 21175 -
Allow embedding of CAs, Certificate of private keys for anything that support TLS in ouputs and inputs. 21179
-
Update to Golang 1.12.1. 11330
-
Disable Alibaba Cloud and Tencent Cloud metadata providers by default. 12812
-
API address is a required setting in
add_cloudfoundry_metadata
. 21759 -
Update to ECS 1.7.0. 22571
-
Add support for SCRAM-SHA-512 and SCRAM-SHA-256 in Kafka output. 12867
-
Use alias to report container image in k8s metadata. 24380
-
Set
cleanup_timeout
to zero by default in docker and kubernetes autodiscover in all beats except Filebeat where it is kept to 60 seconds. 24681 -
Update to ECS 1.9.0. 24909
-
Remove id_field_data 25239
Auditbeat
-
File integrity dataset (macOS): Replace unnecessary
file.origin.raw
(type keyword) withfile.origin.text
(typetext
). 12423 15630 -
Change event.kind=error to event.kind=event to comply with ECS. 18870 20685
-
Change network.direction values to ECS recommended values (inbound, outbound). 12445 20695
-
Docker container needs to be explicitly run as user root for auditing. 21202
-
File integrity dataset no longer includes the leading dot in
file.extension
values (e.g. it will report "png" instead of ".png") to comply with ECS. 21644 -
Use ECS 1.7 ingress/egress network directions instead of inbound/outbound. 22991
-
Use ingress/egress instead of inbound/outbound for ECS 1.7 in auditd module. 23000
Auditbeat
Filebeat
-
Fix parsing of Elasticsearch node name by
elasticsearch/slowlog
fileset. 14547 -
Improve ECS field mappings in panw module. event.outcome now only contains success/failure per ECS specification. 16025 17910
-
Improve ECS categorization field mappings for nginx module. http.request.referrer only populated when nginx sets a value 16174 17844
-
Improve ECS field mappings in santa module. move hash.sha256 to process.hash.sha256 & move certificate fields to santa.certificate . 16180 17982
-
With the default configuration the cloud modules (aws, azure, googlecloud, o365, okta) will no longer send the
host
field that contains information about the host Filebeat is running on. This is because thehost
field specifies the host on which the event happened. 13920 18223 -
With the default configuration the following modules will no longer send the
host
field that contains information about the host on which Filebeat is running. You can revert this change by configuring tags for the module and omittingforwarded
from the list. 13920-
CEF 18223
-
Cisco 18753
-
CrowdStrike 19132
-
Fortinet 19133
-
iptables 18756
-
Checkpoint 18754
-
Netflow 19087
-
Zeek 19113 (
forwarded
tag is not included by default) -
Suricata 19107 (
forwarded
tag is not included by default) -
CoreDNS 19134 (
forwarded
tag is not included by default) -
Envoy Proxy 19134 (
forwarded
tag is not included by default)
-
-
Preserve case of http.request.method. ECS prior to 1.6 specified normalizing to lowercase, which lost information. Affects filesets: apache/access, elasticsearch/audit, iis/access, iis/error, nginx/access, nginx/ingress_controller, aws/elb, suricata/eve, zeek/http. 18154 18359
-
Adds check on
<no value>
config option value for the azure inputresource_manager_endpoint
. 18890 -
Okta module now requires objects instead of JSON strings for the
http_headers
,http_request_body
,pagination
,rate_limit
, andssl
variables. 18953 -
With the default configuration the cloud modules (aws, azure, googlecloud, o365, okta)
-
With the default configuration the cef and panw modules will no longer send the
host
forwarded
from the list. 13920 18223 -
Preserve case of http.request.method. ECS prior to 1.6 specified normalizing to lowercase, which lost information. Affects filesets: apache/access, elasticsearch/audit, iis/access, iis/error, nginx/access, nginx/ingress_controller, aws/elb, suricata/eve, zeek/http. 18154 18359
-
Adds
split_events_by
option to httpjson input. 19246 -
Adds
date_cursor
option to httpjson input. 19483 -
Adds Gsuite module with SAML support. 19329
-
Adds Gsuite User Accounts support. 19329
-
Adds Gsuite Login audit support. 19702
-
Adds Gsuite Admin support. 19769
-
Adds Gsuite Drive support. 19704
-
Adds Gsuite Groups support. 19725
-
Move file metrics to dataset endpoint 19977
-
Add
while_pattern
type to multiline reader. 19662 -
Tracking session end reason in panw module. 18705
-
Fix PANW field spelling "veredict" to "verdict" on event.action 18808
-
Removed experimental modules
citrix
,kaspersky
,rapid7
andtenable
. 20706 -
Add support for GMT timezone offsets in
decode_cef
. 20993 -
Fix parsing of Elasticsearch node name by
elasticsearch/slowlog
fileset. 14547 -
API address and shard ID are required settings in the Cloud Foundry input. 21759
-
Rename bad ECS field name tracing.trace.id to trace.id in aws elb fileset. 22571
-
Fix parsing issues with nested JSON payloads in Elasticsearch audit log fileset. 22975
-
Rename
network.direction
values in crowdstrike/falcon toingress
/egress
. 23041 -
Add User Agent Parser for Azure Sign In Logs Ingest Pipeline 23201
-
Changes filebeat httpjson input’s append transform to create a list even with only a single valuehttps://github.com/elastic/pull/25074[25074]
-
Change logging in logs input to structure logging. Some log message formats have changed. 25299
-
All url.* fields apart from url.original in the Apache, Nginx, IIS, Traefik, S3Access, Cisco, F5, Fortinet, Google Workspace, Imperva, Microsoft, Netscout, O365, Sophos, Squid, Suricata, Zeek, Zia, Zoom, and ZScaler modules are now url unescaped due to using the Elasticsearch uri_parts processor. 24699
-
Deprecated the cyberark module (replaced by cyberarkpas). 25261 25505
Heartbeat
Journalbeat
-
Improve parsing of syslog.pid in journalbeat to strip the username when present 16116
Metricbeat
-
Make use of secure port when accessing Kubelet API 16063
-
Add Tomcat overview dashboard 14026
-
Move service config under metrics and simplify metric types. 18691
-
Fix ECS compliance of user.id field in system/users metricset 19019
-
Remove "invalid zero" metrics on Windows and Darwin, don’t report linux-only memory and diskio metrics when running under agent. 21457
-
Change cloud.provider from googlecloud to gcp. 21775
-
API address and shard ID are required settings in the Cloud Foundry module. 21759
-
Rename googlecloud module to gcp module. 22246
-
Fix ECS compliance of user.id field in system/users metricset 19019
-
Use ingress/egress instead of inbound/outbound for system/socket metricset. 22992
-
Add new dashboard for VSphere host cluster and virtual machine 14135
-
kubernetes.container.cpu.limit.cores and kubernetes.container.cpu.requests.cores are now floats. 11975
-
Change types of numeric metrics from Kubelet summary api to double so as to cover big numbers. 23335
-
Add container.image.name and containe.name ECS fields for state_container. 23802
-
Add support for Consul 1.9. 24123
-
Add support for the MemoryPressure, DiskPressure, OutOfDisk and PIDPressure status conditions in state_node. 23905
-
Store
cloudfoundry.container.cpu.pct
in decimal form and asscaled_float
. 24219 -
Remove
index_stats.created
field from Elasticsearch/index Metricset 25113 -
Remove xpack enabled flag on ES, Logstash, Beats and Kibana 24427
-
Adjust host fields to adopt new names from 1.9.0 ECS. 24312
Packetbeat
-
Redis: fix incorrectly handle with two-words redis command. 14872 14873
-
event.category
no longer contains the valuenetwork_traffic
because this is not a valid ECS event category value. 20556 -
Added redact_headers configuration option, to allow HTTP request headers to be redacted whilst keeping the header field included in the beat. 15353
-
Add dns.question.subdomain and dns.question.top_level_domain fields. 14578
Winlogbeat
-
Add support to Sysmon file delete events (event ID 23). 18094
-
Improve ECS field mappings in Sysmon module.
related.hash
,related.ip
, andrelated.user
are now populated. 18364 -
Improve ECS field mappings in Sysmon module. Hashes are now also populated to the corresponding
process.hash
,process.pe.imphash
,file.hash
, orfile.pe.imphash
. 18364 -
Improve ECS field mappings in Sysmon module.
file.name
,file.directory
, andfile.extension
are now populated. 18364 -
Improve ECS field mappings in Sysmon module.
rule.name
is populated for all events when present. 18364 -
Add Powershell module. Support for event ID’s:
400
,403
,600
,800
,4103
,4014
,4105
,4106
. 16262 18526 -
Fix Powershell processing of downgraded engine events. 18966
-
Fix unprefixed fields in
fields.yml
for Powershell module 18984 -
Remove top level
hash
property from sysmon events 20653 -
Use ECS 1.7 ingress/egress instead of inbound/outbound network.direction in sysmon. 22997
Functionbeat
Affecting all Beats
-
Fix events being dropped if they contain a floating point value of NaN or Inf. 25051
-
Fix
add_cloud_metadata
to better support modifying sub-fields with other processors. 13808 -
TLS or Beats that accept connections over TLS and validate client certificates. 14146
-
Fix panics that could result from invalid TLS certificates. This can affect Beats that connect over TLS, or Beats that accept connections over TLS and validate client certificates. 14146
-
Fix panic in the Logstash output when trying to send events to closed connection. 15568
-
Fix a race condition with the Kafka pipeline client, it is possible that
Close()
get called beforeConnect()
. 11945 -
Allow users to configure only
cluster_uuid
setting undermonitoring
namespace. 14338 -
Update replicaset group to apps/v1 15802
-
Fix missing output in dockerlogbeat 15719
-
Do not load dashboards where not available. 15802
-
Fix issue where TLS settings would be ignored when a forward proxy was in use. 15516
-
Update replicaset group to apps/v1 15802
-
Fix issue where default go logger is not discarded when either * or stdout is selected. 10251 15708
-
Upgrade go-ucfg to latest v0.8.1. 15937
-
Fix index names for indexing not always guaranteed to be lower case. 16081
-
Add
ssl.ca_sha256
option to the supported TLS option, this allow to check that a specific certificate is used as part of the verified chain. 15717 -
Fix loading processors from annotation hints. 16348
-
Fix an issue that could cause redundant configuration reloads. 16440
-
Fix k8s pods labels broken schema. 16480
-
Fix k8s pods annotations broken schema. 16554
-
Upgrade go-ucfg to latest v0.8.3. 16450
-
Fix
NewContainerMetadataEnricher
to use default config for kubernetes module. 16857 -
Improve some logging messages for add_kubernetes_metadata processor 16866
-
Fix k8s metadata issue regarding node labels not shown up on root level of metadata. 16834
-
Fail to start if httpprof is used and it cannot be initialized. 17028
-
Fix concurrency issues in convert processor when used in the global context. 17032
-
Fix bug with
monitoring.cluster_uuid
setting not always being exposed via GET /state Beats API. 16732 17420 -
Fix building on FreeBSD by removing build flags from
add_cloudfoundry_metadata
processor. 17486 -
Improve some logging messages for add_kubernetes_metadata processor https://github.com/elastic/beats/pull/16866{16866}
-
Do not rotate log files on startup when interval is configured and rotateonstartup is disabled. 17613
-
Fix goroutine leak and Elasticsearch output file descriptor leak when output reloading is in use. 10491 17381
-
Fix
setup.dashboards.index
setting not working. 17749 -
Fix Elasticsearch license endpoint URL referenced in error message. 17880 18030
-
Fix panic when assigning a key to a
nil
value in an event. 18143 -
Change
decode_json_fields
processor, to merge parsed json objects with existing objects in the event instead of fully replacing them. 17958 -
Gives monitoring reporter hosts, if configured, total precedence over corresponding output hosts. 17937 17991
-
Change
decode_json_fields
processor, to merge parsed json objects with existing objects in the event instead of fully replacing them. 17958 -
[Autodiscover] Check if runner is already running before starting again. 18564
-
Fix an issue where error messages are not accurate in mapstriface. 18662 18663
-
Fix regression in
add_kubernetes_metadata
, so configuredindexers
andmatchers
are used if defaults are not disabled. 18481 18818 -
Fix the
translate_sid
processor’s handling of unconfigured target fields. 18990 18991 -
Fix metrics hints builder to avoid wrong container metadata usage when port is not exposed 18979
-
Server-side TLS config now validates certificate and key are both specified 19584
-
Fix terminating pod autodiscover issue. 20084
-
Fix seccomp policy for calls to
chmod
andchown
. 20054 -
Output errors when Kibana index pattern setup fails. 20121
-
Fix issue in autodiscover that kept inputs stopped after config updates. 20305
-
Log debug message if the Kibana dashboard can not be imported from the archive because of the invalid archive directory structure 12211, 13387
-
Add service resource in k8s cluster role. 20546
-
[Metricbeat][Kubernetes] Change cluster_ip field from ip to keyword. 20571
-
Rename cloud.provider
az
value toazure
inside the add_cloud_metadata processor. 20689 -
Add missing country_name geo field in
add_host_metadata
andadd_observer_metadata
processors. 20796 20811 -
[Autodiscover] Handle input-not-finished errors in config reload. 20915
-
Explicitly detect missing variables in autodiscover configuration, log them at the debug level. 20568 20898
-
Fix
libbeat.output.write.bytes
andlibbeat.output.read.bytes
metrics of the Elasticsearch output. 20752 21197 -
The
o365input
ando365
module now recover from an authentication problem or other fatal errors, instead of terminating. 21258 -
Orderly close processors when processing pipelines are not needed anymore to release their resources. 16349
-
Fix memory leak and events duplication in docker autodiscover and add_docker_metadata. 21851
-
Fixed documentation for commands in beats dev guide 22194
-
Fix duplicated pod events in kubernetes autodiscover for pods with init or ephemeral containers. 22438
-
Fix FileVersion contained in Windows exe files. 22581
-
Fix index template loading when the new index format is selected. 22482 22682
-
Periodic metrics in logs will now report
libbeat.output.events.active
andbeat.memstats.rss
as gauges (rather than counters). 22877 -
Use PROGRAMDATA environment variable instead of C:\ProgramData for windows install service 22874
-
Fix reporting of cgroup metrics when running under Docker 22879
-
Fix typo in config docs 23185
-
Add FAQ entry for madvdontneed variable 23429
-
Fix panic due to unhandled DeletedFinalStateUnknown in k8s OnDelete 23419
-
Fix error loop with runaway CPU use when the Kafka output encounters some connection errors 23484
-
Fix ILM setup log reporting that a policy or an alias was created, even though the creation of any resource was disabled. 24046 24480
-
Fix ILM alias not being created if
setup.ilm.check_exists: false
andsetup.ilm.overwrite: true
has been configured. 24480 -
Fix issue discovering docker containers and metadata after reconnections 24318
-
Allow cgroup self-monitoring to see alternate
hostfs
paths 24334 -
Add
expand_keys
to the list of permitted config fields fordecode_json_fields
{24862}[24862] -
Fix 'make setup' instructions for a new beat 24944
-
Fix discovery of short-living and failing pods in Kubernetes autodiscover 22718 24742
-
Fix panic when overwriting metadata 24741
-
Fix role_arn to work with access keys for AWS. 25446
-
Fix
community_id
processor so that ports greater than 65535 aren’t valid. 25409
Auditbeat
-
system/socket: Fixed compatibility issue with kernel 5.x. 15771
-
system/package: Fix parsing of Installed-Size field of DEB packages. 16661 17188
-
system module: Fix panic during initialisation when /proc/stat can’t be read. 17569
-
system/package: Fix an error that can occur while trying to persist package metadata. 18536 18887
-
system/socket: Fix dataset using 100% CPU and becoming unresponsive in some scenarios. 19033 19764
-
system/socket: Fixed tracking of long-running connections. 19033
-
system/package: Fix librpm loading on Fedora 31/32. NNNN
-
file_integrity: Create fsnotify watcher only when starting file_integrity module 19505
-
auditd: Fix spelling of anomaly in
event.category
. -
auditd: Fix typo in
event.action
ofremoved-user-role-from
. 19300 -
auditd: Fix typo in
event.action
ofused-suspicious-link
. 19300 -
system/socket: Fix kprobe grouping to allow running more than one instance. 20325
-
system/socket: Fixed a crash due to concurrent map read and write. 21192 21690
-
auditd: Fix an error condition causing a lot of
audit_send_reply
kernel threads being created. 22673 -
system/socket: Fixed start failure when run under config reloader. 20851 21693
-
system/socket: Having some CPUs unavailable to Auditbeat could cause startup errors or event loss. 22827
-
Note incompatibility of system/socket on ARM. 23381
Filebeat
-
Fix mapping of fortinet.firewall.mem as integer. 19335
-
Ensure all zeek timestamps include millisecond precision. 14599 16766
-
Fix s3 input hanging with GetObjectRequest API call by adding context_timeout config. 15502 15590
-
Fix mapping error when zeek weird logs do not contain IP addresses. 15906
-
Improve
elasticsearch/audit
fileset to handle timestamps correctly. 15942 -
Prevent Elasticsearch from spewing log warnings about redundant wildcards when setting up ingest pipelines for the
elasticsearch
module. 15840 15900 -
Fix mapping error for cloudtrail additionalEventData field 16088
-
Fix a connection error in httpjson input. 16123
-
Fix integer overflow in S3 offsets when collecting very large files. 22523
-
Fix CredentialsJSON unpacking for
gcp-pubsub
andhttpjson
inputs. 23277 -
Strip Azure Eventhub connection string in debug logs. 25066
-
Fix o365 module config when client_secret contains special characters. 25058
-
Fix issue with m365_defender, when parsing incidents that has no alerts attached: 25421
Filebeat
-
cisco/asa fileset: Fix parsing of 302021 message code. 14519
-
Fix filebeat azure dashboards, event category should be
Alert
. 14668 -
Fix s3 input with cloudtrail fileset reading json file. 16374 16441
-
Rewrite azure filebeat dashboards, due to changes in kibana. 16466
-
Adding the var definitions in azure manifest files, fix for errors when executing command setup. 16270 16468
-
Fix merging of fileset inputs to replace paths and append processors. 16450
-
Add queue_url definition in manifest file for aws module. 16640
-
Fix issue where autodiscover hints default configuration was not being copied. 16987
-
Fix Elasticsearch
_id
field set by S3 and Google Pub/Sub inputs. 17026 -
Add queue_url definition in manifest file for aws module. https://github.com/elastic/beats/pull/16640{16640}
-
Fix default index pattern in IBM MQ filebeat dashboard. 17146
-
Fix
elasticsearch.gc
fileset to not collect all logs when Elasticsearch is running in Docker. 13164 16583 17164 -
Fixed a mapping exception when ingesting CEF logs that used the spriv or dpriv extensions. 17216 17220
-
CEF: Fixed decoding errors caused by trailing spaces in messages. 17253
-
Fixed a mapping exception when ingesting Logstash plain logs (7.4+) with pipeline ids containing non alphanumeric chars. 17242 17243
-
Fixed MySQL slowlog module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. 17086 17156
-
Fix
elasticsearch.audit
data ingest pipeline to be more forgiving with date formats found in Elasticsearch audit logs. 17406 -
CEF: Fixed decoding errors caused by trailing spaces in messages. 17253
-
Fixed activemq module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. 17428
-
Remove migrationVersion map 7.7.0 reference from Kibana dashboard file to fix backward compatibility issues. 17425
-
Fix issue 17734 to retry on rate-limit error in the Filebeat httpjson input. 17734 17735
-
Fixed
cloudfoundry.access
to have the correctcloudfoundry.app.id
contents. 17847 -
Fixing
ingress_controller.
fields to be of type keyword instead of text. 17834 -
Fixed typo in log message. 17897
-
Fix Cisco ASA ASA 3020** and 106023 messages 17964
-
Unescape file name from SQS message. 18370
-
Improve cisco asa and ftd pipelines' failure handler to avoid mapping temporary fields. 18391 18392
-
Fix source.address not being set for nginx ingress_controller 18511
-
Fix PANW module wrong mappings for bytes and packets counters. 18522 18525
-
Fixed ingestion of some Cisco ASA and FTD messages when a hostname was used instead of an IP for NAT fields. 14034 18376
-
Fix a rate limit related issue in httpjson input for Okta module. 18530 18534
-
Fix
googlecloud.audit
pipeline to only take in fields that are explicitly defined by the dataset. 18465 18472 -
Fix
o365.audit
failing to ingest events when ip address is surrounded by square brackets. 18587 18591 -
Fix Kubernetes Watcher goroutine leaks when input config is invalid and
input.reload
is enabled. 18629 18630 -
Okta module now sets the Elasticsearch
_id
field to the Okta UUID value contained in each system log to minimize the possibility of duplicating events. 18953 -
Fix improper nesting of session_issuer object in aws cloudtrail fileset. 18894 18915
-
Fix
o365
module ignoringvar.api
settings. 18948 -
Fix
netflow
module to support 7 bytepad for IPFIX template. 18098 -
Fix Cisco ASA dissect pattern for 313008 & 313009 messages. 19149
-
Fix date and timestamp formats for fortigate module 19316
-
Fix memory leak in tcp and unix input sources. 19459
-
Add missing
default_field: false
to aws filesets fields.yml. 19568 -
Update container name for the azure filesets. 19899
-
Fix
o365
module ignoringvar.api
settings. 18948 -
Fix improper nesting of session_issuer object in aws cloudtrail fileset. 18894 18915
-
Fix Cisco ASA ASA 3020** and 106023 messages 17964
-
Add missing
default_field: false
to aws filesets fields.yml. 19568 -
Fix bug with empty filter values in system/service 19812
-
Fix S3 input to trim delimiter /n from each log line. 19972
-
Ignore missing in Zeek module when dropping unecessary fields. 19984
-
Fix auditd module syscall table for ppc64 and ppc64le. 20052
-
Fix s3 input parsing json file without expand_event_list_from_field. 19902 19962 20370
-
Fix millisecond timestamp normalization issues in CrowdStrike module 20035, 20138
-
Fix support for message code 106100 in Cisco ASA and FTD. 19350 20245
-
Fix event.outcome logic for azure/siginlogs fileset 20254
-
Fix
fortinet
settingevent.timezone
to the system one when notz
field present 20273 -
Fix
okta
geoip lookup in pipeline fordestination.ip
20454 -
Fix mapping exception in the
googlecloud/audit
dataset pipeline. 18465 20465 -
Fix
cisco
asa and ftd parsing of messages 106102 and 106103. 20469 -
Clone value when copy fields in processors to avoid crash. 19206 20500
-
Fix event.type for zeek/ssl and duplicate event.category for zeek/connection 20696
-
Fix event types and categories in auditd module to comply with ECS 20652
-
Update documentation in the azure module filebeat. 20815
-
Provide backwards compatibility for the
set
processor when Elasticsearch is less than 7.9.0. 20908 -
Remove wrongly mapped
tls.client.server_name
fromfortinet/firewall
fileset. 20983 -
Fix an error updating file size being logged when EOF is reached. 21048
-
Fix error when processing AWS Cloudtrail Digest logs. 21086 20943
-
Handle multiple upstreams in ingress-controller. 21215
-
Provide backwards compatibility for the
append
processor when Elasticsearch is less than 7.10.0. 21159 -
Fix checkpoint module when logs contain time field. 20567
-
Add field limit check for AWS Cloudtrail flattened fields. 21388 21382
-
Fix syslog RFC 5424 parsing in the CheckPoint module. 21854
-
Fix incorrect connection state mapping in zeek connection pipeline. 22151 22149
-
Fix handing missing eventtime and assignip field being set to N/A for fortinet module. 22361
-
Fix Zeek dashboard reference to
zeek.ssl.server.name
field. 21696 -
Fix for
field [source] not present as part of path [source.ip]
error in azure pipelines. 22377 -
Drop aws.vpcflow.pkt_srcaddr and aws.vpcflow.pkt_dstaddr when equal to "-". 22721 22716
-
Fix cisco umbrella module config by adding input variable. 22892
-
Fix network.direction logic in zeek connection fileset. 22967
-
Convert the o365 module’s
client.port
andsource.port
to numbers (from strings) in events. 22939 -
Fix Cisco ASA/FTD module’s parsing of WebVPN log message 716002. 22966
-
Fix aws s3 overview dashboard. 23045
-
Fix bad
network.direction
values in Fortinet/firewall fileset. 23072 -
Add support for organization and custom prefix in AWS/CloudTrail fileset. 23109 23126
-
Simplify regex for organization custom prefix in AWS/CloudTrail fileset. 23203 23204
-
Fix concurrent modification exception in Suricata ingest node pipeline. 23534
-
Fix handling of ModifiedProperties field in Office 365. 23777
-
Fix gcp/vpcflow module error where input type was defaulting to file. 24719
-
Improve Cisco ASA/FTD parsing of messages - better support for identity FW messages. Change network.bytes, source.bytes, and destination.bytes to long from integer since value can exceed integer capacity. Add descriptions for various processors for easier pipeline editing in Kibana UI. 23766
-
Fix usage of unallowed ECS event.outcome values in Cisco ASA/FTD pipeline. 24744.
-
Updating Oauth2 flow for m365_defender fileset. 24829
-
Change
checkpoint.source_object
from Long to Keyword. 25124 25145 -
Fix s3 input when there is a blank line in the log file. 25357
-
Remove space from field
sophos.xg.trans_src_ ip
. 25154 25250
Heartbeat
Journalbeat
Metricbeat
-
Add dedot for tags in ec2 metricset and cloudwatch metricset. 15843 15844
-
Use RFC3339 format for timestamps collected using the SQL module. 15847
-
Avoid parsing errors returned from prometheus endpoints. 15712
-
Change lookup_fields from metricset.host to service.address 15883
-
Fixed issue
logstash-xpack
module suddenly ceasing to monitor Logstash. 15974 16044 -
Fix checking tagsFilter using length in cloudwatch metricset. 14525
-
Fixed bug with
elasticsearch/cluster_stats
metricset not recording license expiration date correctly. 14541 14591 -
Log bulk failures from bulk API requests to monitoring cluster. 14303 14356
-
Fixed bug with
elasticsearch/cluster_stats
metricset not recording license ID in the correct field. 14592 -
Change lookup_fields from metricset.host to service.address 15883
-
Fix skipping protocol scheme by light modules. pull
-
Made
logstash-xpack
module once again have parity with internally-collected Logstash monitoring data. 16198 -
Revert changes in
docker
module: add size flag to docker.container. 16600 -
Fix detection and logging of some error cases with light modules. 14706
-
Fix imports after PR was merged before rebase. 16756
-
Add dashboard for
redisenterprise
module. 16752 -
Dynamically choose a method for the system/service metricset to support older linux distros. 16902
-
Reduce memory usage in
elasticsearch/index
metricset. 16503 16538 -
Check if CCR feature is available on Elasticsearch cluster before attempting to call CCR APIs from
elasticsearch/ccr
metricset. 16511 17073 -
Use max in k8s overview dashboard aggregations. 17015
-
Fix Disk Used and Disk Usage visualizations in the Metricbeat System dashboards. 12435 17272
-
Fix missing Accept header for Prometheus and OpenMetrics module. 16870 17291
-
Further revise check for bad data in docker/memory. 17400
-
Fix issue in Jolokia module when mbean contains multiple quoted properties. 17375 17374
-
Combine cloudwatch aggregated metrics into single event. 17345
-
Fix issue in Jolokia module when mbean contains multiple quoted properties. 17375 17374
-
Further revise check for bad data in docker/memory. 17400
-
Fix how we filter services by name in system/service 17400
-
Fix cloudwatch metricset missing tags collection. 17419 17424
-
check if cpuOptions field is nil in DescribeInstances output in ec2 metricset. 17418
-
Fix aws.s3.bucket.name terms_field in s3 overview dashboard. 17542
-
Fix Unix socket path in memcached. 17512
-
Fix azure storage dashboards. 17590
-
Metricbeat no longer needs to be started strictly after Logstash for
logstash-xpack
module to report correct data. 17261 17497 -
Fix pubsub metricset to collect all GA stage metrics from gcp stackdriver. 17154 17600
-
Add privileged option so as mb to access data dir in Openshift. 17606
-
Add privileged option for Auditbeat in Openshift 17637
-
Fix storage metricset to allow config without region/zone. 17623 17624
-
Fix overflow on Prometheus rates when new buckets are added on the go. 17753
-
Remove specific win32 api errors from events in perfmon. 18292 18361
-
Fix application_pool metricset after pdh changes. 18477
-
Fix tags_filter for cloudwatch metricset in aws. 18524
-
Fix panic on
metricbeat test modules
when modules are configured inmetricbeat.modules
. 18789 18797 -
Fix getting gcp compute instance metadata with partial zone/region in config. 18757
-
Add missing network.sent_packets_count metric into compute metricset in googlecloud module. 18802
-
Fix compute and pubsub dashboard for googlecloud module. 18962 18980
-
Fix crash on vsphere module when Host information is not available. 18996 19078
-
Fix incorrect usage of hints builder when exposed port is a substring of the hint 19052
-
Stop counterCache only when already started 19103
-
Fix empty field name errors in the application pool metricset. 19537
-
Set tags correctly if the dimension value is ARN 19111 19433
-
Fix bug incorrect parsing of float numbers as integers in Couchbase module 18949 19055
-
Fix mapping of service start type in the service metricset, windows module. 19551
-
Fix config example in the perfmon configuration files. 19539
-
Add missing info about the rest of the azure metricsets in the documentation. 19601
-
Fix k8s scheduler compatibility issue. 19699
-
Add support for azure light metricset app_stats. 20639
-
Fix ec2 disk and network metrics to use Sum statistic method. 20680
-
Fill cloud.account.name with accountID if account alias doesn’t exist. 20736
-
The Kibana collector applies backoff when errored at getting usage stats 20772
-
Update fields.yml in the azure module, missing metrics field. 20918
-
The
elasticsearch/index
metricset only requests wildcard expansion for hidden indices if the monitored Elasticsearch cluster supports it. 20938 -
Disable Kafka metricsets based on Jolokia by default. They require a different configuration. 20989
-
Fix panic index out of range error when getting AWS account name. 21101 21095
-
Handle missing counters in the application_pool metricset. 21071
-
Fix timestamp handling in remote_write. 21166
-
Fix remote_write flaky test. 21173
-
Visualization title fixes in aws, azure and googlecloud compute dashboards. 21098
-
Add a switch to the driver definition on SQL module to use pretty names 17378
-
Fix retrieving resources by ID for the azure module. 21711 21707
-
Use timestamp from CloudWatch API when creating events. 21498
-
Report the correct windows events for system/filesystem 21758
-
Fix azure storage event format. 21845
-
Fix panic in kubernetes autodiscover related to keystores 21843 21880
-
[Kubernetes] Remove redundant dockersock volume mount 22009
-
Revert change to report
process.memory.rss
asprocess.memory.wss
on Windows. 22055 -
Add a switch to the driver definition on SQL module to use pretty names 17378
-
Remove io.time from windows 22237
-
Add interval information to
monitor
metricset in azure. 22152 -
Change Session ID type from int to string 22359
-
Fix filesystem types on Windows in filesystem metricset. 22531
-
Fix failiures caused by custom beat names with more than 15 characters 22550
-
Stop generating NaN values from Cloud Foundry module to avoid errors in outputs. 22634
-
Update NATS dashboards to leverage connection and route metricsets 22646
-
Fix rate metrics in Kafka broker metricset by using last minute rate instead of mean rate. 22733
-
Fix
logstash
module whenxpack.enabled: true
is set from emitting redundant events. 22808 -
Fix SQL module mapping NULL values as string 18955 elastic#18898[18898
-
Modify doc for app_insights metricset to contain example of config. 20185
-
Add required option for
metrics
in app_insights. 20406 -
Groups same timestamp metric values to one event in the app_insights metricset. 20403
-
Add support for azure light metricset app_stats. 20639
-
Fix remote_write flaky test. 21173
-
Remove io.time from windows 22237
-
Change vsphere.datastore.capacity.used.pct value to betweeen 0 and 1. 23148
-
Fix incorrect types of fields GetHits and Ops in NodeInterestingStats for Couchbase module in Metricbeat 21021 23287
-
Update config in
windows.yml
file. 23027https://github.com/elastic/beats/pull/23327[23327] -
Fix GCP not able to request Cloudfunctions metrics if a region filter was set 24218
-
Fix type of
uwsgi.status.worker.rss
type. 24468 -
Accept text/plain type by default for prometheus client scraping. 24622
-
Use working set bytes to calculate the pod memory limit pct when memory usage is not reported (ie. Windows pods). 25428
-
Fix copy-paste error in libbeat docs. 25448
-
Fix azure billing dashboard. 25554
Packetbeat
Winlogbeat
-
Fix invalid IP addresses in DNS query results from Sysmon data. 18432 18436
-
Fields from Winlogbeat modules were not being included in index templates and patterns. 18983
-
Add source.ip validation for event ID 4778 in the Security module. 19627
-
Protect against accessing undefined variables in Sysmon module. 22219 22236
-
Protect against accessing an undefined variable in Security module. 22937
-
Change
event.code
andwinlog.event_id
from int to keyword. 25176 -
Fix related.ip field in renameCommonAuthFields 24892
Functionbeat
Elastic Logging Plugin
Affecting all Beats
-
Decouple Debug logging from fail_on_error logic for rename, copy, truncate processors 12451
-
Allow a beat to ship monitoring data directly to an Elasticsearch monitoring cluster. 9260
-
Updated go-seccomp-bpf library to v1.1.0 which updates syscall lists for Linux v5.0. 11394
-
add_host_metadata is no GA. 13148
-
Add
providers
setting toadd_cloud_metadata
processor. 13812 -
Ensure that init containers are no longer tailed after they stop 14394
-
Fingerprint processor adds a new xxhash hashing algorithm 15418
-
Add configuration for APM instrumentation and expose the tracer trough the Beat object. 17938
-
Include network information by default on add_host_metadata and add_observer_metadata. 15347 16077
-
Add support for multiple password in redis output. 16058 16206
-
Add support for Histogram type in fields.yml 16570
-
Remove experimental flag from
setup.template.append_fields
16576 -
Add
add_cloudfoundry_metadata
processor to annotate events with Cloud Foundry application data. 16621 -
Add Kerberos support to Kafka input and output. 16781
-
Add
add_cloudfoundry_metadata
processor to annotate events with Cloud Foundry application data. 16621 -
Add support for kubernetes provider to recognize namespace level defaults 16321
-
Add
translate_sid
processor on Windows for converting Windows security identifier (SID) values to names. 7451 16013 -
Add capability of enrich
container.id
with process id inadd_process_metadata
processor 15947 -
Update RPM packages contained in Beat Docker images. 17035
-
Update supported versions of
redis
output. 17198 -
Update documentation for system.process.memory fields to include clarification on Windows os’s. 17268
-
Add
replace
processor for replacing string values of fields. 17342 -
Add optional regex based cid extractor to
add_kubernetes_metadata
processor. 17360 -
Add
urldecode
processor to for decoding URL-encoded fields. 17505 -
Add support for AWS IAM
role_arn
in credentials config. 17658 12464 -
Add keystore support for autodiscover static configurations. 16306
-
Add Kerberos support to Elasticsearch output. 17927
-
Add k8s keystore backend. 18096
-
Add support for fixed length extraction in
dissect
processor. 17191 -
Add support for basic ECS logging. 17974
-
Add config example of how to skip the
add_host_metadata
processor when forwarding logs. 13920 18153 -
When using the
decode_json_fields
processor, decoded fields are now deep-merged into existing event. 17958 -
Add backoff configuration options for the Kafka output. 16777 17808
-
Update documentation for system.process.memory fields to include clarification on Windows os’s. 17268
-
Add
urldecode
processor to for decoding URL-encoded fields. 17505 -
Add keystore support for autodiscover static configurations. {pull]16306[16306]
-
When using the
decode_json_fields
processor, decoded fields are now deep-merged into existing event. 17958 -
Add keystore support for autodiscover static configurations. {pull]16306[16306]
-
Add TLS support to Kerberos authentication in Elasticsearch. 18607
-
Add support for multiple sets of hints on autodiscover 18883
-
Add config option
rotate_on_startup
to file output 19150 19347 -
Add a configurable delay between retries when an app metadata cannot be retrieved by
add_cloudfoundry_metadata
. 19181 -
Added the
max_cached_sessions
option to the script processor. 19562 -
Add support for DNS over TLS for the dns_processor. 19321
-
Add minimum cache TTL for successful DNS responses. 18986
-
Set index.max_docvalue_fields_search in index template to increase value to 200 fields. 20215
-
Add capability of enriching process metadata with contianer id also for non-privileged containers in
add_process_metadata
processor. 19767 -
Add replace_fields config option in add_host_metadata for replacing host fields. 20490 20464
-
Add option to select the type of index template to load: legacy, component, index. 21212
-
Add istiod metricset. 21519
-
Release
add_cloudfoundry_metadata
as GA. 21525 -
Add support for OpenStack SSL metadata APIs in
add_cloud_metadata
. 21590 -
Add cloud.account.id for GCP into add_cloud_metadata processor. 21776
-
Add proxy metricset for istio module. 21751
-
Add kubernetes.node.hostname metadata of Kubernetes node. 22189
-
Enable always add_resource_metadata for Pods and Services of kubernetes autodiscovery. 22189
-
Add add_resource_metadata option setting (always enabled) for add_kubernetes_metadata setting. 22189
-
Added Kafka version 2.2 to the list of supported versions. 22328
-
Add support for ephemeral containers in kubernetes autodiscover and
add_kubernetes_metadata
. 22389 22439 -
Added support for wildcard fields and keyword fallback in beats setup commands. 22521
-
Fix polling node when it is not ready and monitor by hostname 22666
-
Add
expand_keys
option todecode_json_fields
processor andjson
input, to recusively de-dot and expand json keys into hierarchical object structures 22849 -
Update k8s client and release k8s leader lock gracefully 22919
-
Improve equals check. 22778
-
Added "detect_mime_type" processor for detecting mime types 22940
-
Improve event normalization performance 22974
-
Add tini as init system in docker images 22137
-
Added "add_network_direction" processor for determining perimeter-based network direction. 23076
-
Added new
rate_limit
processor for enforcing rate limits on event throughput. 22883 -
Allow node/namespace metadata to be disabled on kubernetes metagen and ensure add_kubernetes_metadata honors host 23012
-
Add new ECS 1.9 field
cloud.service.name
toadd_cloud_metadata
processor. 24993 -
Libbeat: report queue capacity, output batch size, and output client count to monitoring. 24700
-
Add kubernetes.pod.ip field in kubernetes metadata. 25037
-
Discover changes in Kubernetes namespace metadata as soon as they happen. 25117
-
Add support for defining explicitly named dynamic templates without path/type match criteria 25422
-
Add new setting
gc_percent
for tuning the garbage collector limits via configuration file. 25394 -
Add
unit
andmetric_type
properties to fields.yml for populating field metadata in Elasticsearch templates 25419 -
Add new option
suffix
tologging.files
to control how log files are rotated. 25464
Auditbeat
-
Reference kubernetes manifests include configuration for auditd and enrichment with kubernetes metadata. 17431
-
Reference kubernetes manifests mount data directory from the host, so data persist between executions in the same node. 17429
-
Log to stderr when running using reference kubernetes manifests. 174443
-
Fix syscall kprobe arguments for 32-bit systems in socket module. 17500
-
Fix memory leak on when we miss socket close kprobe events. 17500
-
Add system module process dataset ECS categorization fields. 18032
-
Add system module socket dataset ECS categorization fields. 18036
-
Add ECS categories for system module host dataset. 18031
-
Add system module package dataset ECS categorization fields. 18033
-
Add system module login dataset ECS categorization fields. 18034
-
Add system module user dataset ECS categorization fields. 18035
-
Add file integrity module ECS categorization fields. 18012
-
Add
file.mime_type
,file.extension
, andfile.drive_letter
for file integrity module. 18012 -
Add ECS categorization info for auditd module 18596
Filebeat
-
Set event.outcome field based on googlecloud audit log output. 15731
-
Add dashboard for AWS ELB fileset. 15804
-
Add dashboard for AWS vpcflow fileset. 16007
-
container
anddocker
inputs now support reading of labels and env vars written by docker JSON file logging driver. 8358 -
Add
index
option to all inputs to directly set a per-input index value. 14010 -
Add ECS tls fields to zeek:smtp,rdp,ssl and aws:s3access,elb 15757 15936
-
Add custom string mapping to CEF module to support Forcepoint NGFW 14663 15910
-
Add ingress nginx controller fileset 16197
-
move create-[module,fileset,fields] to mage and enable in x-pack/filebeat 15836
-
Add ECS categorization fields to activemq module. 16151 16201
-
Add a TLS test and more debug output to httpjson input 16315
-
Add an SSL config example in config.yml for filebeat MISP module. 16320
-
Improve ECS categorization, container & process field mappings in auditd module. 16153 16280
-
Improve ECS categorization field mappings in googlecloud module. 16030 16500
-
Add cloudwatch fileset and ec2 fileset in aws module. 13716 16579
-
Improve ECS categorization field mappings in kibana module. 16168 16652
-
Improve the decode_cef processor by reducing the number of memory allocations. 16587
-
Add
cloudfoundry
input to send events from Cloud Foundry. 16586 -
Improve ECS categorization field mappings in iis module. 16165 16618
-
Improve ECS categorization field mapping in kafka module. 16167 16645
-
Allow users to override pipeline ID in fileset input config. 9531 16561
-
Add
o365audit
input type for consuming events from Office 365 Management Activity API. 16196 16244 -
Improve ECS categorization field mappings in logstash module. 16169 16668
-
Update filebeat httpjson input to support pagination via Header and Okta module. 16354
-
Improve ECS categorization field mapping in icinga module. 16164 16533
-
Improve ECS categorization field mappings in ibmmq module. 16163 16532
-
Improve ECS categorization, host field mappings in elasticsearch module. 16160 16469
-
Improve ECS categorization field mappings in suricata module. 16181 16843
-
Improve ECS categorization field mappings in iptables module. 16166 16637
-
Add Filebeat Okta module. 16362
-
Add custom string mapping to CEF module to support Check Point devices. 16041 16907
-
Add a TLS test and more debug output to httpjson input 16315
-
Add an SSL config example in config.yml for filebeat MISP module. 16320
-
Improve ECS categorization, container & process field mappings in auditd module. 16153 16280
-
Add cloudwatch fileset and ec2 fileset in aws module. 13716 16579
-
Improve the decode_cef processor by reducing the number of memory allocations. 16587
-
Add custom string mapping to CEF module to support Forcepoint NGFW 14663 15910
-
Improve ECS categorization, host field mappings in elasticsearch module. 16160 16469
-
Added new module
o365
for ingesting Office 365 management activity API events. 16196 16386 -
Add source field in k8s events 17209
-
Added new module
crowdstrike
for ingesting Crowdstrike Falcon streaming API endpoint event data. 16988 -
Added documentation for running Filebeat in Cloud Foundry. 17275
-
Improve ECS categorization field mappings in mongodb module. 16170 17371
-
Improve ECS categorization field mappings for mssql module. 16171 17376
-
Added access_key_id, secret_access_key and session_token into aws module config. 17456
-
Add dashboard for Google Cloud Audit and AWS CloudTrail. 17379
-
Improve ECS categorization field mappings for mysql module. 16172 17491
-
Release Google Cloud module as GA. 17511
-
Add config option to select a different azure cloud env in the azure-eventhub input and azure module. 17649 17659
-
Added new Checkpoint Syslog filebeat module. 17682
-
Improve ECS categorization field mappings for nats module. 16173 17550
-
Add support for v10, v11 and v12 logs on Postgres 13810 17732
-
Enhance
elasticsearch/server
fileset to handle ECS-compatible logs emitted by Elasticsearch. 17715 17714 -
Add support for Google Application Default Credentials to the Google Pub/Sub input and Google Cloud modules. 15668
-
Enhance
elasticsearch/deprecation
fileset to handle ECS-compatible logs emitted by Elasticsearch. 17715 17728 -
Enhance
elasticsearch/slowlog
fileset to handle ECS-compatible logs emitted by Elasticsearch. 17715 17729 -
Improve ECS categorization field mappings in misp module. 16026 17344
-
Added Unix stream socket support as an input source and a syslog input source. 17492
-
Added new Fortigate Syslog filebeat module. 17890
-
Improve ECS categorization field mappings in postgresql module. 16177 17914
-
Improve ECS categorization field mappings in rabbitmq module. 16178 17916
-
Make
decode_cef
processor GA. 17944 -
Improve ECS categorization field mappings in redis module. 16179 17918
-
Improve ECS categorization field mappings for zeek module. 16029 17738
-
Improve ECS categorization field mappings for netflow module. 16135 18108
-
Added documentation for running Filebeat in Cloud Foundry. 17275
-
Added access_key_id, secret_access_key and session_token into aws module config. 17456
-
Release Google Cloud module as GA. 17511
-
Update filebeat httpjson input to support pagination via Header and Okta module. 16354
-
Added new Checkpoint Syslog filebeat module. 17682
-
Added Unix stream socket support as an input source and a syslog input source. 17492
-
Added new Fortigate Syslog filebeat module. 17890
-
Change the
json.*
input settings implementation to merge parsed json objects with existing objects in the event instead of fully replacing them. 17958 -
Improve ECS categorization field mappings in osquery module. 16176 17881
-
Add http_endpoint input. 18298
-
Add support for array parsing in azure-eventhub input. 18585
-
Added
observer.vendor
,observer.product
, andobserver.type
to PANW module events. 18223 -
The
logstash
module can now automatically detect the log file format (JSON or plaintext) and process it accordingly. 9964 18095 -
Added http_endpoint inputhttps://github.com/elastic/pull/18298[18298]
-
Add support for array parsing in azure-eventhub input. 18585
-
Added
observer.vendor
,observer.product
, andobserver.type
to PANW module events. 18223 -
Improve ECS categorization field mappings in coredns module. 16159 18424
-
Improve ECS categorization field mappings in envoyproxy module. 16161 18395
-
Improve ECS categorization field mappings in coredns module. 16159 18424
-
Improve ECS categorization field mappings in cisco module. 16028 18537
-
Add geoip AS lookup & improve ECS categorization in aws cloudtrail fileset. 18644 18958
-
Add support for v1 consumer API in Cloud Foundry input, use it by default. 19125
-
Explicitly set ECS version in all Filebeat modules. 19198
-
Add new mode to multiline reader to aggregate constant number of lines 18352
-
Add automatic retries and exponential backoff to httpjson input. 18956
-
Add awscloudwatch input. 19025
-
Add new mode to multiline reader to aggregate constant number of lines 18352
-
Changed the panw module to pass through (rather than drop) message types other than threat and traffic. 16815 19375
-
Improve ECS categorization field mappings in traefik module. 16183 19379
-
Improve ECS categorization field mappings in azure module. 16155 19376
-
Add text & flattened versions of fields with unknown subfields in aws cloudtrail fileset. 18866 19121
-
Add experimental dataset tomcat/log for Apache TomCat logs 19713
-
Add experimental dataset netscout/sightline for Netscout Arbor Sightline logs 19713
-
Add experimental dataset barracuda/waf for Barracuda Web Application Firewall logs 19713
-
Add experimental dataset f5/bigipapm for F5 Big-IP Access Policy Manager logs 19713
-
Add experimental dataset bluecoat/director for Bluecoat Director logs 19713
-
Add experimental dataset cisco/nexus for Cisco Nexus logs 19713
-
Add experimental dataset citrix/virtualapps for Citrix Virtual Apps logs 19713
-
Add experimental dataset cylance/protect for Cylance Protect logs 19713
-
Add experimental dataset fortinet/clientendpoint for Fortinet FortiClient Endpoint Protection logs 19713
-
Add experimental dataset imperva/securesphere for Imperva Secure Sphere logs 19713
-
Add experimental dataset infoblox/nios for Infoblox Network Identity Operating System logs 19713
-
Add experimental dataset juniper/junos for Juniper Junos OS logs 19713
-
Add experimental dataset kaspersky/av for Kaspersky Anti-Virus logs 19713
-
Add experimental dataset microsoft/dhcp for Microsoft DHCP Server logs 19713
-
Add experimental dataset tenable/nessus_security for Tenable Nessus Security Scanner logs 19713
-
Add experimental dataset rapid7/nexpose for Rapid7 Nexpose logs 19713
-
Add experimental dataset radware/defensepro for Radware DefensePro logs 19713
-
Add experimental dataset sonicwall/firewall for Sonicwall Firewalls logs 19713
-
Add experimental dataset squid/log for Squid Proxy Server logs 19713
-
Add experimental dataset zscaler/zia for Zscaler Internet Access logs 19713
-
Add initial support for configurable file identity tracking. 18748
-
Add support for reading auditd logs that are prefixed with
node=
. 19659 -
Add event.ingested for CrowdStrike module 20138
-
Add support for additional fields and FirewallMatchEvent type events in CrowdStrike module 20138
-
Add event.ingested for Suricata module 20220
-
Add support for custom header and headersecret for filebeat http_endpoint input 20435
-
Add event.ingested to all Filebeat modules. 20386
-
Return error when log harvester tries to open a named pipe. 18682 20450
-
Convert httpjson to v2 input 20226
-
Improve Zeek x509 module with
x509
ECS mappings 20867 -
Improve Zeek SSL module with
x509
ECS mappings 20927 -
Added new properties field support for event.outcome in azure module 20998
-
Add type and sub_type to panw panos fileset 20912
-
Add related.hosts ecs field to all modules 21160
-
Keep cursor state between httpjson input restarts 20751
-
New juniper.srx dataset for Juniper SRX logs. 20017
-
Adding support for Microsoft 365 Defender (Microsoft Threat Protection) 21446
-
Adding support for FIPS in s3 input 21446
-
Adding support for Oracle Database Audit Logs 21991
-
Add SSL option to checkpoint module 19560
-
Add max_number_of_messages config into s3 input. 21993
-
Update Okta documentation for new stateful restarts. 22091
-
Added support for MySQL Enterprise audit logs. 22273
-
Rename googlecloud module to gcp module. 22214
-
Rename awscloudwatch input to aws-cloudwatch. 22228
-
Rename google-pubsub input to gcp-pubsub. 22213
-
Copy tag names from MISP data into events. 21664
-
Added DNS response IP addresses to
related.ip
in Suricata module. 22291 -
Added TLS JA3 fingerprint, certificate not_before/not_after, certificate SHA1 hash, and certificate subject fields to Zeek SSL dataset. 21696
-
Add platform logs in the azure filebeat module. 22371
-
Added
event.ingested
field to data from the Netflow module. 22412 -
Improve panw ECS url fields mapping. 22481
-
Improve Nats filebeat dashboard. 22726
-
Add support for UNIX datagram sockets in
unix
input. {issues}18632[18632] 22699 -
Add new httpjson input features and mark old config ones for deprecation 22320
-
Add support for Snyk Vulnerability and Audit API. 22677
-
Add logic for external network.direction in sophos xg fileset 22973
-
Add
http.request.mime_type
for Elasticsearch audit log fileset. 22975 -
Add configuration option to set external and internal networks for panw panos fileset 22998
-
Add
subbdomain
fields for rsa2elk modules. 23035 -
Add subdomain enrichment for suricata/eve fileset. 23011
-
Add subdomain enrichment for zeek/dns fileset. 23011
-
Add
event.category
"configuration" to auditd module events. 23010 -
Add
event.category
"configuration" to gsuite module events. 23010 -
Add
event.category
"configuration" to o365 module events. 23010 -
Add
event.category
"configuration" to zoom module events. 23010 -
Add
network.direction
to auditd/log fileset. 23041 -
Preserve AWS CloudTrail eventCategory in aws.cloudtrail.event_category. 22776 22805
-
Migrate microsoft/defender_atp to httpjson v2 config 23017
-
Migrate microsoft/m365_defender to httpjson v2 config 23018
-
Add top_level_domain enrichment for suricata/eve fileset. 23046
-
Add top_level_domain enrichment for zeek/dns fileset. 23046
-
Add
observer.egress.zone
andobserver.ingress.zone
for cisco/asa and cisco/ftd filesets. 23068 -
Allow cisco/asa and cisco/ftd filesets to override network directionality based off of zones. 23068
-
Add
network.direction
to netflow/log fileset. 23052 -
Allow cef and checkpoint modules to override network directionality based off of zones 23066
-
Add the ability to override
network.direction
based on interfaces in Fortinet/firewall fileset. 23072 -
Add
network.direction
override by specifyinginternal_networks
in gcp module. 23081 -
Migrate okta to httpjson v2 config 23059
-
Misp improvements: Migration to httpjson v2 config, pagination and deduplication ID 23070
-
Add Google Workspace module and mark Gsuite module as deprecated 22950
-
Mark m365 defender, defender atp, okta and google workspace modules as GA 23113
-
Added
alternative_host
option to google pubsub input 23215 -
Support X-Forwarder-For in IIS logs. 192142
-
Add support for logs generated by servers configured with
log_statement
andlog_duration
in PostgreSQL module. 24607 -
Added fifteen new message IDs to Cisco ASA/FTD pipeline. 24744
-
Added NTP fileset to Zeek module 24224
-
Added
http.request.id
tonginx/ingress_controller
andelasticsearch/audit
. 24994 -
Add
awsfargate
module to collect container logs from Amazon ECS on Fargate. 25041 -
New module
cyberarkpas
for CyberArk Privileged Access Security audit logs. 24803 -
Add
uri_parts
processor to Apache, Nginx, IIS, Traefik, S3Access, Cisco, F5, Fortinet, Google Workspace, Imperva, Microsoft, Netscout, O365, Sophos, Squid, Suricata, Zeek, Zia, Zoom, and ZScaler modules ingest pipelines. 19088 24699 -
New module
zookeeper
for Zookeeper service and audit logs 25061 25128 -
Add parsing for
haproxy.http.request.raw_request_line
field 25480 25482 -
Mark
filestream
input beta. 25560
Heartbeat
Journalbeat
Metricbeat
-
Move the windows pdh implementation from perfmon to a shared location in order for future modules/metricsets to make use of. 15503
-
Add DynamoDB AWS Metricbeat light module 15097
-
Release elb module as GA. 15485
-
Add a
system/network_summary
metricset 15196 -
Add mesh metricset for Istio Metricbeat module 15535
-
Add IBM MQ light-weight Metricbeat module 15301
-
Add mixer metricset for Istio Metricbeat module 15696
-
Add pilot metricset for Istio Metricbeat module 15761
-
Make the
system/cpu
metricset collect normalized CPU metrics by default. 15618 15729 -
Add galley metricset for Istio Metricbeat module 15857
-
Add STAN dashboard 15654
-
Add support for Unix socket in Memcached metricbeat module. 13685 15822
-
Add
up
metric to prometheus metrics collected from host 15948 -
Add citadel metricset for Istio Metricbeat module 15990
-
Add collecting AuroraDB metrics in rds metricset. 14142 16004
-
Reuse connections in SQL module. 16001
-
Improve the
logstash
module (whenxpack.enabled
is set totrue
) to use the overridecluster_uuid
returned by Logstash APIs. 15772 15795 -
Add kubernetes storage class support via kube-state-metrics. 16145
-
Add database_account azure metricset. 15758
-
Add support for NATS 2.1. 16317
-
Add Load Balancing metricset to GCP 15559
-
Add support for Dropwizard metrics 4.1. 16332
-
Add azure container metricset in order to monitor containers. 15751 16421
-
Improve the
haproxy
module to support metrics exposed via HTTPS. 14579 16333 -
Add filtering option for prometheus collector. 16420
-
Add metricsets based on Ceph Manager Daemon to the
ceph
module. 7723 16254 -
Add collecting tags and tags_filter for rds metricset in aws module. 16605 16358
-
Add OpenMetrics Metricbeat module 16596
-
Add
cloudfoundry
module to send events from Cloud Foundry. 16671 -
Add database_account azure metricset. 15758
-
Add Load Balancing metricset to GCP 15559
-
Add OpenMetrics Metricbeat module 16596
-
Add system/users metricset as beta 16569
-
Add additional cgroup fields to docker/diskiohttps://github.com/elastic/pull/16638[16638]
-
Add PubSub metricset to Google Cloud Platform module 15536
-
Add overview dashboard for googlecloud compute metricset. 16534 16819
-
Add Prometheus remote write endpoint 16609
-
Release STAN module as GA. 16980
-
Add query metricset for prometheus module. 17104
-
Add Prometheus remote write endpoint 16609
-
Add dashboard for pubsub metricset in googlecloud module. 17161
-
Replace vpc metricset into vpn, transitgateway and natgateway metricsets. 16892
-
Use Elasticsearch histogram type to store Prometheus histograms 17061
-
Allow to rate Prometheus counters when scraping them 17061
-
Add Storage metricsets to GCP module 15598
-
Added documentation for running Metricbeat in Cloud Foundry. 17275
-
Add test for documented fields check for metricsets without a http input. 17315 17334
-
Add final tests and move label to GA for the azure module in metricbeat. 17319
-
Refactor windows/perfmon metricset configuration options and event output. 17596
-
Add PubSub metricset to Google Cloud Platform module 15536
-
Add final tests and move label to GA for the azure module in metricbeat. 17319
-
Added documentation for running Metricbeat in Cloud Foundry. 17275
-
Reference kubernetes manifests mount data directory from the host when running metricbeat as daemonset, so data persist between executions in the same node. 17429
-
Stack Monitoring modules now auto-configure required metricsets when
xpack.enabled: true
is set. [16471 17609 -
Add aggregation aligner as a config parameter for googlecloud stackdriver metricset. [17141 17719
-
Add static mapping for metricsets under aws module. 17614 17650
-
Add dashboard for googlecloud storage metricset. 18172
-
Stack Monitoring modules now auto-configure required metricsets when
xpack.enabled: true
is set. [16471 17609 -
Collect new
bulk
indexing metrics from Elasticsearch whenxpack.enabled:true
is set. https://github.com/elastic/beats/issues/ 17992 -
Remove requirement to connect as sysdba in Oracle module 15846 18182
-
Update MSSQL module to fix some SSPI authentication and add brackets to USE statements 17862]
-
Add client address to events from http server module 18336
-
Remove required for region/zone and make stackdriver a metricset in googlecloud. 16785 18398
-
Add memory metrics into compute googlecloud. 18802
-
Add Tomcat overview dashboard 14026
-
Accept prefix as metric_types config parameter in googlecloud stackdriver metricset. 19345
-
Add dashboards for googlecloud load balancing metricset. 18369
-
Add support for v1 consumer API in Cloud Foundry module, use it by default. 19268
-
The
elasticsearch/index
metricset now collects metrics for hidden indices as well. 18639 18703 -
Adds support for app insights metrics in the azure module. 18570 18940
-
Added cache and connection_errors metrics to status metricset of MySQL module 16955 19844
-
Update MySQL dashboard with connection errors and cache metrics 19913 16955
-
Add cloud.instance.name into aws ec2 metricset. 20077
-
Add
scope
setting for elasticsearch module, allowing it to monitor an Elasticsearch cluster behind a load-balancing proxy. 18539 18547 -
Add state_daemonset metricset for Kubernetes Metricbeat module 20649
-
Add host inventory metrics to azure compute_vm metricset. 20641
-
Add host inventory metrics to googlecloud compute metricset. 20391
-
Add host inventory metrics to system module. 20415
-
Add billing data collection from Cost Explorer into aws billing metricset. 20527 20103
-
Migrate
compute_vm
metricset to a light one, mapcloud.instance.id
field. 20889 -
Request prometheus endpoints to be gzipped by default 20766
-
Add latency config parameter into aws module. 20875
-
Release all kubernetes
state
metricsets as GA 20901 -
Sanitize
event.host
. 21022 -
Add support for different Azure Cloud environments in the metricbeat azure module. 21044 20988
-
Add overview and platform health dashboards to Cloud Foundry module. 21124
-
Add dashboard for pubsub metricset in googlecloud module. 21326 17137
-
Move Prometheus query & remote_write to GA. 21507
-
Expand unsupported option from namespace to metrics in the azure module. 21486
-
Map cloud data filed
cloud.account.id
to azure subscription. 21483 21381 -
Move s3_daily_storage and s3_request metricsets to use cloudwatch input. 21703
-
Duplicate system.process.cmdline field with process.command_line ECS field name. 22325
-
Add awsfargate module task_stats metricset to monitor AWS ECS Fargate. 22034
-
Add connection and route metricsets for nats metricbeat module to collect metrics per connection/route. 22445
-
Add unit file states to system/service 22557
-
Add io.ops in fields exported by system.diskio. 22066
-
kibana
module:stats
metricset no-longer collects usage-related data. 22732 -
Adjust the Apache status fields in the fleet mode. 22821
-
Add AWS Fargate overview dashboard. 22941
-
Add process.state, process.cpu.pct, process.cpu.start_time and process.memory.pct. 22845
-
Apache: convert status.total_kbytes to status.total_bytes in fleet mode. 23022
-
Release MSSQL as GA 23146
-
Add support for SASL/SCRAM authentication to the Kafka module. 24810
-
Add additional network metrics to docker/network 25354
Packetbeat
-
Add an example to packetbeat.yml of using the
forwarded
tag to disablehost
metadata fields when processing network data from network tap or mirror port. 19209 -
Add ECS fields for x509 certs, event categorization, and related IP info. 19167
-
Add initial SIP protocol support 21221
-
Add support for overriding the published index on a per-protocol/flow basis. 22134
-
Change build process for x-pack distribution 21979
-
Tuned the internal queue size to reduce the chances of events being dropped. 22650
-
Add support for "http.request.mime_type" and "http.response.mime_type". 22940
-
Upgrade to ECS 1.8.0. 23783
-
Add
event.type: [connection]
to flow events and includeend
for final flows. 24564
Functionbeat
Heartbeat
Winlogbeat
-
Add more DNS error codes to the Sysmon module. 15685
-
Add experimental event log reader implementation that should be faster in most cases. 6585 16849
-
Set process.command_line and process.parent.command_line from Sysmon Event ID 1. 17327
-
Add support for event IDs 4673,4674,4697,4698,4699,4700,4701,4702,4768,4769,4770,4771,4776,4778,4779,4964 to the Security module 17517
-
Add registry and code signature information and ECS categorization fields for sysmon module 18058
-
Add new winlogbeat security dashboard 18775
-
Add
event.outcome
to events based on the audit success and audit failure keywords. 20564 -
Add file.pe and process.pe fields to ProcessCreate & LoadImage events in Sysmon module. 17335 22217
-
Add additional event categorization for security and sysmon modules. 22988
-
Add dns.question.subdomain fields for sysmon DNS events. 22999
-
Add dns.question.top_level_domain fields for sysmon DNS events. 23046
Elastic Log Driver
Affecting all Beats
Filebeat
Heartbeat
Journalbeat
Metricbeat
Packetbeat
Winlogbeat
Functionbeat