Skip to content

Commit

Permalink
docs: Prepare Changelog for 8.11.0 (#37058) (#37059)
Browse files Browse the repository at this point in the history 
* docs: Close changelog for 8.11.0

* Update CHANGELOG.asciidoc

* Update CHANGELOG.asciidoc

* Update CHANGELOG.next.asciidoc

* Apply suggestions from code review

Co-authored-by: David Kilfoyle <[email protected]>

---------

Co-authored-by: Pierre HILBERT <[email protected]>
Co-authored-by: David Kilfoyle <[email protected]>
(cherry picked from commit 0308d29)

Co-authored-by: Elastic Machine <[email protected]>
  • Loading branch information
@mergify @elasticmachine
mergify[bot] and elasticmachine authored Nov 7, 2023
1 parent 04b3989 commit 3d5ba42
Show file tree
Hide file tree
Showing 3 changed files with 91 additions and 65 deletions.
85 changes: 85 additions & 0 deletions CHANGELOG.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,91 @@
:issue: https://github.com/elastic/beats/issues/
:pull: https://github.com/elastic/beats/pull/

[[release-notes-8.11.0]]
=== Beats version 8.11.0
https://github.com/elastic/beats/compare/v8.10.4\...v8.11.0[View commits]

==== Breaking changes

*Affecting all Beats*
- The Elasticsearch output now enables compression by default. This decreases network data usage by an average of 70-80%, in exchange for 20-25% increased CPU use and ~10% increased ingestion time. The previous default can be restored by setting the flag `compression_level: 0` under `output.elasticsearch`. {pull}36681[36681]
- The `elastic-agent-autodiscover` library is updated to version 0.6.4, disabling metadata for deployment and cronjob. Pods that will be created from deployments or cronjobs will not have the extra metadata field for `kubernetes.deployment` or `kubernetes.cronjob`, respectively. {pull}36879[36879]

*Filebeat*

- Switch types of `log.file.device`, `log.file.inode`, `log.file.idxhi`, `log.file.idxlo` and `log.file.vol` fields to strings to better align with ECS and integrations. {pull}36697[36697]

*Metricbeat*

- The System module now collects the number of threads per process. The elastic-agent-system-metrics was updated to v0.7.0 as this version collects the number of threads.

==== Bugfixes

*Affecting all Beats*

- Upgrade `elastic-agent-libs` to v0.6.0, allowing a Beat running as a Windows service to receive more than one change request. {pull}36896[36896]

*Filebeat*

- Added a fix for the Crowdstrike pipeline handling of process arrays. {pull}36496[36496]
- Fix handling of response errors in HTTPJSON and CEL request trace logging. {pull}36956[36956]

*Heartbeat*

- Fix retries to trigger on a down monitor with no previous state. {pull}36842[36842]
- Bump NodeJS minor version to 18.18.2. {pull}36961[36961]
- Fix monitor duration calculation with retries. {pull}36900[36900]

*Metricbeat*

- Fix Azure Monitor empty metricnamespace. {pull}36295[36295]
- Fix GCP compute metadata. {pull}36338[36338]
- Add missing 'TransactionType' dimension for Azure Storage Account. {pull}36413[36413]
- Add log error when statsd server fails to start. {pull}36477[36477]
- Fix CassandraConnectionClosures metric configuration. {pull}34742[34742]
- Fix event mapping implementation for statsd module. {pull}36925[36925]

*Winlogbeat*

- Fix User Account Control Attributes Table values for Security module. {issue}36999[36999] {pull}37009[37009]

==== Added

*Affecting all Beats*

- Upgrade to Go 1.20.10. {pull}36846[36846]
- Add support for forward lookups (`A`, `AAAA`, and `TXT`) in DNS processor. {issue}11416[11416] {pull}36394[36394]
- Mark `syslog` processor as GA, improve docs about how processor handles syslog messages. {issue}36416[36416] {pull}36417[36417]
- Add support for AWS external IDs. {issue}36321[36321] {pull}36322[36322]
- Disable `netinfo.enabled` option of `add-host-metadata processor`to enhance `host.ip` and `host.mac`. {pull}36506[36506]
- {Beats} will now connect to older {es} instances by default. {pull}36884[36884]
- Upgrade golang/x/net to v0.17.0. Updates the publicsuffix table used by the registered_domain processor. {pull}36969[36969]

*Filebeat*

- Reduce HTTPJSON metrics allocations. {pull}36282[36282]
- Add support for a simplified input configuraton when running under {agent}. {pull}36390[36390]
- Make HTTPJSON response body decoding errors more informative. {pull}36481[36481]
- Allow fine-grained control of entity analytics API requests for Okta provider. {issue}36440[36440] {pull}36492[36492]
- Add support for expanding `journald.process.capabilities` into the human-readable effective capabilities in the ECS `process.thread.capabilities.effective` field. {issue}36454[36454] {pull}36470[36470]
- Allow fine-grained control of entity analytics API requests for AzureAD provider. {issue}36440[36440] {pull}36441[36441]
- For request tracer logging in CEL and httpjson the request and response body are no longer included in `event.original`. The body is still present in `http.{request,response}.body.content`. {pull}36531[36531]
- Add support for Okta OAuth2 provider in the CEL input. {issue}36336[36336] {pull}36521[36521]
- Improve error logging in HTTPJSON input. {pull}36529[36529]
- Disable warning message about ingest pipeline loading when running under Elastic Agent. {pull}36659[36659]
- Add input metrics to http_endpoint input. {issue}36402[36402] {pull}36427[36427]
- Remove Event Normalization from GCP PubSub Input. {pull}36716[36716]
- Add support for new features & removed partial save mechanism in the Azure Blob Storage input. {issue}35126[35126] {pull}36690[36690]
- Improve template evaluation logging for HTTPJSON input. {pull}36668[36668]
- Add CEL partial value debug function. {pull}36652[36652]
- Add support for new features and remove partial save mechanism in the GCS input. {issue}35847[35847] {pull}36713[36713]
- Add cache processor. {pull}36786[36786]

*Packetbeat*

- Bump Windows Npcap version to v1.76. {issue}36539[36539] {pull}36549[36549]


[[release-notes-8.10.4]]
=== Beats version 8.10.4
https://github.com/elastic/beats/compare/v8.10.3\...v8.10.4[View commits]
Expand Down
70 changes: 5 additions & 65 deletions CHANGELOG.next.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,12 @@ https://github.com/elastic/beats/compare/v8.8.1\...main[Check the HEAD diff]

*Filebeat*

- Switch types of `log.file.device`, `log.file.inode`, `log.file.idxhi`, `log.file.idxlo` and `log.file.vol` fields to strings to better align with ECS and integrations. {pull}36697[36697]

*Heartbeat*
- Decreases the ES default timeout to 10 for the load monitor state requests

*Metricbeat*

- System module now collects the number of threads per process.
The elastic-agent-system-metrics was updated to v0.7.0 as the number of threads
is collected by it.

*Osquerybeat*

Expand Down Expand Up @@ -64,16 +60,7 @@ is collected by it.
- 'add_cloud_metadata' processor - update azure metadata api version to get missing `cloud.account.id` field
- Upgraded apache arrow library used in x-pack/libbeat/reader/parquet from v11 to v12.0.1 in order to fix cross-compilation issues {pull}35640[35640]
- Fix panic when MaxRetryInterval is specified, but RetryInterval is not {pull}35820[35820]
- Do not print context cancelled error message when running under agent {pull}36006[36006]
- Fix recovering from invalid output configuration when running under Elastic-Agent {pull}36016[36016]
- Improve StreamBuf append to improve performance when reading long lines from files. {pull}35928[35928]
- Eliminate cloning of event in deepUpdate {pull}35945[35945]
- Fix ndjson parser to store JSON fields correctly under `target` {issue}29395[29395]
- Support build of projects outside of beats directory {pull}36126[36126]
- Add default cgroup regex for add_process_metadata processor {pull}36484[36484] {issue}32961[32961]
- Fix environment capture by `add_process_metadata` processor. {issue}36469[36469] {pull}36471[36471]
- syslog processor - Fix the ability to use `when` conditions on the processor. {issue}36762[36762]
- upgrade elastic-agent-libs to v0.6.0, allows beat running as a windows service to receive more than one change request. {pull}36896[36896]

*Auditbeat*

Expand All @@ -87,28 +74,9 @@ is collected by it.
- Fixes "Can only start an input when all related states are finished" error when running under Elastic-Agent {pull}35250[35250] {issue}33653[33653]
- [system] sync system/auth dataset with system integration 1.29.0. {pull}35581[35581]
- [GCS Input] - Fixed an issue where bucket_timeout was being applied to the entire bucket poll interval and not individual bucket object read operations. Fixed a map write concurrency issue arising from data races when using a high number of workers. Fixed the flaky tests that were present in the GCS test suit. {pull}35605[35605]
- Fix filestream false positive log error "filestream input with ID 'xyz' already exists" {issue}31767[31767]
- Fix error message formatting from filestream input. {pull}35658[35658]
- Fix error when trying to use `include_message` parser {issue}35440[35440]
- Fix handling of IPv6 unspecified addresses in TCP input. {issue}35064[35064] {pull}35637[35637]
- Fixed a minor code error in the GCS input scheduler where a config value was being used directly instead of the source struct. {pull}35729[35729]
- Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. {pull}35772[35772]
- Fix CEL input JSON marshalling of nested objects. {issue}35763[35763] {pull}35774[35774]
- Fix metric collection in GCPPubSub input. {pull}35773[35773]
- Fix end point deregistration in http_endpoint input. {issue}35899[35899] {pull}35903[35903]
- Fix duplicate ID panic in filestream metrics. {issue}35964[35964] {pull}35972[35972]
- Improve error reporting and fix IPv6 handling of TCP and UDP metric collection. {pull}35996[35996]
- Fix handling of NUL-terminated log lines in Fortinet Firewall module. {issue}36026[36026] {pull}36027[36027]
- Make redact field configuration recommended in CEL input and log warning if missing. {pull}36008[36008]
- Fix handling of region name configuration in awss3 input {pull}36034[36034]
- Fixed concurrency and flakey tests issue in azure blob storage input. {issue}35983[35983] {pull}36124[36124]
- Fix panic when sqs input metrics getter is invoked {pull}36101[36101] {issue}36077[36077]
- Make CEL input's `now` global variable static for evaluation lifetime. {pull}36107[36107]
- Update mito CEL extension library to v1.5.0. {pull}36146[36146]
- Filter out duplicate paths resolved from matching globs. {issue}36253[36253] {pull}36256[36256]
- Fix handling of TCP/UDP address resolution during metric initialization. {issue}35064[35064] {pull}36287[36287]
- Fix handling of Juniper SRX structured data when there is no leading junos element. {issue}36270[36270] {pull}36308[36308]
- Remove erroneous error log in GCPPubSub input. {pull}36296[36296]
- Fix Filebeat Cisco module with missing escape character {issue}36325[36325] {pull}36326[36326]
- Fix panic when redact option is not provided to CEL input. {issue}36387[36387] {pull}36388[36388]
- Remove 'onFilteredOut' and 'onDroppedOnPublish' callback logs {issue}36299[36299] {pull}36399[36399]
Expand All @@ -123,9 +91,6 @@ is collected by it.
*Heartbeat*

- Fix panics when parsing dereferencing invalid parsed url. {pull}34702[34702]
- Fix retries to trigger on a down monitor with no previous state. {pull}36842[36842]
- Bump NodeJS minor version to 18.18.2. {pull}36961[36961]
- Fix monitor duration calculation with retries. {pull}36900[36900]

*Metricbeat*

Expand All @@ -139,12 +104,7 @@ is collected by it.
- Collect missing remote_cluster in elasticsearch ccr metricset {pull}34957[34957]
- Add context with timeout in AWS API calls {pull}35425[35425]
- Fix EC2 host.cpu.usage {pull}35717[35717]
- Resolve statsd module's prematurely halting of metrics parsing upon encountering an invalid packet. {pull}35075[35075]
- Fix the gap in fetching forecast API metrics at the end of each month for Azure billing module {pull}36142[36142]
- Add option in SQL module to execute queries for all dbs. {pull}35688[35688]
- Fix Azure Monitor empty metricnamespace. {pull}36295[36295]
- Fix GCP compute metadata. {pull}36338[36338]
- Add support for api_key authentication in elasticsearch module {pull}36274[36274]
- Add remaining dimensions for azure storage account to make them available for tsdb enablement. {pull}36331[36331]
- Add missing 'TransactionType' dimension for Azure Storage Account. {pull}36413[36413]
- Add log error when statsd server fails to start {pull}36477[36477]
Expand All @@ -160,7 +120,6 @@ is collected by it.

*Winlogbeat*

- Fix User Account Control Attributes Table values for Security module. {issue}36999[36999] {pull}37009[37009]

*Elastic Logging Plugin*

Expand All @@ -169,7 +128,6 @@ is collected by it.

*Affecting all Beats*

- Upgrade to Go 1.20.10. {pull}36846[36846]
- Added append Processor which will append concrete values or values from a field to target. {issue}29934[29934] {pull}33364[33364]
- When running under Elastic-Agent the status is now reported per Unit instead of the whole Beat {issue}35874[35874] {pull}36183[36183]
- Add warning message to SysV init scripts for RPM-based systems that lack `/etc/rc.d/init.d/functions`. {issue}35708[35708] {pull}36188[36188]
Expand All @@ -184,6 +142,7 @@ is collected by it.
- Raise up logging level to warning when attempting to configure beats with unknown fields from autodiscovered events/environments
- elasticsearch output now supports `idle_connection_timeout`. {issue}35616[35615] {pull}36843[36843]
- Upgrade golang/x/net to v0.17.0. Updates the publicsuffix table used by the registered_domain processor. {pull}36969[36969]
Setting environmental variable ELASTIC_NETINFO:false in Elastic Agent pod will disable the netinfo.enabled option of add_host_metadata processor

*Auditbeat*

Expand All @@ -204,27 +163,6 @@ is collected by it.
- Add MySQL authentication message parsing and `related.ip` and `related.user` fields {pull}34810[34810]
- Add nginx ingress_controller parsing if one of upstreams fails to return response {pull}34787[34787]
- Add oracle authentication messages parsing {pull}35127[35127]
- Add sanitization capabilities to azure-eventhub input {pull}34874[34874]
- Add support for CRC validation in Filebeat's HTTP endpoint input. {pull}35204[35204]
- Add support for CRC validation in Zoom module. {pull}35604[35604]
- Add execution budget to CEL input. {pull}35409[35409]
- Add XML decoding support to HTTPJSON. {issue}34438[34438] {pull}35235[35235]
- Add delegated account support when using Google ADC in `httpjson` input. {pull}35507[35507]
- Allow specifying since when to read journald entries. {pull}35408[35408]
- Add metrics for filestream input. {pull}35529[35529]
- Add support for collecting `httpjson` metrics. {pull}35392[35392]
- Add XML decoding support to CEL. {issue}34438[34438] {pull}35372[35372]
- Mark CEL input as GA. {pull}35559[35559]
- Add metrics for gcp-pubsub input. {pull}35614[35614]
- [GCS] Added scheduler debug logs and improved the context passing mechanism by removing them from struct params and passing them as function arguments. {pull}35674[35674]
- Allow non-AWS endpoints for awss3 input. {issue}35496[35496] {pull}35520[35520]
- Under elastic-agent the input metrics will now be included in agent diagnostics dumps. {pull}35798[35798]
- Add Okta input package for entity analytics. {pull}35611[35611]
- Expose harvester metrics from filestream input {pull}35835[35835] {issue}33771[33771]
- Add device support for Azure AD entity analytics. {pull}35807[35807]
- Improve CEL input performance. {pull}35915[35915]
- Adding filename details from zip to response for httpjson {issue}33952[33952] {pull}34044[34044]
- Added support for min/max template functions in httpjson input. {issue}36094[36094] {pull}36036[36036]
- Add `clean_session` configuration setting for MQTT input. {pull}35806[16204]
- Add fingerprint mode for the filestream scanner and new file identity based on it {issue}34419[34419] {pull}35734[35734]
- Add file system metadata to events ingested via filestream {issue}35801[35801] {pull}36065[36065]
Expand Down Expand Up @@ -262,6 +200,7 @@ is collected by it.
- Add support for Digest Authentication to CEL input. {issue}35514[35514] {pull}36932[36932]
- Use filestream input with file_identity.fingerprint as default for hints autodiscover. {issue}35984[35984] {pull}36950[36950]
- Add network processor in addition to interface based direction resolution. {pull}37023[37023]
- Add setup option `--force-enable-module-filesets`, that will act as if all filesets have been enabled in a module during setup. {issue}30915[30915] {pull}99999[99999]

*Auditbeat*

Expand Down Expand Up @@ -291,8 +230,6 @@ is collected by it.

*Packetbeat*

- Improve efficiency of sniffers by deduplicating interface configurations. {issue}36574[36574] {pull}36576[36576]
- Bump Windows Npcap version to v1.76. {issue}36539[36539] {pull}36549[36549]

*Packetbeat*

Expand Down Expand Up @@ -364,6 +301,9 @@ is collected by it.









Expand Down
1 change: 1 addition & 0 deletions libbeat/docs/release.asciidoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ This section summarizes the changes in each release. Also read
<<breaking-changes>> for more detail about changes that affect
upgrade.

* <<release-notes-8.11.0>>
* <<release-notes-8.10.4>>
* <<release-notes-8.10.3>>
* <<release-notes-8.10.2>>
Expand Down

0 comments on commit 3d5ba42

Please sign in to comment.