v2.7.0

21st October 2021

Log Courier / Log Carver

• Added last_error and last_error_time to lc-admin for endpoints, so that the last error can be inspected
• Added the negotiated TLS version to connection messages and added additional logging where a remote does not support protocol handshakes
• Improved to random transport method so that a failed endpoint remains active and retrying until the switch happens, allowing it's status and last error to be seen in lc-admin instead of endpoints: none
• Improved moving average speed calculations
• Fixed panic in test transport when a payload containing a single event is encountered
• Fixed startup hang if the pipeline fails to start, for example when a port is already in use

Log Carver

• Added max pending payloads configuration to receiver section, to ensure clients cannot DoS Log Carver
• Fix a connection failing during attempt to gracefully shut it down
• Fix a possible deadlock in receiver shutdown due to late acknowledgements for a failed connection during shutdown

lc-admin

• Breaking Change: Removed the prompt when lc-admin is run without arguments and replaced it with an interactive console
• Added screens for monitoring the prospector, receiver and publisher, which refresh every second
• Note that scrolling is not yet implemented and so a larger terminal screen may be required to see all data for busy instances

Logstash Input Plugin

• Breaking Change: Obsoleted and removed the zmq transport option
• Updated dependencies to newer versions
• Added min_tls_version configuration option that now defaults to 1.2 (#357)
• Added protocol handshake support to output version of connecting clients
• Added new log messages to output the negotiated TLS version of each connection and, where a handshake occurs, the remote's product and version

v2.6.4

14th October 2021

Log Courier / Log Carver

• Simplified networking logic and fixed some deadlocks in publisher and scheduler
• Improved logging of transports, receivers, endpoints and publisher
• Faster TCP/TLS shutdown if the transport is an unusable state
• The name and version of the remote is now logged for new connections as part of the HELO/VERS handshake

Log Carver

• Implemented controlled shutdown of log-carver's log-courier connections to ensure all received events are acknowledged, so that log-courier does not resend any events already sent to the transport when it reconnects
• Added additional timeouts to ensure that all dead connections to log-carver are detected and closed
• Fixed shutdown hanging forever if an ES transport is unable to retrieve node information

v2.6.3

20th September 2021

Log Courier

• Fix debug level logging outputting spurious messages (#385)
• Fix syslog entries progname to only have the binary name and not the full path (#384)
• Fix hold time settings not closing files properly and causing a notice every 10 seconds (#382)
• Fix a deadlock in spooler if the pipeline completely stopped

Log Carver

• Fix syslog entries progname to only have the binary name and not the full path (#384)
• Fixed missing home directory on RPM installations (it was unused but caused unnecessary warnings in some cases)
• Fix a deadlock in spooler if the pipeline completely stopped

v2.6.2

23rd March 2021

Log Carver

• Further improvements and fixes to if/else if/else conditional parsing
• Fix to grok action causing events with an empty field name
• Improved configuration error reporting

v2.6.1

23rd March 2021

Log Courier

• Fixes for panics when loading configuration

Log Carver

• Fixes for else conditionals reporting as configuration errors

v2.6.0

23rd March 2021

Log Courier

• Fix broken includes configuration that was broken in 2.5.0 and add preventative tests
• Added new reader configuration to files entries that defaults to "line"
• Added a "json" reader that can read JSON files containing objects without line separators or line endings and decode them into events

Log Carver

• Improved speed of root level field lookups in expressions

v2.5.6

17th February 2021

Log Courier

• Resolve crash with orphaned file processing

v2.5.5

9th February 2021

Log Courier

• Fix severe registrar corruption that prevented Log Courier from resuming files at the correct offset
• Fixed a crash when add offset field is set to false and enable ecs is set to true
• Implement hold time configuration option with a default of 96 hours. Log Courier will now, by default, only hold open deleted files for a maximum of 96 hours after deletion is detected, regardless of whether its contents finish processing. A warning is logged if data has been lost when the file closed. This ensures disks do not fill when the pipeline is blocked. 96 hours was chosen as the default to allow a minimum of a few days to detect and repair a pipeline issue, as some roll over configurations delete the file during the very first rollover to replace it with a compressed version.
• The dead time configuration will no longer be checked if the pipeline is completely blocked. Previously, it would be processed during complete pipeline blockage only, meaning a deleted file could be closed and data lost if the pipeline was completely blocked for the specified dead time period. This was unintended behaviour and would not trigger if the pipeline was extremely slow as the dead time would reset upon each successful read. Documentation has also been updated to clarify that dead time is not based on the modification time of the file, but the time of the last successful read when the pipeline is moving, however slow that may be.

v2.5.4

8th February 2021

Log Courier

• Resolve lc-admin regression with unix sockets - it now reports unix-socket instead of log-courier-address

v2.5.3

8th February 2021

Log Courier

• Fix lc-admin missing default configuration on RPM builds
• Fix lc-admin reporting unknown configuration options it should never be concerned with
• Fix lc-admin reporting log-courier-address as the address in errors