Releases: zeek/zeek
v7.0.0
We would like to thank the following people for their contributions to this
release: Christopher Knill (@cknill), Jan Grashöfer (@J-Gras), Martin van
Hensbergen (@mvhensbergen), Matti Bispham (@mbispham), Mike Dopheide
(@dopheide-esnet), Oleksandr Pastushkov (@opastushkov), Peter Cullen (@pbcullen),
Steve Smoot (@stevesmoot), Tanner Kvarfordt (@Kardbord), Victor Dvornikov
(@lydiym).
Breaking Changes
-
The Telemetry framework has had a major rework, and includes a number of
breaking changes. The biggest change is a move towards a Prometheus-first
model. This removes the internal aggregation of metrics from nodes onto the
manager node, replacing it with a Prometheus service discovery endpoint. The
usage of this endpoint is described in the updated documentation for the
Telemetry framework. Using this endpoint also requires adding a new
metrics_port
field to each node in the cluster configuration, denoting
what port to connect to for each node.All of the metrics-related script-level options, type, and methods have been
moved to the Telemetry framework:- Option
Broker::metrics_port
is nowTelemetry::metrics_port
- Option
Broker::metrics_export_endpoint_name
is nowTelemetry::metrics_endpoint_name
The following options have been removed:
Broker::metrics_export_interval
Broker::metrics_export_topic
Broker::metrics_import_topics
Broker::metrics_export_prefixes
The
unit
field has been removed from the telemetry log.All of the
BROKER_METRICS_*
environment variables have been removed. Use
the equivalent script-level variables for setting configuration settings. For
cluster settings, a newmetrics_port
option was added to thenode.cfg
options and can be used to set the port number. The cluster layout record
defined incluster-layout.zeek
has the same option, and it can be
automatically populated byzeekctl
during theinstall
ordeploy
commands.The instruments that previously supported
count
in scripts andint64_t
in C++ were removed in favor of only providingdouble
versions. Prometheus
only handlesdouble
underneath the Zeek code, so it makes sense to only
support it directly in Zeek as well. This also simplifies the code
significantly.The
is_sum
argument has been removed from the constructors/creation
methods for all of the instruments. This again follows how Prometheus works,
wherecounter
instruments are always considered sums andgauge
instruments are not.Histogram
instruments don't have the concept of
summing. - Option
-
Zeekctl now sets
FileExtract::prefix
tospool/extract_files/<node>
to avoid
deletion of extracted files when stopping worker nodes. To revert to the
previous behavior, setFileExtractDir
to an empty string inzeekctl.cfg
.If you never enabled Zeek's file extraction functionality, there's no impact.
New Functionality
-
Support
delete
on tables, sets and vectors to clear their contents.global v = vector(1, 2, 3);
delete v;
assert |v| == 0; -
A new helper function
can_load()
backed by a new biffind_in_zeekpath()
was added to determine if a non-relative@load
directive might work. This
can be used to guard@load
directives when script packages may or may not
be installed. -
Zeek packagers can now include a "local" addition into Zeek's version string.
Configure your Zeek build with--localversion=XXX
to add the provided
string, dash-separated, at the end of Zeek's regular version string. The build
setup will refuse strings that contain dashes, to avoid confusing the version
components. For debug builds, the-debug
suffix remains at the end. For
example, the version string on a debug-enabled build with local addition "XXX"
might be "7.0.0-dev.114-XXX-debug". For Docker builds, theLOCALVERSION
environment variable configures the addition. -
SMB2 packets containing multiple PDUs now correctly parse all of the headers,
instead of just the first one and ignoring the rest. This may cause increased
CPU load on SMB2-heavy networks. -
The new built-in function
lookup_connection_analyzer_id()
retrieves the
numeric identifier of an analyzer associated with a connection. This enables
the use of thedisable_analyzer()
BiF outside of the analyzer
confirmation/violation events that have so far been the only providers of
those identifiers. For example, this allows the suppression of an analyzer
from the outset for specific connections:event connection_established(c: connection):
{
if ( no_http_for_this_conn_wanted(c) )
{
local aid = lookup_connection_analyzer_id(c$id, Analyzer::ANALYZER_HTTP);
if ( aid > 0 )
disable_analyzer(c$id, aid, T, T);
}
}Use
Analyzer::get_tag()
if you need to obtain an analyzer's tag from its
name (such as "HTTP"). -
The
from_json()
function now supports ingesting JSON representations of
tables as produced by theto_json()
function. It now also supports reading
the object-based representation of ports thatto_json()
generates for that
Zeek type. -
The
analyzer.log
now optionally supports logging of disabled analyzers
through the new optionAnalyzer::logging::include_disabling
.
Changed Functionality
-
The
ftp.log
fuid field is now cleared after handling a command with a fuid
associated with it. Previously, fuid was sticky and any subsequent FTP command
would reproduce the same fuid, even if the command itself did not result in
a file transfer over a data connection (e.g., CWD, DEL, PASV, SIZE). -
The type_name field populated by
global_ids()
now aligns with the value
returned bytype_name()
for each identifier. E.g,Site::local_nets
has a type_name ofset[subnet]
rather thantable
. -
The ISO 9660 file signature has been moved into the policy directory. The
signature has previously been non-functional due to implicit anchoring. Further,
this signature requires users to significantly increase their
default_file_bof_buffer_size
. Users can now enable this signature by loading
frameworks/signatures/iso-9660
which also increases the BOF buffer sufficiently.
Note, doing so may increase memory and CPU usage significantly. -
The
val_footprint()
BiF now factors in the size of strings when reporting
footprints, roughly equating a string's size with the number of elements
comparable to that length. As before, footprints are not meant to be precise
but mainly for providing comparisons, which is why this is not a breaking
change. -
The tuning/defaults policy has been deprecated and will be removed in
v7.1. This policy was already being loaded by default via local.zeek. The
settings contained within have become the overall defaults for Zeek now,
instead of having to load the policy. The two changes here are that fragments
now timeout after 5 minutes by default instead of no timeout, and extracted
files now have a default size limit of 100MB instead of unlimited. -
If a Spicy protocol analyzers feeds data into file analysis, it now
needs to call Zeek'sFiles::register_protocol()
and provide a
callback for computing file handles. If that's missing, Zeek will
issue a warning. While this was not necessary in previous versions,
it aligns with the same requirement for traditional analyzers and
enables customizing file handles for protocol-specific semantics. -
The Supervisor's API now returns NodeConfig records with a cluster table whose
ClusterEndpoints have a port value of 0/unknown, rather than 0/tcp, to
indicate that the node in question has no listening port.
Removed Functionality
Deprecated Functionality
-
The
--disable-archiver
configure flag no longer does anything and will be
removed in 7.1. zeek-archiver has moved into the zeek-aux repository. -
The policy/frameworks/telemetry/prometheus.zeek script has been deprecated
and will be removed with Zeek 7.1. Setting themetrics_port
field on a
Cluster::Node
implies listening on that port and exposing telemetry
in Prometheus format.
v6.0.5
This release fixes the following bugs:
-
The Mozilla CA and Google CT lists were updated to their latest versions.
-
Connection IDs now correctly propogate into files.log.
-
A rare crash in CAF that happened when shutting down Zeek was resolved.
-
Binary addresses passed to Zeekctl were previously assumed to be valid
unicode, which wasn't always the case. Some additional checking was added to
ensure that's the case and to provide better error messaging when it isn't.
v6.2.1
-
A crash with ICMP packets involving errant length checking was fixed. Thank
you to Ronny Barkan from Microsoft for reporting this issue. -
When a shadow file is empty/missing during rotation, Zeek aborts with an error
message, but if the shadow file was empty, it will still be there after the
restart. This results in an endless restart loop. This has been corrected to
overwrite the existing shadow file, using the default file extension and
post processing function. -
A new function
remove_exclude
was added to thePacketFilter
framework
which can be used to remove a previously added exclude filter by name. -
A new option
--localversion
was added to theconfigure
script. This
option allows a caller to add custom strings to the end of the Zeek version
reported byzeek -v
.
v6.0.4
This release fixes the following bugs:
-
The Mozilla CA and Google CT lists were updated to their latest versions.
-
A crash with ICMP packets involving errant length checking was fixed. Thank
you to Ronny Barkan from Microsoft for reporting this issue. -
When a shadow file is empty/missing during rotation, Zeek aborts with an error
message, but if the shadow file was empty, it will still be there after the
restart. This results in an endless restart loop. This has been corrected to
overwrite the existing shadow file, using the default file extension and
post processing function. -
A new function
remove_exclude
was added to thePacketFilter
framework
which can be used to remove a previously added exclude filter by name. -
A new option
--localversion
was added to theconfigure
script. This
option allows a caller to add custom strings to the end of the Zeek version
reported byzeek -v
. -
The highwayhash submodule was updated to fix a build issue for FreeBSD. Thank
you to community member Craig Leres for reporting this.
v6.2.0
We would like to thank Anthony Verez (netantho), Bijoy Das (mute019), Jan
Grashöfer (J-Gras), Matti Bispham (mbispham), Phil Rzewski (philrz), and
xb-anssi for their contributions to this release.
Breaking Changes
-
The methods
Dispatcher::Lookup()
andAnalyzer::Lookup()
in the packet_analysis
namespace were changed to return a reference to a std::shared_ptr instead of a copy
for performance reasons. -
Zeek's
OPENSSL_INCLUDE_DIR
is not automatically added to an external plugin's
include path anymore. A plugin using OpenSSL functionality directly can use the
following explicit entry to re-use Zeek'sOPENSSL_INCLUDE_DIR
:zeek_add_plugin(
Namespace Name
INCLUDE_DIRS "${OPENSSL_INCLUDE_DIR}"
SOURCES ...
) -
The "segment_profiling" functionality and
load_sample
event have been removed
without deprecation. This functionality was unmaintained and not known to be used. -
Certain
ldap.log
andldap_search.log
fields have been renamed from
plural to singular and their types changed to scalars. This maps better onto
the expected request-response protocol used between client and server. Additionally,
it removes the burden of working with non-scalar columns from downstream systems.Specifically, for
ldap.log
:arguments: vector of string
is nowargument: string
diagnostic_messages: vector of string
is nowdiagnostic_message: string
objects: vector of string
is nowobject: string
opcodes: set[string]
is nowopcode: string
results: set[string]
is nowresult: string
For
ldap_search.log
, the following fields were changed:base_objects: vector of string
is nowbase_object: string
derefs: set[string]
is nowderef_aliases: string
diagnostic_messages: vector of string
is nowdiagnostic_message: string
results: set[string]
is nowresult: string
scopes: set[string]
is nowscope: string
In the unlikely scenario that a request-response pair with the same message
identifier is observed, containing different values for certain fields, new
weirds are raised and will appear inweird.log
, including the old and new
values as well as the LDAP message identifier. The value within the LDAP logs
will be the most recently observed one. -
BIF methods now return a
ValPtr
directly instead of aBifReturnVal
object
which was just a thin wrapper aroundValPtr
. This may cause compilation errors
in C++ code that was calling BIF methods directly.
New Functionality
-
The table type was extended to allow parallel regular expression matching
when a table's index is a pattern. Indexing such tables yields a vector
containing all values of matching patterns for keys of type string.As an example, the following snippet outputs
[a, a or b], [a or b]
.global tbl: table[pattern] of string;
tbl[/a/] = "a";
tbl[/a|b/] = "a or b";
tbl[/c/] = "c";
print tbl["a"], tbl["b"];Depending on the patterns and input used for matching, memory growth may
be observed over time as the underlying DFA is constructed lazily. Users are
advised to test with realistic and adversarial input data with focus on
memory growth. The DFA's state can be reset by removal/addition of a single
pattern. For observability, a new biftable_pattern_matcher_stats()
can be used to gatherMatcherStats
. -
Support for delaying log writes.
The logging framework offers two new functions
Log::delay()
andLog::delay_finish()
to delay aLog::write()
operation. This new functionality allows delaying of
a specific log record within the logging pipeline for a variable but bounded
amount of time. This can be used, for example, to query and wait for additional
information to attach to the pending record, or even change its final verdict.Conceptually, delaying a log record happens after the execution of the global
Log::log_stream_policy
hook for a givenLog::write()
and before the
execution of filter policy hooks. Any mutation of the log record within the
delay period will be visible to filter policy hooks. CallingLog::delay()
is currently only allowed within the context of theLog::log_stream_policy
hook
for the activeLog::write()` operation (or during the execution of post delay callbacks). While this may appear restrictive, it makes it explicit which
Log::write()``
operation is subject to the delay.Interactions, semantics and conflicts of this feature when writing the same
log record multiple times to the same or different log streams need to be taken
into consideration by script writers.Given this is the first iteration of this feature, feedback around usability and
use-cases that aren't covered are more than welcome. -
A WebSocket analyzer has been added together with a new
websocket.log
.The WebSocket analyzer is instantiated when a WebSocket handshake over HTTP is
recognized. By default, the payload of WebSocket messages is fed into Zeek's dynamic
protocol detection framework, possibly discovering and analyzing tunneled protocols.The format of the log and the event semantics should be considered preliminary until
the arrival of the next long-term-stable release (7.0).To disable the analyzer in case of fatal errors or unexpected resource usage,
use theAnalyzer::disabled_analyzers
pattern:redef Analyzer::disabled_analyzers += {
Analyzer::ANALYZER_WEBSOCKET,
}; -
The SMTP analyzer was extended to recognize and properly handle the BDAT command
from RFC 3030. This improves visibility into the SMTP protocol when mail agents
and servers support and use this extension. -
The event keyword in signatures was extended to support choosing a custom event
to raise instead ofsignature_match()
. This can be more efficient in certain
scenarios compared to funneling every match through a single event.The new syntax is to put the name of the event before the string used for the
msg
argument. As an extension, it is possible to only provide an event name,
skippingmsg
. In this case, the framework expects the event's parameters to
consist of only state and data as follows:signature only-event {
payload /.*root/
event found_root
}event found_root(state: signature_state, data: string) { }
Using the
msg
parameter with a custom event looks as follows. The custom
event's parameters need to align with those for ``signature_match()` event:signature event-with-msg {
payload /.*root/
event found_root_with_msg "the-message"
}event found_root_with_msg(state: signature_state, msg: string, data: string) { }
Note, the message argument can currently still be specified as a Zeek identifier
referring to a script-level string value. If used, this is disambiguated behind
the scenes for the first variant. Specifyingmsg
as a Zeek identifier has
been deprecated with the new event support and will be removed in the future.Note that matches for signatures with custom events will not be recorded in
signatures.log
. This log is based on the generation ofsignature_match()
events. -
The QUIC analyzer has been extended to support analyzing QUIC Version 2
INITIAL packets (RFC 9369). Additionally, prior draft and some of
Facebook's mvfst versions are supported. Unknown QUIC versions will now be
reported inquic.log
as an entry with aU
history field. -
Conditional directives (
@if
,@ifdef
,@ifndef
,@else
and
@endif
) can now be placed within a record's definition to conditionally
define or extend a record type's fields.type r: record { c: count; @if ( cond ) d: double; @else d: count; @endif };
Note that generally you should prefer record extension in conditionally loaded
scripts rather than using conditional directives in the original record definition. -
The 'X' code can now appear in a connection's history. It is meant to indicate
situations where Zeek stopped analyzing traffic due to exceeding certain limits or
when encountering unknown/unsupported protocols. Its first use is to indicate
Tunnel::max_depth
being exceeded. -
A new
Intel::seen_policy
hook has been introduced to allow intercepting
and changing ``Intel::seen` behavior:hook Intel::seen_policy(s: Intel::Seen, found: bool)
-
A new
NetControl::rule_added_policy
hook has been introduced to allow modification
of NetControl rules after they have been added. -
The IP geolocation / ASN lookup features in the script layer provide better
configurability. The file names of MaxMind databases are now configurable via
the newmmdb_city_db
,mmdb_country_db
, andmmdb_asn_db
constants,
and the previously hardwired fallback search path when not using an
mmdb_dir
value is now adjustable via themmdb_dir_fallbacks
vector. Databases opened explicitly via themmdb_open_location_db
and
mmdb_open_asn_db
functions now behave more predictably when updated or
removed. For details, see:
https://docs.zeek.org/en/master/customizations.html#address-geolocation-and-as-lookups -
The
zeek-config
script now provides a set of--have-XXX
checks for
features optionally compiled in. Each check reports "yes"/"no" to stdout and
exits with 0/1, respectively.
Changed Functionality
-
The
split_string
family of functions now respect the beginning-of-line ^ and
end-of-line $ anchors. Previously, an anchored pattern would be matched anywhere
in the input string. -
The
sub()
and ``gsub()` functions now respect the beginning-of-l...
v6.1.1
This release fixes the following security issues:
- A specially-crafted series of packets containing nested MIME entities can
cause Zeek to spend large amounts of time parsing the entities. Due to the
possibility of receiving these packets from remote hosts, this is a DoS
risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
that limits the depth the parser will attempt to follow the entity nesting. If
the limit is reached an exceeded_mime_max_depth weird is generated.
This release fixes the following bugs:
-
CMake correctly passes along third-party package information when building
plugins. This ensures that, for example, the same paths to OpenSSL used in a
Zeek build are provided to a plugin build. -
Fix a problem with the HTTP analyzer where a signature regex ending in '$'
used to match against 'http-request-body' or 'http-reply-bdoy' will never
succeed. Thank you to GitHub user xb-anssi for this fix. -
The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.
-
The SMB::State$recent_files field was not correctly expiring entries, leading
to unbounded state growth. This is fixed to correctly follow the &read_expire
condition on the field. Thank you to Slack user ya-sato for reporting this. -
The &create_expire attribute is now kept valid after clearing a table. After
switching the known scripts away from broker stores, the &create_expire value
of the local tables/sets of the known scripts wasn't in effect due to
Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
the &create_expire attribute intact. This broke the "log hosts every 24h"
behavior. -
Zeek builds using the --binary-package argument and including Spicy will now
include all necessary Spicy symbols.
v6.0.3
This release fixes the following security issues:
- A specially-crafted series of packets containing nested MIME entities can
cause Zeek to spend large amounts of time parsing the entities. Due to the
possibility of receiving these packets from remote hosts, this is a DoS
risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
that limits the depth the parser will attempt to follow the entity nesting. If
the limit is reached an exceeded_mime_max_depth weird is generated.
This release fixes the following bugs:
-
CMake correctly passes along third-party package information when building
plugins. This ensures that, for example, the same paths to OpenSSL used in a
Zeek build are provided to a plugin build. -
Fix a problem with the HTTP analyzer where a signature regex ending in '$'
used to match against 'http-request-body' or 'http-reply-bdoy' will never
succeed. Thank you to GitHub user xb-anssi for this fix. -
The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.
-
The SMB::State$recent_files field was not correctly expiring entries, leading
to unbounded state growth. This is fixed to correctly follow the &read_expire
condition on the field. Thank you to Slack user ya-sato for reporting this. -
The &create_expire attribute is now kept valid after clearing a table. After
switching the known scripts away from broker stores, the &create_expire value
of the local tables/sets of the known scripts wasn't in effect due to
Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
the &create_expire attribute intact. This broke the "log hosts every 24h"
behavior. -
Zeek builds using the --binary-package argument and including Spicy will now
include all necessary Spicy symbols.
v6.1.0
Breaking Changes
-
assert
is now a reserved keyword for the newassert
statement. -
The
__bro_plugin__
file that gets generated as part of plugin builds was
renamed to__zeek_plugin__
. This will affect the ability for older
versions ofzkg
to use thezkg unload
andzkg load
commands. This
should only cause breakage for people using a version of ``zkg` that doesn't
come bundled with Zeek (which we generally don't recommend doing). -
Zeek does not traverse into dot directories to find plugins or hlto files
anymore. Any dot directories found below the directories specified in
ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are now skipped. Dot directories
explicitly listed in ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are not
skipped. -
External plugins will fail to configure if their minimum required CMake
version is below 3.15. This was a warning with Zeek 6.0, but has caused user
confusion due to unhelpful error messages around the IN_LIST operator policy. -
The FindBISON, FindOpenSSL, FindPackageHandleStandardArgs, FindPackageMessage,
and SelectLibraryConfigurations cmake files were removed from our cmake
repository in favor of the versions that come with CMake. This should not
cause any breakage, but it is possible in the case that someone was using
these in a plugin.
New Functionality
-
Zeek now includes the LDAP protocol analyzer from the zeek/spicy-ldap project
(https://github.com/zeek/spicy-ldap). This analyzer is enabled by default. The
analyzer's events and itsldap.log
andldap_search.log
should be
considered preliminary and experimental until the arrival of Zeek's next
long-term-stable release (7.0).If you observe unusually high CPU consumption or other issues due to this
analyzer being enabled by default, the easiest way to disable it is via the
Analyzer::disabled_analyzers
const as follows:redef Analyzer::disabled_analyzers += {
Analyzer::ANALYZER_LDAP_UDP,
Analyzer::ANALYZER_LDAP_TCP,
};Please do report issues to us including diagnostic information in case this is
necessary in your environment. We're also open to general feedback about the
structure of the new logs. -
Zeek now includes the QUIC protocol analyzer from the zeek/spicy-quic project
(https://github.com/zeek/spicy-quic). This project is a fork of Fox-IT's
initial implementation (https://github.com/fox-ds/spicy-quic).As for the LDAP analyzer, the analyzer's events and the new
quic.log
should be considered preliminary and experimental until the arrival of Zeek's
next long-term-stable release (7.0). As above, any feedback and contributions
to this analyzer and the new log are welcome.The analyzer's functionality is limited to decryption of the INITIAL packets
of QUIC version 1. If decryption of these packets is successful, the handshake
data is forwarded to Zeek's SSL analyzer. Anssl.log
entry will appear in
ssl.log
for QUIC connections. The entry in theconn.log
will contain
quic
andssl
in the service field.To disable the analyzer in case of issues, use the following snippet:
redef Analyzer::disabled_analyzers += {
Analyzer::ANALYZER_QUIC,
}; -
Added a new
assert
statement for assertion based testing and asserting
runtime state.assert <expr: bool>[, <message: string>];
This statement comes with two hooks. First,
assertion_failure()
that is
invoked for every failing assert statement. Second,assertion_result()
which is invoked for every assert statement and its outcome. The latter allows
to construct a summary of failing and passing assert statements. Both hooks
receive the location and call stack for theassert
statement via a
Backtrace
vector.A failing assert will abort execution of the current event handler similar to
scripting errors. By default, a reporter error message is logged. Using the
break statement withinassertion_failure()
orassertion_result()
allows to suppress the default message. -
Add a new
&default_insert
attribute for tables. This behaves as
&default
with the addition that the default value is inserted into the
table upon a failed lookup. Particularly for tables with nested container
values, the&default
behavior of not inserting the value can be of little
use. -
The
from_json()
function now takes an optional key_func argument to
normalize JSON object key names. This can be useful if the keys in a JSON
object are not valid Zeek identifiers or reserved keywords. -
Module names are now included in
global_ids()
. Their key in the returned
table is prefixed with "module " and their value will have thetype_name
field set to "module". -
Identifiers in the global scope can now be referenced and defined from within
modules by prefixing their names with::
. Previously, these required an
explicitGLOBAL::
prefix to be used. UsingGLOBAL::
has been
deprecated. -
The
as
keyword now supports casting betweenset
andvector
values
with the same element type. Convertingset
values with multiple index
values is not supported. We plan to extend the use of theas
keyword to
support more type conversions in the future. -
Added new packet analyzer to handle PCAP files DLT_PPP link type.
-
Fixed appending of
any
tovector of any
. -
The ModBus analyzer's function support was expanded, with new handling of the
Encapsulation Interface Transport (function 28) And Diagnostics (function 8)
functions. This adds newmodbus_encap_interface_transport_{request,response}
andmodbus_diagnostics_{request,response}
events. -
The ModBus file record read and write events now provide the full data from
the request and response messages as part of the event data. -
The full PDU length was added to the
ModBusHeader
record type passed with
all of the ModBus events.
Changed Functionality
-
A connection's value is now updated in-place when its directionality is
flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
protocol specific approaches). Previously, a connection's value was discarded
when flipped, including any values set in anew_connection()
handler. A
newconnection_flipped()
event is added to allow updating custom state in
script-land. -
Loading
policy/frameworks/notice/community-id.zeek
now also automatically
community ID logging. In the past, loading the script had no effect unless
policy/protocols/conn/community-id-logging.zeek
was loaded before. This
was fairly unusual and hard to debug behavior. -
Connections to broadcast addresses are not flipped based on
likely_server_ports
anymore. Previously, broadcast packets originating
from a likely server port resulted in 255.255.255.255 being the originator in
conn.log
. -
When too many HTTP requests are pending, Zeek will now log them at once and
reset request/response correlation instead of running into unbounded state
growth. This behavior is configurable via a new option
HTTP::max_pending_requests
. The default is100
. -
Fix deferred initialization of nested records containing non-const &default
attributes. -
Parameter lists for functions, events and hooks now use commas instead of
semicolons in error messages or when printing such functions. -
The IO buffer size used for PCAP file reading is now always 128kb. This new
default can be changed viaPcap::bufsize_offline_bytes
. -
The input framework now provides better information in error messages when
encountering missing non-optional field while loading data. -
The SSL analyzer will now parse a configurable maximum of 10 SSL Alerts per
SSL message. For TLS 1.3, the maximum is implicitly 1 as defined by RFC 8446.
If there are more alerts, a new weird "SSL_excessive_alerts_in_record" is raised.
For non-TLS 1.3, the maximum can be redefined viaSSL::max_alerts_per_record
. -
The
ssl_history
field in the ssl.log is now capped at a configurable
limit of 100 characters prevent unbounded growth. The limit can be changed
via the optionSSL::max_ssl_history_length
. When reached, a new weird
named "SSL_max_ssl_history_length_reached" is raised.
Deprecated Functionality
-
Accessing globals with
GLOBAL::name
has been deprecated and will be
removed with Zeek 7.1. Use::name
instead. -
The original
trigger::Trigger
constructor has been deprecated and will be
removed with Zeek 7.1. Use the new alternative constructor (per
src/Trigger.h
) instead, including replacing any use ofnew ...
with
make_intrusive<...>
. The new constructor differs only in the placement of
thetimeout
parameter, and in that - unlike the original - it always
returns a valid pointer, which must be Unref()'d after construction, either
explicitly (if usingnew
) or implicitly (if using
make_intrusive<...>
).
v6.0.2
This release fixes the following security issues:
-
A specially-crafted SSL packet could cause Zeek to leak memory and potentially
crash. Due to the possibility of receiving these packets from remote hosts,
this is a DoS risk. The fix included adds additional memory cleanup to the
x509 file analyzer. -
A specially-crafted series of FTP packets could cause Zeek to log entries for
requests that have already been completed, using resources unnecessarily and
potentially causing Zeek to lose other traffic. Due to the possibility of
receiving these packets from remote hosts, this is a DoS risk. The fix
included changes the way that we track the pending FTP commands, avoiding
possibly reusing the same value for subsequent commands. -
A specially-crafted series of SSL packets could cause Zeek to output a very
large number of unnecessary alerts for the same record. Due to the possibility
of receiving these packets from remote hosts, this is a DoS risk. The fix
included adds a new option SSL::max_alerts_per_record that caps the number of
alerts that can be generated for an SSL record. For TLS 1.3 this is capped at
1 as defined in RFC 8446. For non-TLS 1.3 it is a configurable value. A
SSL_excessive_alerts_in_record weird will be raised if the cap is exceeded. -
A specially-crafted series of SSL packets could cause Zeek to generate very
long ssl_history fields in the ssl.log, potentially using a large amount of
memory due to unbounded state growth. Due to the possibility of receiving
these packets from remote hosts, this is a DoS risk. The fix included adds a
new option SSL::max_ssl_history_length that caps this to 100 characters by
default. A SSL_max_ssl_history_length_reached weird will be raised if the cap
is exceeded. -
A specially-crafted IEEE802.11 packet could cause Zeek to overflow memory and
potentially crash. Due to the possibility of receiving these packets from
remote hosts, this is a DoS risk. The fix included adds additional bounds
checking to the IEEE802.11 packet analyzer.
This release fixes the following bugs:
-
Fixed Spicy type names from causing collisions with existing Zeek types.
-
On some systems with low values for the maximum number of file descriptors, it
was possible to run into crashes when doing DNS lookups if all of the file
descriptors were used. This is now avoided with better checking for the number
of available file descriptors before trying a lookup. Thank you to Zeek Slack
user h-mikami for reporting this issue. -
Tables backed by a Broker backend now correctly support deletion if they have
complex index types. Zeek previously reported an error when trying to delete
elements from these tables. -
A significant performance issue with Zeek's supervisor code was fixed,
revolving around the re-initialization of the Event Manager object used to
track events. Thank you to Jan Grashoefer for reporting this issue. -
The MaxMind DB code now cleans up after itself, resolving a memory leak with
the loaded database files. -
The ZeekJS submodule was updated to version 0.9.6, bringing fixes for
zeek.invoke and zeek.event crashes, garbage collection, and an issue where
Zeek may stop executing events from ZeekJS.
v6.0.1
This release fixes the following security issues:
-
File extraction limits were not correctly enforced for files containing large
amounts of missing bytes. Crafting files with large amounts of missing bytes
in them could cause Zeek to spend a long time processing data, allocate a lot
of main memory, and write a lot of data to disk. Due to the possibility of
receiving these packets from remote hosts, this is a DoS risk. The fix
included makes Zeek correctly enforce file size limits. We also added a new
option (default_limit_includes_missing) which allows to customize the behavior
when encountering large amounts of missed bytes in file. This issue was
discovered by Luca Cigarini. -
Sessions are sometimes not cleaned up completely within Zeek during shutdown,
potentially causing a crash when using the -B dpd flag for debug logging. This
is low priority because it only happens at shutdown and only when using that
flag. The fix included is to reorder shutdown to cleanup all the sessions
prior to tearing down the analyzers. -
A specially-crafted HTTP packet can cause Zeek’s filename extraction code to
take a long time to process the data. Due to the possibility of receiving
these packets from remote hosts, this is a DoS risk. The fix included adjusts
the regular expression used in the extract_filename_from_content_disposition()
script function to more-correctly match the data. -
A specially-crafted series of FTP packets made up of a CWD request followed by
a large amount of ERPT requests may cause Zeek to spend a long time logging
the commands. Due to the possibility of receiving these packets from remote
hosts, this is a DoS risk. The fix included is to prevent logging of pending
commands for FTP packets. -
A specially-crafted VLAN packet can cause Zeek to overflow memory and
potentially crash. Due to the possibility of receiving these packets from
remote hosts, this is a DoS risk. The fix included is to add some additional
length checking to the VLAN analyzer.
This release fixes the following bugs:
-
Fixed a base64 decoding issue with the authorization field of HTTP request
headers that was sometimes causing Zeek to output error messages. Thank you
to GitHub user @progmboy for reporting and providing a fix for this issue. -
Ensure that Zeek builds use the internal version of Spicy instead of external
installations, unless specifically configured for that mode. -
Support was added for
switch
fields when exporting Spicy types to Zeek. -
A number of fixes were added to protect against potential unbounded state
growth with the SMB and DCE-RPC analyzers. SMB close requests will properly
tear down an related DCE-RPC analyzers. A newSMB::max_dce_rpc_analyzers
script variable was added that allows finer control of how many DCE-RPC
analyzers are allowed to be created per SMB connection. Thanks to Zeek Slack
user Takomi Sugawara for reporting this issue. -
Fixed a regression in the UDP and TCP analyzers that was causing more data
than necessary to be forwarded to the next analyzer in the chain. Thanks to
Zeek Slack user Hiep Long Tan for reporting this issue. -
A connection's value is now updated in-place when its directionality is
flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
protocol specific approaches). Previously, a connection's value was discarded
when flipped, including any values set in anew_connection()
handler. A
newconnection_flipped()
event is added to allow updating custom state in
script-land. -
Fixed undefined symbols being reported from Spicy when building some of the
binary packages for Zeek. -
Loading
policy/frameworks/notice/community-id.zeek
now also automatically
community ID logging. In the past, loading the script had no effect unless
policy/protocols/conn/community-id-logging.zeek
was loaded before. This
was fairly unusual and hard to debug behavior. -
Spicy no longer registers an extra port for every port registered in a
plugin's .evt file. -
Timeouts in DNS resolution no longer cause uncontrolled memory growth.
-
Fix check to skip DNS hostname lookups for notices that are not delivered via
email inpolicy/frameworks/notice/extend-email/hostnames
. Due to that
policy script being loaded in the Zeek's defaultlocal.zeek
, this
previously caused unneeded DNS lookups for every generated notice instead of
just those delivered via email.