Releases · zeek/zeek

Breaking Changes

The OpaqueVal::DoSerialize and OpaqueVal::DoUnserialize methods were
marked as deprecated in v7.0 and have now been removed as per the Zeek
deprecation policy. Plugins that were overriding these methods and were not
updated will fail to compile. Those plugins should be updated to override the
new OpaqueVal::DoSerializeData and OpaqueVal::DoUnserializeData
methods.
Certain internal methods on the broker and logging classes have been changed to
accept std::vectorthreading::Value parameters instead of threading::Value**
to leverage automatic memory management, reduce the number of allocations
and use move semantics to express ownership.

The DoWrite() and HookLogWrite() methods which can be provided by plugins
are not affected by this change, so we keep backwards compatibility with
existing log writers.
Func::Name() was deprecated, use Func::GetName() instead.

New Functionality

The following dependencies have had updates:
- The bundled version of Spicy was updated to 1.12.0. See
  https://github.com/zeek/spicy/releases/tag/v1.12.0 for notes on what's new
  with Spicy.
- The bundled version of c-ares has been updated to v1.34.2, which required
  some updates to Zeek's internal DNS resolver due to changes in the c-ares
  API. At least version v1.28.0 is now required to build Zeek.
- Python 3.9 is now required for Zeek and all of it's associated subprojects.
IP-based connections that were previously not logged due to using an unknown
IP protocol (e.g. not TCP, UDP, or ICMP) now appear in conn.log. All conn.log
entries have a new ip_proto column that indicates the numeric IP protocol
identifier used by the connection. A new policy script at
policy/protocols/conn/ip-proto-name-logging.zeek can be loaded to also add
an ip_proto_name column with a string version of the ip_proto value.
This entire feature can be disabled by loading the new
policy/protocols/conn/disable-unknown-ip-proto-support.zeek policy script.
New Cluster::publish(), Cluster::subscribe() and Cluster::unsubscribe()
functions have been added. In contrast to their Broker counterparts, these
will operator on whichever cluster backend is enabled. Going forward, in-tree
Broker::publish() usages will be replaced with Cluster::publish() and
script writers should opt to prefer these over the Broker-specific functions.
Zeek now includes a PostgreSQL protocol analyzer. This analyzer is enabled
by default. The analyzer's events and its postgresql.log should be
considered preliminary and experimental until the arrival of Zeek's next
long-term-stable release (8.0).

If you observe unusually high CPU consumption or other issues due to this
analyzer being enabled by default, the easiest way to disable it is via the
Analyzer::disabled_analyzers const as follows:

redef Analyzer::disabled_analyzers += {
Analyzer::ANALYZER_POSTGRESQL,
};

If you observe PostgreSQL traffic in your environment, please provide feedback
about the analyzer and structure of the new log.
Broker's message I/O buffering now operates on per-peering granularity at the
sender (it was previously global) and provides configurable overflow handling
when a fast sender overwhelms a slow receiver, via the following new tunables
in the Broker module:

const peer_buffer_size = 2048 &redef;
const peer_overflow_policy = "disconnect" &redef;
const web_socket_buffer_size = 512 &redef;
const web_socket_overflow_policy = "disconnect" &redef;

When a send buffer overflows (i.e., it is full when a node tries to transmit
another message), the sender may drop the message and unpeer the slow receiver
(policy disconnect, the default), drop the newest message in the buffer
(drop_newest), or drop the oldest (drop_oldest). Buffer sizes are
measured in number of messages, not bytes. Note that "sender" and "receiver"
are independent of the direction in which Zeek established the peering. After
disconnects Zeek automatically tries to re-establish peering with the slow
node, in case it recovers.

Zeek notifies you in two ways of such disconnects:
- A cluster.log entry for the sending node indicates that a slow peered node
  has been removed. Here node worker01 has removed a peered ``proxy01`:
1733468802.626622 worker01 removed due to backpressure overflow: 127.0.0.1:42204/tcp (proxy01)
- The labeled counter metric zeek_broker_backpressure_disconnects_total
  in the telemetry framework tracks the number of times such disconnects
  happen between respective nodes. The following scraped telemetry indicates
  the same disconnect as above:
zeek_broker_backpressure_disconnects_total{endpoint="worker01",peer="proxy01"} 1

To implement custom handling of a backpressure-induced disconnect, add a
Broker::peer_removed event handler, as follows:

event Broker::peer_removed(endpoint: Broker::EndpointInfo, msg: string)
{
if ( "caf::sec::backpressure_overflow" !in msg )
return;
```
  # The local node has disconnected the given endpoint,
  # add your logic here.
          }
```
These new policies fix a problem in which misbehaving nodes could trigger
cascading "lockups" of nodes, each ceasing to transmit any messages.
The LDAP analyzer now supports handling of non-sealed GSS-API WRAP tokens.
StartTLS support was added to the LDAP analyzer. The SSL analyzer is enabled
for connections where client and server negotiate to TLS through the extended
request/response mechanism.
The unknown_protocols() event now includes the name of all packet
analyzer used for processing the packet when the event is raised. The
unknown_protocol.log file was extended to include this information.
The MySQL analyzer now generates a mysql_user_change() event when the user
changes mid-session via the COM_USER_CHANGE command.
The DNS analyzer was extended to support TKEY RRs (RFC 2390). A corresponding
dns_TKEY event was added.
The signature_match() and custom signature events now receive the end of
match offset within the data parameter as an optional parameter named
end_of_match.

event signature_match(state: signature_state, msg: string, data: string, end_of_match: count);
A new plugin hook InitPreExecution() has been added to allow introspection
of Zeek's AST after ZAM optimizations ran. This hook executes right before
the zeek_init() event is enqueued.
The SQLite logger now supports setting the value of the SQLite synchronous mode,
as well as of the journal mode. For example, WAL mode can be enabled by setting:

redef LogSQLite::journal_mode=LogSQLite::SQLITE_JOURNAL_MODE_WAL;
A pseudo protocol analyzer StreamEvent has been added. Attaching this analyzer
to TCP connections allows processing the connection's stream data in the
scripting layer. One example use-case is interactive terminal sessions over
HTTP connections upgraded to TCP.

redef HTTP::upgrade_analyzers += {
["tcp"] = Analyzer::ANALYZER_STREAM_EVENT,
};

event stream_deliver(c: connection, is_orig: bool, data: string);

This comes with performance caveats: For use-cases with high-data rates
a native protocol analyzer with dedicated events will be far more efficient.
Experimental support for pluggable cluster backends has been added. New plugin
components have been introduced to support switching Zeek's Broker-based
publish-subscribe and remote logging functionality to alternative implementations.

redef Cluster::backend = Cluster::CLUSTER_BACKEND_ZEROMQ;

Besides the backend, the serialization format used for events and log-writes
has become pluggable as well.
The Zeek distribution now includes an experimental ZeroMQ based cluster backend.
To experiment with it, load the following script on each cluster node.

@load frameworks/cluster/backend/zeromq/connect

Note that Broker-dependent scripts or integrations will become non-functional
when doing so as Zeek nodes will not listen on Broker ports anymore, nor will
they establish a peering to other nodes.
Zeek now ships with an experimental Spicy-based SSL analyzer, which is
disabled by default. This analyzer can be enabled using the
--enable-spicy-ssl conifgure-time option. The Spicy-based analyzer has
full support for SSL and TLS, just like the current binpac analyzer. It does,
however, not support any version of DTLS. Enabling it will disable DTLS
parsing in Zeek.

The analyzer is currently mostly interesting if you want to experiment with
SSL; we do not yet recommend to enable it in normal Zeek deployments.
The majority of the metrics reported via stats.log are also now reported via
the Telemetry framework, and are visible in the output passed to Prometheus.
A new weird DNS_unknown_opcode was added to the DNS analyzer to report
when it receives opcodes that it cannot process.

Changed Functionality

Heuristics for parsing SASL encrypted and signed LDAP traffic have been
made more strict and predictable. Please provide input if this results in
less visibility in your environment.
The MySQL analyzer has been improved to better support plugin authentication
mechanisms, like caching_sha2_password, as well as recognizing MySQL query
attributes.
The mysql.log for user change commands will contain just the username
instead of the remaining parts of the command, including auth plugin data.
The POP3 parser has been hardened to avoid unbounded state growth in the
face of one-sided traffic capture or when enab...

This release fixes the following bugs:

The community-id-logging.zeek policy script was used to set
c$conn$community_id during new_connection() rather than
connection_state_remove(), allowing other scripts to reuse its value
early.
The input framework will no longer get stuck and use 100% of the CPU when
encountering lines not immediately terminated by a new line.
The Modbus analyzer added some additional protocol checks and should no longer
over-match on traffic that's not specifically on port 502.
ZeekJS was updated to version v0.13.2, which brings support for newer versions
of Node.js and a fix for a segfault when running under Alpine.
A minor bug was fixed in the detect-sqli policy script to handle spaces being
encoded as plus signs.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Breaking Changes

New Functionality

Changed Functionality

Contributors

Releases: zeek/zeek

v7.1.0-rc1

Breaking Changes

New Functionality

Changed Functionality

Contributors

v7.0.5

v7.0.4

v6.0.9

v7.0.3

v6.0.8

v6.0.7

v7.0.2

v7.0.1

v6.0.6