Skip to content

Releases: zeek/zeek

v7.0.0

11 Jul 21:09
Compare
Choose a tag to compare

We would like to thank the following people for their contributions to this
release: Christopher Knill (@cknill), Jan Grashöfer (@J-Gras), Martin van
Hensbergen (@mvhensbergen), Matti Bispham (@mbispham), Mike Dopheide
(@dopheide-esnet), Oleksandr Pastushkov (@opastushkov), Peter Cullen (@pbcullen),
Steve Smoot (@stevesmoot), Tanner Kvarfordt (@Kardbord), Victor Dvornikov
(@lydiym).

Breaking Changes

  • The Telemetry framework has had a major rework, and includes a number of
    breaking changes. The biggest change is a move towards a Prometheus-first
    model. This removes the internal aggregation of metrics from nodes onto the
    manager node, replacing it with a Prometheus service discovery endpoint. The
    usage of this endpoint is described in the updated documentation for the
    Telemetry framework. Using this endpoint also requires adding a new
    metrics_port field to each node in the cluster configuration, denoting
    what port to connect to for each node.

    All of the metrics-related script-level options, type, and methods have been
    moved to the Telemetry framework:

    • Option Broker::metrics_port is now Telemetry::metrics_port
    • Option Broker::metrics_export_endpoint_name is now Telemetry::metrics_endpoint_name

    The following options have been removed:

    • Broker::metrics_export_interval
    • Broker::metrics_export_topic
    • Broker::metrics_import_topics
    • Broker::metrics_export_prefixes

    The unit field has been removed from the telemetry log.

    All of the BROKER_METRICS_* environment variables have been removed. Use
    the equivalent script-level variables for setting configuration settings. For
    cluster settings, a new metrics_port option was added to the node.cfg
    options and can be used to set the port number. The cluster layout record
    defined in cluster-layout.zeek has the same option, and it can be
    automatically populated by zeekctl during the install or deploy
    commands.

    The instruments that previously supported count in scripts and int64_t
    in C++ were removed in favor of only providing double versions. Prometheus
    only handles double underneath the Zeek code, so it makes sense to only
    support it directly in Zeek as well. This also simplifies the code
    significantly.

    The is_sum argument has been removed from the constructors/creation
    methods for all of the instruments. This again follows how Prometheus works,
    where counter instruments are always considered sums and gauge
    instruments are not. Histogram instruments don't have the concept of
    summing.

  • Zeekctl now sets FileExtract::prefix to spool/extract_files/<node> to avoid
    deletion of extracted files when stopping worker nodes. To revert to the
    previous behavior, set FileExtractDir to an empty string in zeekctl.cfg.

    If you never enabled Zeek's file extraction functionality, there's no impact.

New Functionality

  • Support delete on tables, sets and vectors to clear their contents.

    global v = vector(1, 2, 3);
    delete v;
    assert |v| == 0;

  • A new helper function can_load() backed by a new bif find_in_zeekpath()
    was added to determine if a non-relative @load directive might work. This
    can be used to guard @load directives when script packages may or may not
    be installed.

    @if ( can_load("my-package") )
    @load my-package
    @endif

  • Zeek packagers can now include a "local" addition into Zeek's version string.
    Configure your Zeek build with --localversion=XXX to add the provided
    string, dash-separated, at the end of Zeek's regular version string. The build
    setup will refuse strings that contain dashes, to avoid confusing the version
    components. For debug builds, the -debug suffix remains at the end. For
    example, the version string on a debug-enabled build with local addition "XXX"
    might be "7.0.0-dev.114-XXX-debug". For Docker builds, the LOCALVERSION
    environment variable configures the addition.

  • SMB2 packets containing multiple PDUs now correctly parse all of the headers,
    instead of just the first one and ignoring the rest. This may cause increased
    CPU load on SMB2-heavy networks.

  • The new built-in function lookup_connection_analyzer_id() retrieves the
    numeric identifier of an analyzer associated with a connection. This enables
    the use of the disable_analyzer() BiF outside of the analyzer
    confirmation/violation events that have so far been the only providers of
    those identifiers. For example, this allows the suppression of an analyzer
    from the outset for specific connections:

    event connection_established(c: connection):
    {
    if ( no_http_for_this_conn_wanted(c) )
    {
    local aid = lookup_connection_analyzer_id(c$id, Analyzer::ANALYZER_HTTP);
    if ( aid > 0 )
    disable_analyzer(c$id, aid, T, T);
    }
    }

    Use Analyzer::get_tag() if you need to obtain an analyzer's tag from its
    name (such as "HTTP").

  • The from_json() function now supports ingesting JSON representations of
    tables as produced by the to_json() function. It now also supports reading
    the object-based representation of ports that to_json() generates for that
    Zeek type.

  • The analyzer.log now optionally supports logging of disabled analyzers
    through the new option Analyzer::logging::include_disabling.

Changed Functionality

  • The ftp.log fuid field is now cleared after handling a command with a fuid
    associated with it. Previously, fuid was sticky and any subsequent FTP command
    would reproduce the same fuid, even if the command itself did not result in
    a file transfer over a data connection (e.g., CWD, DEL, PASV, SIZE).

  • The type_name field populated by global_ids() now aligns with the value
    returned by type_name() for each identifier. E.g, Site::local_nets
    has a type_name of set[subnet] rather than table.

  • The ISO 9660 file signature has been moved into the policy directory. The
    signature has previously been non-functional due to implicit anchoring. Further,
    this signature requires users to significantly increase their
    default_file_bof_buffer_size. Users can now enable this signature by loading
    frameworks/signatures/iso-9660 which also increases the BOF buffer sufficiently.
    Note, doing so may increase memory and CPU usage significantly.

  • The val_footprint() BiF now factors in the size of strings when reporting
    footprints, roughly equating a string's size with the number of elements
    comparable to that length. As before, footprints are not meant to be precise
    but mainly for providing comparisons, which is why this is not a breaking
    change.

  • The tuning/defaults policy has been deprecated and will be removed in
    v7.1. This policy was already being loaded by default via local.zeek. The
    settings contained within have become the overall defaults for Zeek now,
    instead of having to load the policy. The two changes here are that fragments
    now timeout after 5 minutes by default instead of no timeout, and extracted
    files now have a default size limit of 100MB instead of unlimited.

  • If a Spicy protocol analyzers feeds data into file analysis, it now
    needs to call Zeek's Files::register_protocol() and provide a
    callback for computing file handles. If that's missing, Zeek will
    issue a warning. While this was not necessary in previous versions,
    it aligns with the same requirement for traditional analyzers and
    enables customizing file handles for protocol-specific semantics.

  • The Supervisor's API now returns NodeConfig records with a cluster table whose
    ClusterEndpoints have a port value of 0/unknown, rather than 0/tcp, to
    indicate that the node in question has no listening port.

Removed Functionality

Deprecated Functionality

  • The --disable-archiver configure flag no longer does anything and will be
    removed in 7.1. zeek-archiver has moved into the zeek-aux repository.

  • The policy/frameworks/telemetry/prometheus.zeek script has been deprecated
    and will be removed with Zeek 7.1. Setting the metrics_port field on a
    Cluster::Node implies listening on that port and exposing telemetry
    in Prometheus format.

v6.0.5

01 Aug 20:24
Compare
Choose a tag to compare

This release fixes the following bugs:

  • The Mozilla CA and Google CT lists were updated to their latest versions.

  • Connection IDs now correctly propogate into files.log.

  • A rare crash in CAF that happened when shutting down Zeek was resolved.

  • Binary addresses passed to Zeekctl were previously assumed to be valid
    unicode, which wasn't always the case. Some additional checking was added to
    ensure that's the case and to provide better error messaging when it isn't.

v6.2.1

16 May 17:41
Compare
Choose a tag to compare
  • A crash with ICMP packets involving errant length checking was fixed. Thank
    you to Ronny Barkan from Microsoft for reporting this issue.

  • When a shadow file is empty/missing during rotation, Zeek aborts with an error
    message, but if the shadow file was empty, it will still be there after the
    restart. This results in an endless restart loop. This has been corrected to
    overwrite the existing shadow file, using the default file extension and
    post processing function.

  • A new function remove_exclude was added to the PacketFilter framework
    which can be used to remove a previously added exclude filter by name.

  • A new option --localversion was added to the configure script. This
    option allows a caller to add custom strings to the end of the Zeek version
    reported by zeek -v.

v6.0.4

16 May 17:40
Compare
Choose a tag to compare

This release fixes the following bugs:

  • The Mozilla CA and Google CT lists were updated to their latest versions.

  • A crash with ICMP packets involving errant length checking was fixed. Thank
    you to Ronny Barkan from Microsoft for reporting this issue.

  • When a shadow file is empty/missing during rotation, Zeek aborts with an error
    message, but if the shadow file was empty, it will still be there after the
    restart. This results in an endless restart loop. This has been corrected to
    overwrite the existing shadow file, using the default file extension and
    post processing function.

  • A new function remove_exclude was added to the PacketFilter framework
    which can be used to remove a previously added exclude filter by name.

  • A new option --localversion was added to the configure script. This
    option allows a caller to add custom strings to the end of the Zeek version
    reported by zeek -v.

  • The highwayhash submodule was updated to fix a build issue for FreeBSD. Thank
    you to community member Craig Leres for reporting this.

v6.2.0

20 Feb 20:17
Compare
Choose a tag to compare

We would like to thank Anthony Verez (netantho), Bijoy Das (mute019), Jan
Grashöfer (J-Gras), Matti Bispham (mbispham), Phil Rzewski (philrz), and
xb-anssi for their contributions to this release.

Breaking Changes

  • The methods Dispatcher::Lookup() and Analyzer::Lookup() in the packet_analysis
    namespace were changed to return a reference to a std::shared_ptr instead of a copy
    for performance reasons.

  • Zeek's OPENSSL_INCLUDE_DIR is not automatically added to an external plugin's
    include path anymore. A plugin using OpenSSL functionality directly can use the
    following explicit entry to re-use Zeek's OPENSSL_INCLUDE_DIR:

    zeek_add_plugin(
    Namespace Name
    INCLUDE_DIRS "${OPENSSL_INCLUDE_DIR}"
    SOURCES ...
    )

  • The "segment_profiling" functionality and load_sample event have been removed
    without deprecation. This functionality was unmaintained and not known to be used.

  • Certain ldap.log and ldap_search.log fields have been renamed from
    plural to singular and their types changed to scalars. This maps better onto
    the expected request-response protocol used between client and server. Additionally,
    it removes the burden of working with non-scalar columns from downstream systems.

    Specifically, for ldap.log:

    • arguments: vector of string is now argument: string
    • diagnostic_messages: vector of string is now diagnostic_message: string
    • objects: vector of string is now object: string
    • opcodes: set[string] is now opcode: string
    • results: set[string] is now result: string

    For ldap_search.log, the following fields were changed:

    • base_objects: vector of string is now base_object: string
    • derefs: set[string] is now deref_aliases: string
    • diagnostic_messages: vector of string is now diagnostic_message: string
    • results: set[string] is now result: string
    • scopes: set[string] is now scope: string

    In the unlikely scenario that a request-response pair with the same message
    identifier is observed, containing different values for certain fields, new
    weirds are raised and will appear in weird.log, including the old and new
    values as well as the LDAP message identifier. The value within the LDAP logs
    will be the most recently observed one.

  • BIF methods now return a ValPtr directly instead of a BifReturnVal object
    which was just a thin wrapper around ValPtr. This may cause compilation errors
    in C++ code that was calling BIF methods directly.

New Functionality

  • The table type was extended to allow parallel regular expression matching
    when a table's index is a pattern. Indexing such tables yields a vector
    containing all values of matching patterns for keys of type string.

    As an example, the following snippet outputs [a, a or b], [a or b].

    global tbl: table[pattern] of string;
    tbl[/a/] = "a";
    tbl[/a|b/] = "a or b";
    tbl[/c/] = "c";
    print tbl["a"], tbl["b"];

    Depending on the patterns and input used for matching, memory growth may
    be observed over time as the underlying DFA is constructed lazily. Users are
    advised to test with realistic and adversarial input data with focus on
    memory growth. The DFA's state can be reset by removal/addition of a single
    pattern. For observability, a new bif table_pattern_matcher_stats()
    can be used to gather MatcherStats.

  • Support for delaying log writes.

    The logging framework offers two new functions Log::delay() and Log::delay_finish()
    to delay a Log::write() operation. This new functionality allows delaying of
    a specific log record within the logging pipeline for a variable but bounded
    amount of time. This can be used, for example, to query and wait for additional
    information to attach to the pending record, or even change its final verdict.

    Conceptually, delaying a log record happens after the execution of the global
    Log::log_stream_policy hook for a given Log::write() and before the
    execution of filter policy hooks. Any mutation of the log record within the
    delay period will be visible to filter policy hooks. Calling Log::delay()
    is currently only allowed within the context of the Log::log_stream_policy hook
    for the active Log::write()` operation (or during the execution of post delay callbacks). While this may appear restrictive, it makes it explicit which Log::write()``
    operation is subject to the delay.

    Interactions, semantics and conflicts of this feature when writing the same
    log record multiple times to the same or different log streams need to be taken
    into consideration by script writers.

    Given this is the first iteration of this feature, feedback around usability and
    use-cases that aren't covered are more than welcome.

  • A WebSocket analyzer has been added together with a new websocket.log.

    The WebSocket analyzer is instantiated when a WebSocket handshake over HTTP is
    recognized. By default, the payload of WebSocket messages is fed into Zeek's dynamic
    protocol detection framework, possibly discovering and analyzing tunneled protocols.

    The format of the log and the event semantics should be considered preliminary until
    the arrival of the next long-term-stable release (7.0).

    To disable the analyzer in case of fatal errors or unexpected resource usage,
    use the Analyzer::disabled_analyzers pattern:

    redef Analyzer::disabled_analyzers += {
    Analyzer::ANALYZER_WEBSOCKET,
    };

  • The SMTP analyzer was extended to recognize and properly handle the BDAT command
    from RFC 3030. This improves visibility into the SMTP protocol when mail agents
    and servers support and use this extension.

  • The event keyword in signatures was extended to support choosing a custom event
    to raise instead of signature_match(). This can be more efficient in certain
    scenarios compared to funneling every match through a single event.

    The new syntax is to put the name of the event before the string used for the
    msg argument. As an extension, it is possible to only provide an event name,
    skipping msg. In this case, the framework expects the event's parameters to
    consist of only state and data as follows:

    signature only-event {
    payload /.*root/
    event found_root
    }

    event found_root(state: signature_state, data: string) { }

    Using the msg parameter with a custom event looks as follows. The custom
    event's parameters need to align with those for ``signature_match()` event:

    signature event-with-msg {
    payload /.*root/
    event found_root_with_msg "the-message"
    }

    event found_root_with_msg(state: signature_state, msg: string, data: string) { }

    Note, the message argument can currently still be specified as a Zeek identifier
    referring to a script-level string value. If used, this is disambiguated behind
    the scenes for the first variant. Specifying msg as a Zeek identifier has
    been deprecated with the new event support and will be removed in the future.

    Note that matches for signatures with custom events will not be recorded in
    signatures.log. This log is based on the generation of signature_match()
    events.

  • The QUIC analyzer has been extended to support analyzing QUIC Version 2
    INITIAL packets (RFC 9369). Additionally, prior draft and some of
    Facebook's mvfst versions are supported. Unknown QUIC versions will now be
    reported in quic.log as an entry with a U history field.

  • Conditional directives (@if, @ifdef, @ifndef, @else and
    @endif) can now be placed within a record's definition to conditionally
    define or extend a record type's fields.

    type r: record {
    	c: count;
    @if ( cond )
    	d: double;
    @else
    	d: count;
    @endif
    };
    

    Note that generally you should prefer record extension in conditionally loaded
    scripts rather than using conditional directives in the original record definition.

  • The 'X' code can now appear in a connection's history. It is meant to indicate
    situations where Zeek stopped analyzing traffic due to exceeding certain limits or
    when encountering unknown/unsupported protocols. Its first use is to indicate
    Tunnel::max_depth being exceeded.

  • A new Intel::seen_policy hook has been introduced to allow intercepting
    and changing ``Intel::seen` behavior:

    hook Intel::seen_policy(s: Intel::Seen, found: bool)

  • A new NetControl::rule_added_policy hook has been introduced to allow modification
    of NetControl rules after they have been added.

  • The IP geolocation / ASN lookup features in the script layer provide better
    configurability. The file names of MaxMind databases are now configurable via
    the new mmdb_city_db, mmdb_country_db, and mmdb_asn_db constants,
    and the previously hardwired fallback search path when not using an
    mmdb_dir value is now adjustable via the mmdb_dir_fallbacks
    vector. Databases opened explicitly via the mmdb_open_location_db and
    mmdb_open_asn_db functions now behave more predictably when updated or
    removed. For details, see:
    https://docs.zeek.org/en/master/customizations.html#address-geolocation-and-as-lookups

  • The zeek-config script now provides a set of --have-XXX checks for
    features optionally compiled in. Each check reports "yes"/"no" to stdout and
    exits with 0/1, respectively.

Changed Functionality

  • The split_string family of functions now respect the beginning-of-line ^ and
    end-of-line $ anchors. Previously, an anchored pattern would be matched anywhere
    in the input string.

  • The sub() and ``gsub()` functions now respect the beginning-of-l...

Read more

v6.1.1

22 Jan 16:46
Compare
Choose a tag to compare

This release fixes the following security issues:

  • A specially-crafted series of packets containing nested MIME entities can
    cause Zeek to spend large amounts of time parsing the entities. Due to the
    possibility of receiving these packets from remote hosts, this is a DoS
    risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
    that limits the depth the parser will attempt to follow the entity nesting. If
    the limit is reached an exceeded_mime_max_depth weird is generated.

This release fixes the following bugs:

  • CMake correctly passes along third-party package information when building
    plugins. This ensures that, for example, the same paths to OpenSSL used in a
    Zeek build are provided to a plugin build.

  • Fix a problem with the HTTP analyzer where a signature regex ending in '$'
    used to match against 'http-request-body' or 'http-reply-bdoy' will never
    succeed. Thank you to GitHub user xb-anssi for this fix.

  • The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.

  • The SMB::State$recent_files field was not correctly expiring entries, leading
    to unbounded state growth. This is fixed to correctly follow the &read_expire
    condition on the field. Thank you to Slack user ya-sato for reporting this.

  • The &create_expire attribute is now kept valid after clearing a table. After
    switching the known scripts away from broker stores, the &create_expire value
    of the local tables/sets of the known scripts wasn't in effect due to
    Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
    the &create_expire attribute intact. This broke the "log hosts every 24h"
    behavior.

  • Zeek builds using the --binary-package argument and including Spicy will now
    include all necessary Spicy symbols.

v6.0.3

22 Jan 16:41
Compare
Choose a tag to compare

This release fixes the following security issues:

  • A specially-crafted series of packets containing nested MIME entities can
    cause Zeek to spend large amounts of time parsing the entities. Due to the
    possibility of receiving these packets from remote hosts, this is a DoS
    risk.The fix included adds a new option (MIME::max_depth) to the MIME parser
    that limits the depth the parser will attempt to follow the entity nesting. If
    the limit is reached an exceeded_mime_max_depth weird is generated.

This release fixes the following bugs:

  • CMake correctly passes along third-party package information when building
    plugins. This ensures that, for example, the same paths to OpenSSL used in a
    Zeek build are provided to a plugin build.

  • Fix a problem with the HTTP analyzer where a signature regex ending in '$'
    used to match against 'http-request-body' or 'http-reply-bdoy' will never
    succeed. Thank you to GitHub user xb-anssi for this fix.

  • The DNS analyzer now understands the Ed25519 and Ed448 signature algorithms.

  • The SMB::State$recent_files field was not correctly expiring entries, leading
    to unbounded state growth. This is fixed to correctly follow the &read_expire
    condition on the field. Thank you to Slack user ya-sato for reporting this.

  • The &create_expire attribute is now kept valid after clearing a table. After
    switching the known scripts away from broker stores, the &create_expire value
    of the local tables/sets of the known scripts wasn't in effect due to
    Cluster::node_up() and Cluster::node_down() re-assigning these without keeping
    the &create_expire attribute intact. This broke the "log hosts every 24h"
    behavior.

  • Zeek builds using the --binary-package argument and including Spicy will now
    include all necessary Spicy symbols.

v6.1.0

13 Oct 18:57
Compare
Choose a tag to compare

Breaking Changes

  • assert is now a reserved keyword for the new assert statement.

  • The __bro_plugin__ file that gets generated as part of plugin builds was
    renamed to __zeek_plugin__. This will affect the ability for older
    versions of zkg to use the zkg unload and zkg load commands. This
    should only cause breakage for people using a version of ``zkg` that doesn't
    come bundled with Zeek (which we generally don't recommend doing).

  • Zeek does not traverse into dot directories to find plugins or hlto files
    anymore. Any dot directories found below the directories specified in
    ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are now skipped. Dot directories
    explicitly listed in ZEEK_PLUGIN_PATH or ZEEK_SPICY_MODULE_PATH are not
    skipped.

  • External plugins will fail to configure if their minimum required CMake
    version is below 3.15. This was a warning with Zeek 6.0, but has caused user
    confusion due to unhelpful error messages around the IN_LIST operator policy.

  • The FindBISON, FindOpenSSL, FindPackageHandleStandardArgs, FindPackageMessage,
    and SelectLibraryConfigurations cmake files were removed from our cmake
    repository in favor of the versions that come with CMake. This should not
    cause any breakage, but it is possible in the case that someone was using
    these in a plugin.

New Functionality

  • Zeek now includes the LDAP protocol analyzer from the zeek/spicy-ldap project
    (https://github.com/zeek/spicy-ldap). This analyzer is enabled by default. The
    analyzer's events and its ldap.log and ldap_search.log should be
    considered preliminary and experimental until the arrival of Zeek's next
    long-term-stable release (7.0).

    If you observe unusually high CPU consumption or other issues due to this
    analyzer being enabled by default, the easiest way to disable it is via the
    Analyzer::disabled_analyzers const as follows:

    redef Analyzer::disabled_analyzers += {
    Analyzer::ANALYZER_LDAP_UDP,
    Analyzer::ANALYZER_LDAP_TCP,
    };

    Please do report issues to us including diagnostic information in case this is
    necessary in your environment. We're also open to general feedback about the
    structure of the new logs.

  • Zeek now includes the QUIC protocol analyzer from the zeek/spicy-quic project
    (https://github.com/zeek/spicy-quic). This project is a fork of Fox-IT's
    initial implementation (https://github.com/fox-ds/spicy-quic).

    As for the LDAP analyzer, the analyzer's events and the new quic.log
    should be considered preliminary and experimental until the arrival of Zeek's
    next long-term-stable release (7.0). As above, any feedback and contributions
    to this analyzer and the new log are welcome.

    The analyzer's functionality is limited to decryption of the INITIAL packets
    of QUIC version 1. If decryption of these packets is successful, the handshake
    data is forwarded to Zeek's SSL analyzer. An ssl.log entry will appear in
    ssl.log for QUIC connections. The entry in the conn.log will contain
    quic and ssl in the service field.

    To disable the analyzer in case of issues, use the following snippet:

    redef Analyzer::disabled_analyzers += {
    Analyzer::ANALYZER_QUIC,
    };

  • Added a new assert statement for assertion based testing and asserting
    runtime state.

    assert <expr: bool>[, <message: string>];

    This statement comes with two hooks. First, assertion_failure() that is
    invoked for every failing assert statement. Second, assertion_result()
    which is invoked for every assert statement and its outcome. The latter allows
    to construct a summary of failing and passing assert statements. Both hooks
    receive the location and call stack for the assert statement via a
    Backtrace vector.

    A failing assert will abort execution of the current event handler similar to
    scripting errors. By default, a reporter error message is logged. Using the
    break statement within assertion_failure() or assertion_result()
    allows to suppress the default message.

  • Add a new &default_insert attribute for tables. This behaves as
    &default with the addition that the default value is inserted into the
    table upon a failed lookup. Particularly for tables with nested container
    values, the &default behavior of not inserting the value can be of little
    use.

  • The from_json() function now takes an optional key_func argument to
    normalize JSON object key names. This can be useful if the keys in a JSON
    object are not valid Zeek identifiers or reserved keywords.

  • Module names are now included in global_ids(). Their key in the returned
    table is prefixed with "module " and their value will have the type_name
    field set to "module".

  • Identifiers in the global scope can now be referenced and defined from within
    modules by prefixing their names with ::. Previously, these required an
    explicit GLOBAL:: prefix to be used. Using GLOBAL:: has been
    deprecated.

  • The as keyword now supports casting between set and vector values
    with the same element type. Converting set values with multiple index
    values is not supported. We plan to extend the use of the as keyword to
    support more type conversions in the future.

  • Added new packet analyzer to handle PCAP files DLT_PPP link type.

  • Fixed appending of any to vector of any.

  • The ModBus analyzer's function support was expanded, with new handling of the
    Encapsulation Interface Transport (function 28) And Diagnostics (function 8)
    functions. This adds new modbus_encap_interface_transport_{request,response}
    and modbus_diagnostics_{request,response} events.

  • The ModBus file record read and write events now provide the full data from
    the request and response messages as part of the event data.

  • The full PDU length was added to the ModBusHeader record type passed with
    all of the ModBus events.

Changed Functionality

  • A connection's value is now updated in-place when its directionality is
    flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
    protocol specific approaches). Previously, a connection's value was discarded
    when flipped, including any values set in a new_connection() handler. A
    new connection_flipped() event is added to allow updating custom state in
    script-land.

  • Loading policy/frameworks/notice/community-id.zeek now also automatically
    community ID logging. In the past, loading the script had no effect unless
    policy/protocols/conn/community-id-logging.zeek was loaded before. This
    was fairly unusual and hard to debug behavior.

  • Connections to broadcast addresses are not flipped based on
    likely_server_ports anymore. Previously, broadcast packets originating
    from a likely server port resulted in 255.255.255.255 being the originator in
    conn.log.

  • When too many HTTP requests are pending, Zeek will now log them at once and
    reset request/response correlation instead of running into unbounded state
    growth. This behavior is configurable via a new option
    HTTP::max_pending_requests. The default is 100.

  • Fix deferred initialization of nested records containing non-const &default
    attributes.

  • Parameter lists for functions, events and hooks now use commas instead of
    semicolons in error messages or when printing such functions.

  • The IO buffer size used for PCAP file reading is now always 128kb. This new
    default can be changed via Pcap::bufsize_offline_bytes.

  • The input framework now provides better information in error messages when
    encountering missing non-optional field while loading data.

  • The SSL analyzer will now parse a configurable maximum of 10 SSL Alerts per
    SSL message. For TLS 1.3, the maximum is implicitly 1 as defined by RFC 8446.
    If there are more alerts, a new weird "SSL_excessive_alerts_in_record" is raised.
    For non-TLS 1.3, the maximum can be redefined via SSL::max_alerts_per_record.

  • The ssl_history field in the ssl.log is now capped at a configurable
    limit of 100 characters prevent unbounded growth. The limit can be changed
    via the option SSL::max_ssl_history_length. When reached, a new weird
    named "SSL_max_ssl_history_length_reached" is raised.

Deprecated Functionality

  • Accessing globals with GLOBAL::name has been deprecated and will be
    removed with Zeek 7.1. Use ::name instead.

  • The original trigger::Trigger constructor has been deprecated and will be
    removed with Zeek 7.1. Use the new alternative constructor (per
    src/Trigger.h) instead, including replacing any use of new ... with
    make_intrusive<...>. The new constructor differs only in the placement of
    the timeout parameter, and in that - unlike the original - it always
    returns a valid pointer, which must be Unref()'d after construction, either
    explicitly (if using new) or implicitly (if using
    make_intrusive<...>).

v6.0.2

27 Oct 19:51
Compare
Choose a tag to compare

This release fixes the following security issues:

  • A specially-crafted SSL packet could cause Zeek to leak memory and potentially
    crash. Due to the possibility of receiving these packets from remote hosts,
    this is a DoS risk. The fix included adds additional memory cleanup to the
    x509 file analyzer.

  • A specially-crafted series of FTP packets could cause Zeek to log entries for
    requests that have already been completed, using resources unnecessarily and
    potentially causing Zeek to lose other traffic. Due to the possibility of
    receiving these packets from remote hosts, this is a DoS risk. The fix
    included changes the way that we track the pending FTP commands, avoiding
    possibly reusing the same value for subsequent commands.

  • A specially-crafted series of SSL packets could cause Zeek to output a very
    large number of unnecessary alerts for the same record. Due to the possibility
    of receiving these packets from remote hosts, this is a DoS risk. The fix
    included adds a new option SSL::max_alerts_per_record that caps the number of
    alerts that can be generated for an SSL record. For TLS 1.3 this is capped at
    1 as defined in RFC 8446. For non-TLS 1.3 it is a configurable value. A
    SSL_excessive_alerts_in_record weird will be raised if the cap is exceeded.

  • A specially-crafted series of SSL packets could cause Zeek to generate very
    long ssl_history fields in the ssl.log, potentially using a large amount of
    memory due to unbounded state growth. Due to the possibility of receiving
    these packets from remote hosts, this is a DoS risk. The fix included adds a
    new option SSL::max_ssl_history_length that caps this to 100 characters by
    default. A SSL_max_ssl_history_length_reached weird will be raised if the cap
    is exceeded.

  • A specially-crafted IEEE802.11 packet could cause Zeek to overflow memory and
    potentially crash. Due to the possibility of receiving these packets from
    remote hosts, this is a DoS risk. The fix included adds additional bounds
    checking to the IEEE802.11 packet analyzer.

This release fixes the following bugs:

  • Fixed Spicy type names from causing collisions with existing Zeek types.

  • On some systems with low values for the maximum number of file descriptors, it
    was possible to run into crashes when doing DNS lookups if all of the file
    descriptors were used. This is now avoided with better checking for the number
    of available file descriptors before trying a lookup. Thank you to Zeek Slack
    user h-mikami for reporting this issue.

  • Tables backed by a Broker backend now correctly support deletion if they have
    complex index types. Zeek previously reported an error when trying to delete
    elements from these tables.

  • A significant performance issue with Zeek's supervisor code was fixed,
    revolving around the re-initialization of the Event Manager object used to
    track events. Thank you to Jan Grashoefer for reporting this issue.

  • The MaxMind DB code now cleans up after itself, resolving a memory leak with
    the loaded database files.

  • The ZeekJS submodule was updated to version 0.9.6, bringing fixes for
    zeek.invoke and zeek.event crashes, garbage collection, and an issue where
    Zeek may stop executing events from ZeekJS.

v6.0.1

12 Sep 20:46
Compare
Choose a tag to compare

This release fixes the following security issues:

  • File extraction limits were not correctly enforced for files containing large
    amounts of missing bytes. Crafting files with large amounts of missing bytes
    in them could cause Zeek to spend a long time processing data, allocate a lot
    of main memory, and write a lot of data to disk. Due to the possibility of
    receiving these packets from remote hosts, this is a DoS risk. The fix
    included makes Zeek correctly enforce file size limits. We also added a new
    option (default_limit_includes_missing) which allows to customize the behavior
    when encountering large amounts of missed bytes in file. This issue was
    discovered by Luca Cigarini.

  • Sessions are sometimes not cleaned up completely within Zeek during shutdown,
    potentially causing a crash when using the -B dpd flag for debug logging. This
    is low priority because it only happens at shutdown and only when using that
    flag. The fix included is to reorder shutdown to cleanup all the sessions
    prior to tearing down the analyzers.

  • A specially-crafted HTTP packet can cause Zeek’s filename extraction code to
    take a long time to process the data. Due to the possibility of receiving
    these packets from remote hosts, this is a DoS risk. The fix included adjusts
    the regular expression used in the extract_filename_from_content_disposition()
    script function to more-correctly match the data.

  • A specially-crafted series of FTP packets made up of a CWD request followed by
    a large amount of ERPT requests may cause Zeek to spend a long time logging
    the commands. Due to the possibility of receiving these packets from remote
    hosts, this is a DoS risk. The fix included is to prevent logging of pending
    commands for FTP packets.

  • A specially-crafted VLAN packet can cause Zeek to overflow memory and
    potentially crash. Due to the possibility of receiving these packets from
    remote hosts, this is a DoS risk. The fix included is to add some additional
    length checking to the VLAN analyzer.

This release fixes the following bugs:

  • Fixed a base64 decoding issue with the authorization field of HTTP request
    headers that was sometimes causing Zeek to output error messages. Thank you
    to GitHub user @progmboy for reporting and providing a fix for this issue.

  • Ensure that Zeek builds use the internal version of Spicy instead of external
    installations, unless specifically configured for that mode.

  • Support was added for switch fields when exporting Spicy types to Zeek.

  • A number of fixes were added to protect against potential unbounded state
    growth with the SMB and DCE-RPC analyzers. SMB close requests will properly
    tear down an related DCE-RPC analyzers. A new SMB::max_dce_rpc_analyzers
    script variable was added that allows finer control of how many DCE-RPC
    analyzers are allowed to be created per SMB connection. Thanks to Zeek Slack
    user Takomi Sugawara for reporting this issue.

  • Fixed a regression in the UDP and TCP analyzers that was causing more data
    than necessary to be forwarded to the next analyzer in the chain. Thanks to
    Zeek Slack user Hiep Long Tan for reporting this issue.

  • A connection's value is now updated in-place when its directionality is
    flipped due to Zeek's heuristics (for example, SYN/SYN-ACK reversal or
    protocol specific approaches). Previously, a connection's value was discarded
    when flipped, including any values set in a new_connection() handler. A
    new connection_flipped() event is added to allow updating custom state in
    script-land.

  • Fixed undefined symbols being reported from Spicy when building some of the
    binary packages for Zeek.

  • Loading policy/frameworks/notice/community-id.zeek now also automatically
    community ID logging. In the past, loading the script had no effect unless
    policy/protocols/conn/community-id-logging.zeek was loaded before. This
    was fairly unusual and hard to debug behavior.

  • Spicy no longer registers an extra port for every port registered in a
    plugin's .evt file.

  • Timeouts in DNS resolution no longer cause uncontrolled memory growth.

  • Fix check to skip DNS hostname lookups for notices that are not delivered via
    email in policy/frameworks/notice/extend-email/hostnames. Due to that
    policy script being loaded in the Zeek's default local.zeek, this
    previously caused unneeded DNS lookups for every generated notice instead of
    just those delivered via email.