Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: various space management non-functional changes #18314

Merged

Commits on May 8, 2024

  1. storage: rename log_manager::collection_threshold

    I found the name a bit vague in trying to understand what this code was
    doing. This updates it to lowest_ts_to_retain(), and updates the
    disk_space_manager to use it rather than duplicating the logic.
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    d1ba0ee View commit details
    Browse the repository at this point in the history
  2. resource_mgmt: clarify variable names in manage_disk_space

    target_excess -> adjusted_target_excess: I find the name more intuitive,
    despite being more verbose at call sites, as a reader I prefer the
    reminder that the value has been tweaked
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    0597b47 View commit details
    Browse the repository at this point in the history
  3. resource_mgmt: tweak info log in space management

    I found the existing logging a bit confusing because "tiered storage
    retention" isn't widely used verbiage. This tweaks it to refer to this
    as "space management of tiered storage topics".
    
    Also s/recover/remove: it feels a bit clearer, segment removal isn't
    referred to as "recovering" in other contexts.
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    c8eb9e7 View commit details
    Browse the repository at this point in the history
  4. storage: rename maybe_override_retention_config()

    The previous name is pretty vague: as a reader it's unclear what it's
    actually doing other than modifying the config in some way.
    
    Updates it to explicitly mention that it is handling local storage
    overrides. IMO this is easier to reason about when reading the method
    and its call sites.
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    986264e View commit details
    Browse the repository at this point in the history
  5. storage: rename override_retention_config

    Similar to maybe_override_retention_config(), updates
    override_retention_config() to be called
    apply_local_storage_overrides().
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    fded43a View commit details
    Browse the repository at this point in the history
  6. storage: rename apply_base_overrides

    Renames apply_base_overrides() to apply_kafka_retention_overrides().
    I think it's a bit easier to infer what the Kafka retention overrides
    are (i.e. the ones that are Kafka compatible: retention.ms,
    retention.bytes).
    
    Others considered but not taken:
    - total_retention
    - full_retention
    - consumable_retention
    - topic_retention
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    de686ce View commit details
    Browse the repository at this point in the history
  7. storage: refactor retention_adjust_timestamps

    Pulls out a couple instances of duplicate code, in an effort to clarify
    naming of various overrides and adjustments done around retention.
    andrwng committed May 8, 2024
    Configuration menu
    Copy the full SHA
    e81fc13 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. storage: tweak strict_retention condition for clarity

    With the original code, it was easy to get tripped up by the nots and
    ors. Tweaks the implementation and adds a comment to make this easier to
    read.
    
    No functional changes.
    andrwng committed May 10, 2024
    Configuration menu
    Copy the full SHA
    6705a4a View commit details
    Browse the repository at this point in the history
  2. resource_mgmt: add clarity to info+ logs

    We have seen issues where the lack of tiered storage resulted in disk
    filling up. The info logs didn't make it clear what was happening and
    it was assumed to be a bug in code, rather than a misconfiguration
    issue. This commit bumps a log line to info level so that it would have
    been clear that a specific error path wasn't being taken.
    
    Also tweaks wording to makes a stronger connection to tiered storage.
    andrwng committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9425c18 View commit details
    Browse the repository at this point in the history