Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[v24.1.x] Fix some concurrent memory access problems in partition balancer #18406

Merged
merged 4 commits into from
May 11, 2024
Merged

[v24.1.x] Fix some concurrent memory access problems in partition balancer #18406

merged 4 commits into from
May 11, 2024

Commits on May 10, 2024

  1. c/topic_table: bump _topics_map_revision in more places 

    Partition balancer relies on _topics_map_revision checks to safely iterate
over topic table collections with partition granularity (i.e. references
to partition data and replica sets are stored and accessed across yield
points). To make this safe, increment _topics_map_revision every time
_topics, _updates_in_progress, _disabled_partitions or nested
collections are modified in a way that invalidates references or
iterators.

(cherry picked from commit 30bbfb1)
    @ztlpn @vbotbuildovich
    ztlpn authored and vbotbuildovich committed May 10, 2024
    Configuration menu
    Copy the full SHA
    0a9deb5 View commit details
    Browse the repository at this point in the history

  2. utils: introduce concurrent_modification_error exception 

    It is handy to have a base class for all instances of concurrent
modifications.

(cherry picked from commit 627db4d)
    @ztlpn @vbotbuildovich
    ztlpn authored and vbotbuildovich committed May 10, 2024
    Configuration menu
    Copy the full SHA
    bc99aaa View commit details
    Browse the repository at this point in the history

  3. c/scheduling: detect concurrent allocation_state replacements 

    Shard-local allocation_state object is replaced when we are applying
a controller snapshot. After this happens, all live allocated_partition
objects become invalid. Detect this and throw
concurrent_modification_error in case these objects are still used and
make destructors no-op.

(cherry picked from commit a57a575)
    @ztlpn @vbotbuildovich
    ztlpn authored and vbotbuildovich committed May 10, 2024
    Configuration menu
    Copy the full SHA
    0d0f3ec View commit details
    Browse the repository at this point in the history

  4. c/partition_balancer/ut: add concurrent topic_table updates test 

    (cherry picked from commit 1c63990)
    @ztlpn @vbotbuildovich
    ztlpn authored and vbotbuildovich committed May 10, 2024
    Configuration menu
    Copy the full SHA
    9ab30ef View commit details
    Browse the repository at this point in the history