Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tablet throttler: read and use MySQL host metrics #16904

Merged
merged 43 commits into from
Nov 20, 2024
Merged

Tablet throttler: read and use MySQL host metrics #16904

merged 43 commits into from
Nov 20, 2024

Commits on Sep 23, 2024

  1. mysqlctl proto: adding ThrottlerMetrics rpc 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    71d98fe View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. naming: SystemMetrics instead of ThrottlerMetrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    9922731 View commit details
    Browse the repository at this point in the history

  2. implement SystemMetrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    214db0d View commit details
    Browse the repository at this point in the history

  3. tabletmanager: MysqlSystemMetrics rpc 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    5660b77 View commit details
    Browse the repository at this point in the history

  4. implement SystemMetrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    2ec9f75 View commit details
    Browse the repository at this point in the history

Commits on Sep 26, 2024

  1. pass TabletAlias to throttler 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0741a2b View commit details
    Browse the repository at this point in the history

  2. implement MysqlSystemMetrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    0b3666b View commit details
    Browse the repository at this point in the history

  3. force check result 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    20e0577 View commit details
    Browse the repository at this point in the history

  4. unit test 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    df99856 View commit details
    Browse the repository at this point in the history

  5. resolved conflict 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    a5d76d3 View commit details
    Browse the repository at this point in the history

  6. remove bogus files 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    098e6a4 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into throttler-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 26, 2024
    Configuration menu
    Copy the full SHA
    338c52c View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2024

  1. resolved conflict 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    4250909 View commit details
    Browse the repository at this point in the history

  2. remove excessive file, caused by merge conflict 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    07a3461 View commit details
    Browse the repository at this point in the history

  3. Rename 'SystemMetrics'->'HostMetrics' 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    2c48d36 View commit details
    Browse the repository at this point in the history

  4. better doclet 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    0d73d30 View commit details
    Browse the repository at this point in the history

  5. use vtrpcpb.RPCError 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    e3fccae View commit details
    Browse the repository at this point in the history

  6. "datadir-used-ratio" 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 29, 2024
    Configuration menu
    Copy the full SHA
    757cada View commit details
    Browse the repository at this point in the history

Commits on Sep 30, 2024

  1. internal refactor: SelfMetricReadParams 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Sep 30, 2024
    Configuration menu
    Copy the full SHA
    7dd4b8b View commit details
    Browse the repository at this point in the history

Commits on Oct 1, 2024

  1. Merge branch 'main' into throttler-use-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 1, 2024
    Configuration menu
    Copy the full SHA
    73fd43f View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. using threads_running as loadavg does not produce meaningful info on … 

    …MacOS, and MacOS is beign used in local dev env unit and endtoend testing

Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    9166093 View commit details
    Browse the repository at this point in the history

  2. adding MysqldLoadAvgMetricName and MysqldDatadirUsedRatioMetricName 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    545112f View commit details
    Browse the repository at this point in the history

  3. moredebug info in case of error 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    72b09b9 View commit details
    Browse the repository at this point in the history

  4. return empty (zero) result for loadavg when not on linux 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    91d2e42 View commit details
    Browse the repository at this point in the history

  5. return empty (zero) result for loadavg when not on linux 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    38b9d73 View commit details
    Browse the repository at this point in the history

  6. SelfMetricReadParams includes TabletManagerClient and TabletInfo 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    adfcf6d View commit details
    Browse the repository at this point in the history

  7. pass tmClient and tabletInfo to self metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d8b6ec2 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into throttler-use-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    6726285 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2024

  1. rename ThrottlerMetricsPublisher->MetricsPublisher 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    79b964f View commit details
    Browse the repository at this point in the history

  2. rename MetricsPublisher->metricsPublisher 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    838e610 View commit details
    Browse the repository at this point in the history

  3. require.Failf 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    9d9d551 View commit details
    Browse the repository at this point in the history

  4. capture iteration number 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    e25437a View commit details
    Browse the repository at this point in the history

  5. clarify intention of test 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    14bae55 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'main' into throttler-use-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 20, 2024
    Configuration menu
    Copy the full SHA
    4c097f9 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. endtoend check for MySQLHostMetrics, specifically checking for existe… 

    …nce of mysqld-* throttler values, and for nonzero value of mysqld-datadir-used-ratio

Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    80891a0 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into throttler-use-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    3b45b2b View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2024

  1. Merge branch 'main' into throttler-use-mysql-metrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    6a37aa2 View commit details
    Browse the repository at this point in the history

  2. proper multi-word camel case, to export metric names such as Throttle… 

    …rAggregatedShardMysqldDatadirUsedRatio

Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Oct 31, 2024
    Configuration menu
    Copy the full SHA
    5675ea8 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2024

  1. terminology: PascalCase 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    ceb2f81 View commit details
    Browse the repository at this point in the history

  2. introduce osutil.LoadAvg function, works on linux and darwin 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    50f7f27 View commit details
    Browse the repository at this point in the history

  3. use osutil.LoadAvg in both LoadAvgSelfMetric and mysqld.hostMetrics 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    c2c3384 View commit details
    Browse the repository at this point in the history

  4. 1 second caching for loadavg metric 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Nov 11, 2024
    Configuration menu
    Copy the full SHA
    b3f3199 View commit details
    Browse the repository at this point in the history

Commits on Nov 12, 2024

  1. OS-specific build files 

    Signed-off-by: Shlomi Noach <[email protected]>
    @shlomi-noach
    shlomi-noach committed Nov 12, 2024
    Configuration menu
    Copy the full SHA
    8adc7eb View commit details
    Browse the repository at this point in the history