Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] Get disk throughput rates via gFlag for use in disk full protection #23373

Closed
1 task done
hari90 opened this issue Aug 2, 2024 · 0 comments
Closed
1 task done

[DocDB] Get disk throughput rates via gFlag for use in disk full protection #23373

hari90 opened this issue Aug 2, 2024 · 0 comments
Assignees
@hari90
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@hari90
Copy link
Contributor

hari90 commented Aug 2, 2024 •
edited by jira bot
Loading

Jira Link: DB-12295

Description

Flags reject_writes_min_disk_space_check_interval_sec and reject_writes_min_disk_space_aggressive_check_mb need to be adjusted based on the disk throughput. Instead of setting two flags, add a max_disk_throughput_mbps flag that can be used to determine the value for the other gFlags.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@hari90 hari90 added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Aug 2, 2024
@hari90 hari90 self-assigned this Aug 2, 2024
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Aug 2, 2024
@yugabyte-ci yugabyte-ci removed the status/awaiting-triage Issue awaiting triage label Aug 26, 2024
@hari90 hari90 closed this as completed in 417092a Aug 29, 2024
hari90 added a commit that referenced this issue Aug 29, 2024
@hari90
[BACKPORT 2.23.0][#23373] DocDB: Add max_disk_throughput_mbps gflag t…
0dfe97d 
…o control disk full rejection

Summary:
Original commit: 417092a / D37604
New gflag `--max_disk_throughput_mbps` is used for rejecting writes when disk is full. This replaces `--reject_writes_min_disk_space_aggressive_check_mb` and automates the value for `--reject_writes_min_disk_space_mb`. `--reject_writes_min_disk_space_mb` can be manually set to non zero value to override the automated value.

Fixes #23373
Jira: DB-12295

Test Plan:
YCqlDiskFullTest.TestDiskFull
YSqlDiskFullTest.TestDiskFull

Reviewers: yyan, rthallam

Reviewed By: yyan

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D37637
jasonyb pushed a commit that referenced this issue Aug 29, 2024
@yugabyte-ci @jaki
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 88
f15bc6c 
Summary:
 bfd17b5 [PLAT-14601]Support restore to Point in time part 2 - Backup/Restore/Restore preflight related changes - Add JSON Property
 91d000a [doc] Thirdparty to integrations (#23598)
 Excluded: 6456777 [#23626] allow loading old dumps that do not have index pg_class OIDs
 c8b8be3 [DB-12586] yugabyted: Update schema migration UI (#23217)
 Excluded: 4ea354b [#23521] YSQL: Cost YB Bitmap Table Scan remote filters
 b80999d [PLAT-14974][PLAT-15045] Added prometheus user as part of yugabyte group
 63f1d65 [#23669] YSQL: Add more logs to debug an assertion failure
 1ad4795 [docs] [TA] Added TA-23476: YCQL currenttimestamp() precision (#23642)
 41ae6b4 [#23653] docdb: Adjust waits for MasterPathHandlersItest.TestUndeletedParentTablet in TSAN
 78b0ae4 [DB-12587] yugabyted: Update data migration UI (#23291)
 d234b3a [PLAT-15046] Create log directory with correct permissions to allow users to export logs without using sudo
 8713c18 [doc][yba] Clarify pre-req for cloud provider image upgrades (#23285)
 9be5c91 [#23448] YSQL: fix failing test PgAutoAnalyzeTest.CheckTableMutationsCount
 Excluded: 9d54710 [#22147] YSQL, QueryDiagnostics: Pgss support for query diagnostics
 417092a [#23373] DocDB: Add max_disk_throughput_mbps gflag to control disk full rejection
 e3a1a36 [PLAT-15035] Add support to sync gflags secret mount location to actual gflag file used by services
 23a6a4c [PLAT-14525][PLAT-14953] Add local provider tests for switchover, failover, change replica, and restart
 6026029 [PLAT-15100][Master]Observed two Scheduled Backup Policies tabs in Backup page
 2cf648b [#23581] CDCSDK: Support dynamic table addition with table removal
 b14851d [#23702] xClusterDDLRepl: Add extra logging
 8a0d6ff [#23645] docdb: Reorder heartbeat handling logic to fix regression.
 2b30b5e [Docs] Changes for Experimental AI (#23714)

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, tfoucher

Differential Revision: https://phorge.dev.yugabyte.com/D37645
hari90 added a commit that referenced this issue Aug 29, 2024
@hari90
[BACKPORT 2024.1][#23373] DocDB: Add max_disk_throughput_mbps gflag t…
c30aef2 
…o control disk full rejection

Summary:
Original commit: 417092a / D37604
New gflag `--max_disk_throughput_mbps` is used for rejecting writes when disk is full. This replaces `--reject_writes_min_disk_space_aggressive_check_mb` and automates the value for `--reject_writes_min_disk_space_mb`. `--reject_writes_min_disk_space_mb` can be manually set to non zero value to override the automated value.

Fixes #23373
Jira: DB-12295

Test Plan:
YCqlDiskFullTest.TestDiskFull
YSqlDiskFullTest.TestDiskFull

Reviewers: yyan, rthallam

Reviewed By: yyan

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D37651
@rthallamko3 rthallamko3 mentioned this issue Sep 26, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hari90 hari90

Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hari90 @yugabyte-ci