Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kv: implement allocator support for evaluating decommission viability #91570

Closed
AlexTalks opened this issue Nov 9, 2022 · 0 comments
Closed
Assignees
Labels
A-kv-decom-rolling-restart Decommission and Rolling Restarts A-kv-distribution Relating to rebalancing and leasing. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@AlexTalks
Copy link
Contributor

AlexTalks commented Nov 9, 2022

As part of #90752, we need to add support to the allocator to be able to evaluate the viability of node decommission, by validating that we have valid upreplication targets for all replicas on the node we intend to decommission. In order to do this, we need to add allocator support for determining actions and allocation targets with states not currently represented in the StorePool.

Part of #91570.

Jira issue: CRDB-21321

@AlexTalks AlexTalks added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv-distribution Relating to rebalancing and leasing. A-kv-decom-rolling-restart Decommission and Rolling Restarts T-kv KV Team labels Nov 9, 2022
@AlexTalks AlexTalks self-assigned this Nov 9, 2022
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 9, 2022
This change refactors the usage of `StorePool` in the allocator to a new
interface, `AllocatorStorePool`, in order to be able to utilize a store
pool with overriden liveness to properly evaluate decommission
pre-flight checks.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 15, 2022
This change refactors the usage of `StorePool` in the allocator to a new
interface, `AllocatorStorePool`, in order to be able to utilize a store
pool with overriden liveness to properly evaluate decommission
pre-flight checks.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 15, 2022
This change refactors the usage of `StorePool` in the allocator to a new
interface, `AllocatorStorePool`, in order to be able to utilize a store
pool with overriden liveness to properly evaluate decommission
pre-flight checks.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 16, 2022
While previously the allocator only evaluated using liveness obtained
from gossip, this change introduces a new `OverrideStorePool` struct
which can be used to override the liveness of a node for the purposes of
evaluating allocator actions and targets.  This `OverrideStorePool` is
backed by an existing actual `StorePool`, which retains the majority of
its logic.

Depends on cockroachdb#91461.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 16, 2022
This change adds methods to be to evaluate allocator actions and targets
utilizing a passed-in `StorePool` object, allowing for the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Depends on cockroachdb#91461, cockroachdb#91965.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 17, 2022
This change refactors the usage of `StorePool` in the allocator to a new
interface, `AllocatorStorePool`, in order to be able to utilize a store
pool with overriden liveness to properly evaluate decommission
pre-flight checks.

Part of cockroachdb#91570.

Release note: None
craig bot pushed a commit that referenced this issue Nov 17, 2022
91461: allocator: refactor StorePool usage to interface r=AlexTalks a=AlexTalks

This change refactors the usage of `StorePool` in the allocator to a new interface, `AllocatorStorePool`, in order to be able to utilize a store pool with overriden liveness to properly evaluate decommission pre-flight checks.

Part of #91570.

Release note: None

91681: sql/opt: Support oid type in foldStringToRegclassCast r=rafiss a=e-mbrown

Informs: #91022

This commit adds allows queries like
`select * from pg_class where oid ='my_table'::regclass::oid` to use index on `pg_class(oid)`.

Release note: None

91860: ui: rename "active" execution types and functions to "recent" r=amyyq2 a=amyyq2

This change renames the active execution types and functions to "recent" types and functions. This is in preparation for the addition of recent executions to the current active executions table.

The file names will be changed in a later PR.

Release note: None

91951: kvserver,logstore: move log appends to logstore r=tbg a=pavelkalinnikov

This change moves a few components related to storing the newly appended Raft entries from `kvserver` to the `logstore` package.

Part of #91979
Release note: None

92073: ci: use GCS instead of S3 to download binaries r=rickystewart a=rail

Previously, CI used S3 to download the binaries. Now that we are moving the primary location to GCS and will stop uploading to s3 at some point, it's time to start using GCS for this operation.

Part of RE-342
Release note: None

Co-authored-by: Alex Sarkesian <[email protected]>
Co-authored-by: e-mbrown <[email protected]>
Co-authored-by: Amy Qian <[email protected]>
Co-authored-by: amyyq2 <[email protected]>
Co-authored-by: Pavel Kalinnikov <[email protected]>
Co-authored-by: Rail Aliiev <[email protected]>
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 18, 2022
While previously the allocator only evaluated using liveness obtained
from gossip, this change introduces a new `OverrideStorePool` struct
which can be used to override the liveness of a node for the purposes of
evaluating allocator actions and targets.  This `OverrideStorePool` is
backed by an existing actual `StorePool`, which retains the majority of
its logic.

Depends on cockroachdb#91461.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Nov 18, 2022
This change adds methods to be to evaluate allocator actions and targets
utilizing a passed-in `StorePool` object, allowing for the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Depends on cockroachdb#91461, cockroachdb#91965.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 21, 2022
This fully decouples the StorePool from the allocator by moving
ownership of the configured view of peer stores to the store
configuration (the original source), the store rebalancer, and various
structures in the allocator simulator.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 21, 2022
This change adds methods to evaluate allocator actions and targets
utilizing a passed-in `StorePool` object, allowing for the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Depends on cockroachdb#91461, cockroachdb#91965.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 21, 2022
This fully decouples the StorePool from the allocator by moving
ownership of the configured view of peer stores to the store
configuration (the original source), the store rebalancer, and various
structures in the allocator simulator.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 21, 2022
This change adds methods to evaluate allocator actions and targets
utilizing a passed-in `StorePool` object, allowing for the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Depends on cockroachdb#91461, cockroachdb#91965.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 21, 2022
This fully decouples the StorePool from the allocator by moving
ownership of the configured view of peer stores to the store
configuration (the original source), the store rebalancer, and various
structures in the allocator simulator.

Part of cockroachdb#91570.

Release note: None
craig bot pushed a commit that referenced this issue Dec 22, 2022
91941: kvserver,allocator: decouple allocator from storepool r=AlexTalks a=AlexTalks

This change refactors the allocator to accept the store pool as an
argument, moving it closer to being stateless and purely functional.
This is done by moving to using the storepool through its primary 
owner (the store configuration), or through references held by 
users of the allocator.  The refactoring enables the the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Part of #91570.

Release Note: None

94045: storage: fix `CheckSSTConflicts` handling of MVCC range keys r=erikgrinaker a=erikgrinaker

`CheckSSTConflicts` keeps an SST and engine iterator in sync while checking for conflicts. To avoid seek costs, it attempts to step the engine iterator in case the corresponding SST key is nearby. However, this stepping could step right past an MVCC range key overlapping the SST key, which would break conflict checks with MVCC range tombstones.

This patch seeks the engine iterator back to the SST key if we step past it and the range key changed.

Resolves #93968.

Release note (bug fix): When experimental MVCC range tombstones are enabled (they're disabled by default), a bulk ingestion (e.g. an import) could in some situations fail to properly check for conflicts with existing MVCC range tombstones. This could cause the ingestion to write below a recently written MVCC range tombstone, in turn losing the ingested data. This could only happen in rare circumstances where a bulk ingestion was applied concurrently with an import cancellation.

Co-authored-by: Alex Sarkesian <[email protected]>
Co-authored-by: Erik Grinaker <[email protected]>
adityamaru pushed a commit to adityamaru/cockroach that referenced this issue Dec 22, 2022
This change adds methods to evaluate allocator actions and targets
utilizing a passed-in `StorePool` object, allowing for the allocator to
consider potential scenarios rather than those simply based on the
current liveness.

Depends on cockroachdb#91461, cockroachdb#91965.

Part of cockroachdb#91570.

Release note: None
adityamaru pushed a commit to adityamaru/cockroach that referenced this issue Dec 22, 2022
This fully decouples the StorePool from the allocator by moving
ownership of the configured view of peer stores to the store
configuration (the original source), the store rebalancer, and various
structures in the allocator simulator.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Dec 22, 2022
While previously we checked that we can avoid a fragile quorum during
the addition of new voters in the replicate queue, this change moves the
check logic into the allocator code itself, allowing it to be reused by
other users of the allocator. This enables us to perform this check when
evaluating decommission viability, or anywhere else that uses the
allocator for new voter replicas.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Jan 5, 2023
While previously we checked that we can avoid a fragile quorum during
the addition of new voters in the replicate queue, this change moves the
check logic into the allocator code itself, allowing it to be reused by
other users of the allocator. This enables us to perform this check when
evaluating decommission viability, or anywhere else that uses the
allocator for new voter replicas.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Jan 7, 2023
While previously we checked that we can avoid a fragile quorum during
the addition of new voters in the replicate queue, this change moves the
check logic into the allocator code itself, allowing it to be reused by
other users of the allocator. This enables us to perform this check when
evaluating decommission viability, or anywhere else that uses the
allocator for new voter replicas.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Jan 7, 2023
This change exposes support via a store for checking the allocator
action and upreplication target (if applicable) for any range descriptor.
The range does not need to have a replica on the given store, nor is it
required to evaluate given the current state of the cluster (i.e. the
store's configured StorePool), as a store pool override can be
provided in order to evaluate possible future states.

Depends on cockroachdb#94114.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Jan 10, 2023
This change exposes support via a store for checking the allocator
action and upreplication target (if applicable) for any range descriptor.
The range does not need to have a replica on the given store, nor is it
required to evaluate given the current state of the cluster (i.e. the
store's configured StorePool), as a store pool override can be
provided in order to evaluate possible future states.

Depends on cockroachdb#94114.

Part of cockroachdb#91570.

Release note: None
AlexTalks added a commit to AlexTalks/cockroach that referenced this issue Jan 10, 2023
This change exposes support via a store for checking the allocator
action and upreplication target (if applicable) for any range descriptor.
The range does not need to have a replica on the given store, nor is it
required to evaluate given the current state of the cluster (i.e. the
store's configured StorePool), as a store pool override can be
provided in order to evaluate possible future states.

Depends on cockroachdb#94114.

Part of cockroachdb#91570.

Release note: None
craig bot pushed a commit that referenced this issue Jan 10, 2023
94024: kvserver: support checking allocator action and target by range r=AlexTalks a=AlexTalks

This change exposes support via a store for checking the allocator
action and upreplication target (if applicable) for any range descriptor.
The range does not need to have a replica on the given store, nor is it
required to evaluate given the current state of the cluster (i.e. the
store's configured StorePool), as a store pool override can be
provided in order to evaluate possible future states.

Depends on #94114.

Part of #91570.

Release note: None

Co-authored-by: Alex Sarkesian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-decom-rolling-restart Decommission and Rolling Restarts A-kv-distribution Relating to rebalancing and leasing. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
None yet
Development

No branches or pull requests

1 participant