server: name sqlmigrations.Manager vars more appropriately #56474
+10
−10
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We intend to use "migration manager" for the more general purpose migration infrastructure we're introducing as part of long running migrations. Release note: None
|
This change is |
This was referenced Nov 10, 2020
tbg
approved these changes
Nov 10, 2020
There was a problem hiding this comment.
Reviewed 3 of 3 files at r1.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @knz)
knz
approved these changes
Nov 10, 2020
There was a problem hiding this comment.
Reviewed 3 of 3 files at r1.
Reviewable status:
|
bors r+ |
|
Build succeeded: |
craig bot
pushed a commit
that referenced
this pull request
Nov 11, 2020
56476: server: introduce the `Migration` service r=irfansharif a=irfansharif The upcoming migration manager (prototyped in #56107) will want to execute a few known RPCs on every node in the cluster. Part of being the "migration infrastructure", we also want authors of individual migrations to be able to define arbitrary node-level operations to execute on each node in the system. To this end we introduce a `Migration` service, and populate it with the two known RPCs the migration manager will want to depend on: - ValidateTargetClusterVersion: used to verify that the target node is running a binary that's able to support the given cluster version. - BumpClusterVersion: used to inform the target node about a (validated) cluster version bump. Both these RPCs are not currently wired up to anything, and BumpClusterVersion will be fleshed out just a tiny bit further in a future PR, but they'll both be used to propagate cluster version bumps across the crdb cluster through direct RPCs, supplanting our existing gossip based distribution mechanism. This will let the migration manager bump version gates in a more controlled fashion. See #56107 for what that will end up looking like, and see the long-running migrations RFC (#48843) for the motivation. Like we mentioned earlier, we expect this service to pick up more RPCs over time to service specific migrations. Release note: None --- Ignore the first four commits. They're from #56368 and #56474 Co-authored-by: irfan sharif <[email protected]>
craig bot
pushed a commit
that referenced
this pull request
Nov 11, 2020
56480: settings,migration: disconnect cluster version from gossip r=irfansharif a=irfansharif ...in favor of direct RPCs to all nodes in the cluster. It uses the building blocks we've added thus far to replace the use of gossip to disseminate the cluster version. It does so by sending out individual RPCs to each node in the cluster, informing them of a version bump, all the while retaining the same guarantees provided by our (now previously) gossip-backed mechanism. This is another in the series of PRs to introduce long running migrations (#48843), pulled out of our original prototype in #56107. This diff has the following "pieces": - We disconnect the version setting updates through gossip (by disconnecting the setting type within the updater process) - We use the `Migration` service to send out RPCs to each node in the cluster, containing the payload that each node would otherwise receive through gossip. We do this by first introducing two primitives in pkg/migrations: - `RequiredNodes` retrieves a list of all nodes that are part of the cluster. It's powered by `pkg/../liveness`. - `EveryNode` is a shorthand that allows us to send out node-level migration RPCs to every node in the cluster. We combine these primitives with the RPCs introduced in #56476 (`ValidateTargetClusterVersion`, `BumpClusterVersion`) to actually carry out the version bumps. - We expand the `clusterversion.Handle` interface to allow setting the active version directly through it. We then make use of it in the implementation for `BumpClusterVersion`. - Within `BumpClusterVersion`, we persists the cluster version received from the client node first, within `keys.StoreClusterVersionKey`, before bumping the version gate. This is a required invariant in the system in order for us to not regress our cluster version on restart. It was previously achieved by attaching callbacks on the version handle (`SetBeforeChange`). - We no longer need the callbacks attached to gossip to persist cluster versions to disk. We're doing it as part of the `BumpClusterVersion` RPC. We remove them entirely. - We use the active version provided by the join RPC to set the version setting directly (after having persisted it first). This too was previously achieved through gossip + the callback. Release note: None --- Only the last commit is of interest. All prior commits should be reviewed across #56476, #56474 and #56368. Co-authored-by: irfan sharif <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We intend to use "migration manager" for the more general purpose
migration infrastructure we're introducing as part of long running
migrations.
Release note: None