-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
71898: schemachanger: inject all dependencies r=postamar a=postamar Although the declarative schema changer had been written with dependency injection in mind, the level of abstraction was still too low for straightforward end-to-end testing. The schema changer doesn't care about kv.Txn, jobs.JobRegistry, etc. Fundamentally, here's how it interacts with the rest of the system: 1. The builder in scbuild: a. The builder queries the catalog for the current descriptor and namespace state. b. The builder also needs a valid tree.SemaContext and tree.EvalContex, but only in a limited way. DDLs don't require the evaluation of built-in functions or the like, meaning that suitable context objects can be built with few indirect dependencies like the codec, session data, etc. 2. The executor in scexec: a. Naturally the executor also needs access to the catalog for reading and writing. In the scmutationexec package these dependencies had already been abstract enough, meaning there was little left to do except formalize read-only vs. mutating dependencies a bit. b. The same holds for the backfill operations, with the IndexBackfiller and JobProgressTracker interfaces. c. The executor needs to know about testing knobs and other debug info. In addition to making dependencies explicit as described above in scbuild.Dependencies and scexec.Dependencies interfaces, this commit also makes dependencies explicit for the actual running of the schema changer, to allow end-to-end testing. Concretely, we added a layer of abstraction around the execution of each phase: 1. For scop.StatementPhase, the schema changer receives a target state, runs the planner to generate ops-stages, and executes these ops. 2. For scop.PreCommitPhase, it does exactly that, but also creates and enqueues a job, which will deal with the subsequent scop.PostCommitPhase asynchronously. This job creation depends on things like the job registry, which we don't care for if we're doing end-to-end testing: in that case we'll want to run the post-commit phase synchronously after this one. 3. For scop.PostCommitPhase, it's like the previous phases, except in the context of the Resume method of the schema change job, which requires a bit of plumbing to update the progress of the job itself, and also to update the descriptors affected by the job, because those hold a reference to this job which serves as a lock, blocking other concurrent schema changes. This new layer of abstraction is defined in the new scrun package. The actual implementations of the dependency interfaces are all moved to a new scdeps package. This commit also introduces test implementations of these dependency interfaces, which so far are used in the builder tests alongside the regular implementations. Adoption in the executor tests, as well as end-to-end testing of the declarative schema changer, will be added in a subsequent commit. Release note: None 71942: util/tracing: remove handling for impossible error r=andreimatei a=andreimatei InjectMetaInto can no longer return an error (and I don't forsee it ever returning an error again), so this patch cleans up the unused retval. Release note: None 72046: dev: override bazel timeout when run under stress r=irfansharif a=irfansharif When running under stress and no timeout is specified, we want to respect the timeout passed down to stress[^1]. Not doing so has bazel default to a timeout based on the test target's size[^2], which is not what we want. [^1]: Through --stress-arg=-maxtime or if nothing is specified, a -maxtime of 0 that's taken as "run forever") [^2]: https://docs.bazel.build/versions/main/test-encyclopedia.html#role-of-the-test-runner Release note: None 72103: deps: bump elastic/gosigar to to v0.14.1 r=stevendanna a=knz This adds support for macOS M1 variants and fixes a couple of bugs. Release note: None Co-authored-by: Marius Posta <[email protected]> Co-authored-by: Andrei Matei <[email protected]> Co-authored-by: irfan sharif <[email protected]> Co-authored-by: Raphael 'kena' Poss <[email protected]>
- Loading branch information
Showing
68 changed files
with
3,216 additions
and
1,552 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.