-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
colexec: miscellaneous cleanup #63371
Conversation
Release note: None
118f803
to
dbc3b22
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r1, 4 of 4 files at r2.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @michae2 and @yuzefovich)
pkg/sql/colexec/colbuilder/execplan.go, line 1537 at r2 (raw file):
ctx context.Context, flowCtx *execinfra.FlowCtx, name string, ) (*mon.BoundAccount, string) { monitorName := r.getMemMonitorName(name + "-limited")
nit: it's a bit confusing that the name (which appears to be the monitor name) is an argument, but a modified monitor name is also returned. Not sure if it'd necessarily be better, but you could change the name string
argument here and in the function below to opName string, pid execinfrapb.ProcessorID
, and pass both and a suffix string
to a getMemMonitorName(opName string, pid execinfrapb.ProcessorID, suffix string)
. (Not sure opName
is the right term, maybe there's something better). This would eliminate some duplicat fmt.Sprintf
's I think.
dbc3b22
to
d893bc0
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @michae2)
pkg/sql/colexec/colbuilder/execplan.go, line 1537 at r2 (raw file):
Previously, mgartner (Marcus Gartner) wrote…
nit: it's a bit confusing that the name (which appears to be the monitor name) is an argument, but a modified monitor name is also returned. Not sure if it'd necessarily be better, but you could change the
name string
argument here and in the function below toopName string, pid execinfrapb.ProcessorID
, and pass both and asuffix string
to agetMemMonitorName(opName string, pid execinfrapb.ProcessorID, suffix string)
. (Not sureopName
is the right term, maybe there's something better). This would eliminate some duplicatfmt.Sprintf
's I think.
I like this idea, done. Overall, this mechanism of monitor name matching is quite brittle, and the code is not very pretty, but at least it does the job :)
d893bc0
to
69bfc37
Compare
TFTR! bors r+ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 2 files at r1, 3 of 4 files at r2, 1 of 1 files at r3.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @yuzefovich)
pkg/sql/colexec/colbuilder/execplan.go, line 1537 at r2 (raw file):
Previously, yuzefovich (Yahor Yuzefovich) wrote…
I like this idea, done. Overall, this mechanism of monitor name matching is quite brittle, and the code is not very pretty, but at least it does the job :)
Nit: Now that you've changed it, I don't think you need the hyphen on "-limited" any more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bors r-
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @mgartner and @michae2)
pkg/sql/colexec/colbuilder/execplan.go, line 1537 at r2 (raw file):
Previously, michae2 (Michael Erickson) wrote…
Nit: Now that you've changed it, I don't think you need the hyphen on "-limited" any more.
Indeed, nice catch!
Canceled. |
Our disk-spilling infrastructure (namely, `diskSpillerBase`) uses the name of the limited memory monitor of its in-memory operator in order to distinguish the "memory budget exceeded" errors between the ones it needs to catch and the ones it needs to propagate further. In particular, previously it was possible that the in-memory chains within two disk-backed sorters created for the external distinct (one in the fallback strategy and another when the output ordering is needed, on top of the external distinct) would have the same memory monitor names. This is now fixed by appending a unique suffix to all names and is enforced via an invariants assertion in the test setup. Note that even with the previous behavior no actual bug would occur because the monitors with the same name were on "different levels of the tree" and there was the correct catcher in-between. So this commit simply restores the assumption that we had implicitly without fixing any production bugs. Release note: None
69bfc37
to
7d8097a
Compare
bors r+ |
Build succeeded: |
colexec: clean up external distinct test a bit
Release note: None
colbuilder: enforce unique memory monitor names
Our disk-spilling infrastructure (namely,
diskSpillerBase
) uses thename of the limited memory monitor of its in-memory operator in order to
distinguish the "memory budget exceeded" errors between the ones it
needs to catch and the ones it needs to propagate further. In
particular, previously it was possible that the in-memory chains within
two disk-backed sorters created for the external distinct (one in the
fallback strategy and another when the output ordering is needed, on top
of the external distinct) would have the same memory monitor names. This
is now fixed by appending a unique suffix to all names and is enforced
via an invariants assertion in the test setup.
Note that even with the previous behavior no actual bug would occur
because the monitors with the same name were on "different levels of the
tree" and there was the correct catcher in-between. So this commit
simply restores the assumption that we had implicitly without fixing any
production bugs.
Release note: None