-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: pool some of the processor allocations #88608
Conversation
bbdc020
to
eaf86e0
Compare
This comment was marked as outdated.
This comment was marked as outdated.
322c829
to
b07d9a3
Compare
71f6f60
to
d016fd2
Compare
First commit is #88831 and should be ignored here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 10 of 10 files at r5, 1 of 1 files at r6, 2 of 2 files at r7.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @DrewKimball, @michae2, and @yuzefovich)
pkg/sql/colexec/columnarizer.go
line 168 at r8 (raw file):
execinfra.ProcStateOpts{ // We append input to inputs to drain below in order to reuse the same // underlying slice from the pooled noopProcessor.
[nit] noopProcessor -> columnarizer
d016fd2
to
cef2220
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTR!
bors r+
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @DrewKimball and @michae2)
pkg/sql/colexec/columnarizer.go
line 168 at r8 (raw file):
Previously, DrewKimball (Drew Kimball) wrote…
[nit] noopProcessor -> columnarizer
Done.
Build failed (retrying...): |
Build failed (retrying...): |
Build failed (retrying...): |
Build failed: |
bors r+ |
Build failed: |
bors r+ |
Build failed: |
This commit fixes the way we reuse the ProcOutputHelpers. Previously, we would not perform the deep reset of the render expressions which could lead to those expressions being live for longer than needed (namely till the helper is reused the next time). This matters when the ProcessorBase is reused which currently only happens when TableReaders are used from the sync.Pool. This probably was a pretty bad issue before the vectorized engine, but nowadays its impact seems minor. Still, it's good to fix it, especially given the following commit which will introduce more pooling of processors. Release note: None
This commit makes it so that we reuse the output row in the ProcOutputHelper if possible. Additionally, it removes a row alloc object since it is not helpful and only leeds to wasteful allocations. It also removes a redundant integer for the number of "internal columns". Release note: None
Alright, these bors failures were legitimate, need to make some tweaks. |
cef2220
to
d2717cc
Compare
I'm still working through some things, so please hold off on reviewing - I'll ping once it's ready for a look. |
01beaa7
to
1328362
Compare
It's not as easy of a fix, so I removed the last commit that exposed a pre-existing bug (will file an issue soon) and will merge just the first two simple commits. bors r+ |
Build succeeded: |
execinfra: fix temporary memory leak of expressions
This commit fixes the way we reuse the ProcOutputHelpers. Previously, we
would not perform the deep reset of the render expressions which could
lead to those expressions being live for longer than needed (namely till
the helper is reused the next time). This matters when the ProcessorBase
is reused which currently only happens when TableReaders are used from
the sync.Pool. This probably was a pretty bad issue before the
vectorized engine, but nowadays its impact seems minor. Still, it's good
to fix it, especially given the following commit which will introduce
more pooling of processors.
Release note: None
execinfra: reuse output row allocation in the helper
This commit makes it so that we reuse the output row in the
ProcOutputHelper if possible. Additionally, it removes a row alloc
object since it is not helpful and only leeds to wasteful allocations.
It also removes a redundant integer for the number of "internal
columns".
Release note: None