Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/catalog/descs: remove allocations from hot path #88614

Merged
merged 1 commit into from
Oct 4, 2022

Conversation

ajwerner
Copy link
Contributor

The lookup by ID path gets called constantly. This was over 1% of objects allocated in some workloads. Here's a microbenchmark:

name                                                                    old time/op    new time/op    delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16    2.62µs ± 1%    2.18µs ± 1%   -16.63%  (p=0.000 n=10+8)

name                                                                    old alloc/op   new alloc/op   delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16      150B ± 0%        4B ± 0%   -97.33%  (p=0.001 n=8+9)

name                                                                    old allocs/op  new allocs/op  delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16      12.0 ± 0%       0.0       -100.00%  (p=0.000 n=10+10)

Release note: None

@ajwerner ajwerner requested a review from a team September 23, 2022 20:01
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@ajwerner
Copy link
Contributor Author

@postamar this one seems like something for you to look at.

Copy link
Contributor

@postamar postamar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this seems like an easy win.

// slice containing the correspondence between an index to a validation level.
type validationLevelMap struct {
m util.FastIntMap
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to use this in the implementation of StoredCatalog as well!

flags tree.CommonLookupFlags,
descs []catalog.Descriptor,
ids ...descpb.ID,
) (err error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no point in maintaining both getDescriptorsByID and getDescriptorsByIDWithSlice now. Get rid of the former, take the latter and rename it to the former, and inline it everywhere. If need be define a helper getDescriptorByID which returns a (desc, err) pair.

The lookup by ID path gets called constantly. This was over 1% of objects
allocated in some workloads. Here's a microbenchmark:

```
name                                                                    old time/op    new time/op    delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16    2.62µs ± 1%    2.18µs ± 1%   -16.63%  (p=0.000 n=10+8)

name                                                                    old alloc/op   new alloc/op   delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16      150B ± 0%        4B ± 0%   -97.33%  (p=0.001 n=8+9)

name                                                                    old allocs/op  new allocs/op  delta
ResolveExistingObject/CREATE_SCHEMA_sc;CREATE_TABLE_sc.foo_()sc.foo-16      12.0 ± 0%       0.0       -100.00%  (p=0.000 n=10+10)
```

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/optimize-descs-col branch from 6ea236f to c0b535c Compare October 4, 2022 02:32
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @postamar)


pkg/sql/catalog/descs/descriptor.go line 109 at r1 (raw file):

Previously, postamar (Marius Posta) wrote…

There's no point in maintaining both getDescriptorsByID and getDescriptorsByIDWithSlice now. Get rid of the former, take the latter and rename it to the former, and inline it everywhere. If need be define a helper getDescriptorByID which returns a (desc, err) pair.

Honestly, having two functions here is helpful. Not returning the slice is painful because you have to allocate the slice etc. I refactored it and it felt somewhat worse. I did take your advice on the single descriptor helper. I made it a function to clear up any confusion. I think it reads fine.


pkg/sql/catalog/descs/descriptor.go line 561 at r1 (raw file):

Previously, postamar (Marius Posta) wrote…

You might want to use this in the implementation of StoredCatalog as well!

I'm not that worried down there -- if we're going to issue an RPC, what's a slice allocation? We can do it when a profile asks for it.

@ajwerner
Copy link
Contributor Author

ajwerner commented Oct 4, 2022

TFTR!

bors r=postamar

@craig
Copy link
Contributor

craig bot commented Oct 4, 2022

Build succeeded:

@craig craig bot merged commit 60928a6 into cockroachdb:master Oct 4, 2022
@blathers-crl
Copy link

blathers-crl bot commented Oct 4, 2022

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from c0b535c to blathers/backport-release-22.1-88614: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants