Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hlc: improve Timestamp.IsEmpty() performance #88911

Merged
merged 1 commit into from
Sep 28, 2022

Conversation

erikgrinaker
Copy link
Contributor

@erikgrinaker erikgrinaker commented Sep 28, 2022

Timestamp.IsEmpty() saw a significant performance regression with Go 1.19, in particular with empty timestamps, apparently due to the compiler's use of memeqbody for struct comparisons. On EncodeMVCCKey benchmarks, the empty timestamp case showed a ~50% regression.

This patch changes IsEmpty() to avoid struct comparison, instead comparing the member fields directly. This yields a significant performance improvement (using Go 1.19):

name                          old time/op  new time/op  delta
TimestampIsEmpty/empty-24     6.90ns ± 2%  0.79ns ± 0%  -88.56%  (p=0.000 n=10+9)
TimestampIsEmpty/walltime-24  2.97ns ± 1%  0.75ns ± 1%  -74.90%  (p=0.000 n=10+10)
TimestampIsEmpty/all-24       2.97ns ± 1%  0.75ns ± 1%  -74.77%  (p=0.000 n=10+10)

This in turn significantly improves EncodeMVCCKey performance:

name                                            old time/op  new time/op  delta
EncodeMVCCKey/key=empty/ts=empty-24             22.0ns ± 0%  15.7ns ± 5%  -28.67%  (p=0.000 n=9+10)
EncodeMVCCKey/key=empty/ts=walltime-24          21.2ns ± 0%  17.8ns ± 0%  -16.42%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24  21.8ns ± 0%  18.6ns ± 0%  -15.02%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=all-24               22.2ns ± 0%  18.8ns ± 0%  -15.07%  (p=0.000 n=9+9)
EncodeMVCCKey/key=short/ts=empty-24             22.3ns ± 0%  15.8ns ± 0%  -29.06%  (p=0.000 n=8+10)
EncodeMVCCKey/key=short/ts=walltime-24          21.2ns ± 0%  18.0ns ± 0%  -15.02%  (p=0.000 n=9+8)
EncodeMVCCKey/key=short/ts=walltime+logical-24  22.6ns ± 0%  18.8ns ± 0%  -16.53%  (p=0.000 n=9+8)
EncodeMVCCKey/key=short/ts=all-24               22.6ns ± 0%  19.1ns ± 0%  -15.32%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=empty-24              65.2ns ± 1%  58.6ns ± 0%  -10.11%  (p=0.000 n=9+9)
EncodeMVCCKey/key=long/ts=walltime-24           62.6ns ± 0%  59.6ns ± 1%   -4.72%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=walltime+logical-24   63.5ns ± 0%  60.2ns ± 0%   -5.16%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=all-24                63.9ns ± 0%  59.6ns ± 0%   -6.81%  (p=0.000 n=9+9)

It also yields a nice improvement over 22.1 with Go 1.18:

name                                            old time/op  new time/op  delta
EncodeMVCCKey/key=empty/ts=empty-24             15.7ns ± 0%  15.7ns ± 5%     ~     (p=0.497 n=8+10)
EncodeMVCCKey/key=empty/ts=walltime-24          19.5ns ± 0%  17.8ns ± 0%   -8.99%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24  20.2ns ± 0%  18.6ns ± 0%   -8.03%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=all-24               20.4ns ± 0%  18.8ns ± 0%   -7.67%  (p=0.000 n=9+9)
EncodeMVCCKey/key=short/ts=empty-24             16.0ns ± 0%  15.8ns ± 0%   -1.47%  (p=0.000 n=10+10)
EncodeMVCCKey/key=short/ts=walltime-24          20.6ns ± 0%  18.0ns ± 0%  -12.51%  (p=0.000 n=8+8)
EncodeMVCCKey/key=short/ts=walltime+logical-24  20.9ns ± 0%  18.8ns ± 0%   -9.82%  (p=0.000 n=10+8)
EncodeMVCCKey/key=short/ts=all-24               20.9ns ± 0%  19.1ns ± 0%   -8.79%  (p=0.000 n=9+10)
EncodeMVCCKey/key=long/ts=walltime-24           60.4ns ± 0%  59.6ns ± 1%   -1.30%  (p=0.000 n=9+10)
EncodeMVCCKey/key=long/ts=walltime+logical-24   61.4ns ± 0%  60.2ns ± 0%   -1.84%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=all-24                61.4ns ± 0%  59.6ns ± 0%   -2.88%  (p=0.000 n=10+9)
EncodeMVCCKey/key=long/ts=empty-24              59.1ns ± 0%  58.6ns ± 0%   -0.77%  (p=0.000 n=10+9)

Resolves #88818.

Release note: None

@erikgrinaker erikgrinaker self-assigned this Sep 28, 2022
@cockroach-teamcity
Copy link
Member

This change is Reviewable

`Timestamp.IsEmpty()` saw a significant performance regression with Go
1.19, in particular with empty timestamps, apparently due to the
compiler's use of `memeqbody` for struct comparisons. On `EncodeMVCCKey`
benchmarks, the empty timestamp case showed a ~50% regression.

This patch changes `IsEmpty()` to avoid struct comparison, instead
comparing the member fields directly. This yields a significant
performance improvement (using Go 1.19):

```
name                          old time/op  new time/op  delta
TimestampIsEmpty/empty-24     6.90ns ± 2%  0.79ns ± 0%  -88.56%  (p=0.000 n=10+9)
TimestampIsEmpty/walltime-24  2.97ns ± 1%  0.75ns ± 1%  -74.90%  (p=0.000 n=10+10)
TimestampIsEmpty/all-24       2.97ns ± 1%  0.75ns ± 1%  -74.77%  (p=0.000 n=10+10)
```

This in turn significantly improves `EncodeMVCCKey` performance:

```
name                                            old time/op  new time/op  delta
EncodeMVCCKey/key=empty/ts=empty-24             22.0ns ± 0%  15.7ns ± 5%  -28.67%  (p=0.000 n=9+10)
EncodeMVCCKey/key=empty/ts=walltime-24          21.2ns ± 0%  17.8ns ± 0%  -16.42%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24  21.8ns ± 0%  18.6ns ± 0%  -15.02%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=all-24               22.2ns ± 0%  18.8ns ± 0%  -15.07%  (p=0.000 n=9+9)
EncodeMVCCKey/key=short/ts=empty-24             22.3ns ± 0%  15.8ns ± 0%  -29.06%  (p=0.000 n=8+10)
EncodeMVCCKey/key=short/ts=walltime-24          21.2ns ± 0%  18.0ns ± 0%  -15.02%  (p=0.000 n=9+8)
EncodeMVCCKey/key=short/ts=walltime+logical-24  22.6ns ± 0%  18.8ns ± 0%  -16.53%  (p=0.000 n=9+8)
EncodeMVCCKey/key=short/ts=all-24               22.6ns ± 0%  19.1ns ± 0%  -15.32%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=empty-24              65.2ns ± 1%  58.6ns ± 0%  -10.11%  (p=0.000 n=9+9)
EncodeMVCCKey/key=long/ts=walltime-24           62.6ns ± 0%  59.6ns ± 1%   -4.72%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=walltime+logical-24   63.5ns ± 0%  60.2ns ± 0%   -5.16%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=all-24                63.9ns ± 0%  59.6ns ± 0%   -6.81%  (p=0.000 n=9+9)
```

It also yields a nice improvement over 22.1 with Go 1.18:

```
name                                            old time/op  new time/op  delta
EncodeMVCCKey/key=empty/ts=empty-24             15.7ns ± 0%  15.7ns ± 5%     ~     (p=0.497 n=8+10)
EncodeMVCCKey/key=empty/ts=walltime-24          19.5ns ± 0%  17.8ns ± 0%   -8.99%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24  20.2ns ± 0%  18.6ns ± 0%   -8.03%  (p=0.000 n=10+10)
EncodeMVCCKey/key=empty/ts=all-24               20.4ns ± 0%  18.8ns ± 0%   -7.67%  (p=0.000 n=9+9)
EncodeMVCCKey/key=short/ts=empty-24             16.0ns ± 0%  15.8ns ± 0%   -1.47%  (p=0.000 n=10+10)
EncodeMVCCKey/key=short/ts=walltime-24          20.6ns ± 0%  18.0ns ± 0%  -12.51%  (p=0.000 n=8+8)
EncodeMVCCKey/key=short/ts=walltime+logical-24  20.9ns ± 0%  18.8ns ± 0%   -9.82%  (p=0.000 n=10+8)
EncodeMVCCKey/key=short/ts=all-24               20.9ns ± 0%  19.1ns ± 0%   -8.79%  (p=0.000 n=9+10)
EncodeMVCCKey/key=long/ts=walltime-24           60.4ns ± 0%  59.6ns ± 1%   -1.30%  (p=0.000 n=9+10)
EncodeMVCCKey/key=long/ts=walltime+logical-24   61.4ns ± 0%  60.2ns ± 0%   -1.84%  (p=0.000 n=10+10)
EncodeMVCCKey/key=long/ts=all-24                61.4ns ± 0%  59.6ns ± 0%   -2.88%  (p=0.000 n=10+9)
EncodeMVCCKey/key=long/ts=empty-24              59.1ns ± 0%  58.6ns ± 0%   -0.77%  (p=0.000 n=10+9)
```

Release note: None
Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @erikgrinaker and @nicktrav)


pkg/util/hlc/timestamp.go line 199 at r1 (raw file):

// gcassert:inline
func (t Timestamp) IsEmpty() bool {
	return t.WallTime == 0 && t.Logical == 0 && !t.Synthetic

Should we add a reminder to ourselves with a comment in timestamp.proto, that states that the code in timestamp.go relies on the set of fields being unchanged, so remember to update if adding fields?
May be overkill given that this is an extremely rarely changed part of the code.

Copy link
Contributor Author

@erikgrinaker erikgrinaker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @nicktrav and @sumeerbhola)


pkg/util/hlc/timestamp.go line 199 at r1 (raw file):

Previously, sumeerbhola wrote…

Should we add a reminder to ourselves with a comment in timestamp.proto, that states that the code in timestamp.go relies on the set of fields being unchanged, so remember to update if adding fields?
May be overkill given that this is an extremely rarely changed part of the code.

We'd have to adapt a bunch of other methods as well, such as Add and Compare, so I think we'd be careful to audit the small set of methods here for any changes.

@erikgrinaker
Copy link
Contributor Author

bors r=sumeerbhola

@craig
Copy link
Contributor

craig bot commented Sep 28, 2022

Build succeeded:

@craig craig bot merged commit e783f14 into cockroachdb:master Sep 28, 2022
@erikgrinaker erikgrinaker deleted the timestamp-isempty branch October 30, 2022 13:10
jbowens added a commit to jbowens/cockroach that referenced this pull request Sep 18, 2023
Use a struct comparison in Timestamp.IsEmpty. The IsEmpty implementation was
modified to perform a field-by-field comparison in cockroachdb#88911 due to a regression
in Go 1.19. The regression was fixed in Go 1.20.

```
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
                             │ old-hlc.txt  │             new-hlc.txt             │
                             │    sec/op    │    sec/op     vs base               │
TimestampIsEmpty/empty-24      0.9341n ± 0%   0.9350n ± 0%       ~ (p=0.197 n=10)
TimestampIsEmpty/walltime-24    1.370n ± 2%    1.367n ± 1%       ~ (p=0.617 n=10)
TimestampIsEmpty/all-24         1.362n ± 1%    1.374n ± 2%       ~ (p=0.092 n=10)
geomean                         1.203n         1.206n       +0.24%

goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
                                               │   old.txt   │              new.txt               │
                                               │   sec/op    │   sec/op     vs base               │
EncodeMVCCKey/key=short/ts=empty-24              17.59n ± 0%   17.59n ± 0%       ~ (p=0.908 n=10)
EncodeMVCCKey/key=short/ts=walltime-24           19.93n ± 0%   19.90n ± 0%  -0.10% (p=0.043 n=10)
EncodeMVCCKey/key=short/ts=walltime+logical-24   20.89n ± 0%   20.91n ± 0%       ~ (p=1.000 n=10)
EncodeMVCCKey/key=long/ts=empty-24               68.58n ± 0%   68.44n ± 0%  -0.20% (p=0.037 n=10)
EncodeMVCCKey/key=long/ts=walltime-24            70.99n ± 1%   70.11n ± 0%  -1.24% (p=0.005 n=10)
EncodeMVCCKey/key=long/ts=walltime+logical-24    70.40n ± 0%   70.49n ± 0%  +0.14% (p=0.019 n=10)
EncodeMVCCKey/key=empty/ts=empty-24              17.23n ± 0%   17.24n ± 0%       ~ (p=0.290 n=10)
EncodeMVCCKey/key=empty/ts=walltime-24           19.53n ± 0%   19.54n ± 0%       ~ (p=0.362 n=10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24   20.60n ± 0%   20.61n ± 0%       ~ (p=0.116 n=10)
geomean                                          29.59n        29.55n       -0.13%
```

Epic: None
Informs cockroachdb#89199.
Release note: None
craig bot pushed a commit that referenced this pull request Sep 18, 2023
110808: hlc: use struct comparison for Timestamp.IsEmpty r=erikgrinaker a=jbowens

Use a struct comparison in Timestamp.IsEmpty. The IsEmpty implementation was modified to perform a field-by-field comparison in #88911 due to a regression in Go 1.19. The regression was fixed in Go 1.20.

```
goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
                             │ old-hlc.txt  │             new-hlc.txt             │
                             │    sec/op    │    sec/op     vs base               │
TimestampIsEmpty/empty-24      0.9341n ± 0%   0.9350n ± 0%       ~ (p=0.197 n=10)
TimestampIsEmpty/walltime-24    1.370n ± 2%    1.367n ± 1%       ~ (p=0.617 n=10)
TimestampIsEmpty/all-24         1.362n ± 1%    1.374n ± 2%       ~ (p=0.092 n=10)
geomean                         1.203n         1.206n       +0.24%

goos: linux
goarch: amd64
cpu: Intel(R) Xeon(R) CPU @ 2.30GHz
                                               │   old.txt   │              new.txt               │
                                               │   sec/op    │   sec/op     vs base               │
EncodeMVCCKey/key=short/ts=empty-24              17.59n ± 0%   17.59n ± 0%       ~ (p=0.908 n=10)
EncodeMVCCKey/key=short/ts=walltime-24           19.93n ± 0%   19.90n ± 0%  -0.10% (p=0.043 n=10)
EncodeMVCCKey/key=short/ts=walltime+logical-24   20.89n ± 0%   20.91n ± 0%       ~ (p=1.000 n=10)
EncodeMVCCKey/key=long/ts=empty-24               68.58n ± 0%   68.44n ± 0%  -0.20% (p=0.037 n=10)
EncodeMVCCKey/key=long/ts=walltime-24            70.99n ± 1%   70.11n ± 0%  -1.24% (p=0.005 n=10)
EncodeMVCCKey/key=long/ts=walltime+logical-24    70.40n ± 0%   70.49n ± 0%  +0.14% (p=0.019 n=10)
EncodeMVCCKey/key=empty/ts=empty-24              17.23n ± 0%   17.24n ± 0%       ~ (p=0.290 n=10)
EncodeMVCCKey/key=empty/ts=walltime-24           19.53n ± 0%   19.54n ± 0%       ~ (p=0.362 n=10)
EncodeMVCCKey/key=empty/ts=walltime+logical-24   20.60n ± 0%   20.61n ± 0%       ~ (p=0.116 n=10)
geomean                                          29.59n        29.55n       -0.13%
```

Epic: None
Informs #89199.
Release note: None

Co-authored-by: Jackson Owens <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storage: performance regression in EncodeMVCCKey and EncodeMVCCValue
3 participants