Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: restore optimization for inline puts/deletes #112586

Merged
merged 1 commit into from
Oct 24, 2023

Conversation

sumeerbhola
Copy link
Collaborator

We had an optimization that avoided constructing an iterator over the locktable key space when doing inline puts/deletes. This was lost in ac14ebb that constructed a lockTableKeyScanner even for inline puts/deletes. The optimization is restored here to fix the regression discussed in #111919 (comment).

Informs #111919

Epic: none

Release note: None

@sumeerbhola sumeerbhola requested a review from a team as a code owner October 18, 2023 00:59
@sumeerbhola sumeerbhola requested a review from itsbilal October 18, 2023 00:59
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@sumeerbhola
Copy link
Collaborator Author

test failure is unrelated #111481

Copy link
Collaborator

@jbowens jbowens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @nvanbenschoten)

@sumeerbhola sumeerbhola added the backport-23.2.x Flags PRs that need to be backported to 23.2. label Oct 19, 2023
Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @nvanbenschoten)

@sumeerbhola
Copy link
Collaborator Author

bors r=jbowens

@craig
Copy link
Contributor

craig bot commented Oct 19, 2023

Build failed:

@sumeerbhola
Copy link
Collaborator Author

Failure is unrelated #112659

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for tracking this down!

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @sumeerbhola)


pkg/storage/mvcc.go line 2674 at r1 (raw file):

) (int64, error) {
	if timestamp.IsEmpty() {
		return 0, errors.Errorf("increment not supported for inline values")

Has this always been a restriction or are you introducing it now? What is preventing inline increments from working correctly?

Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @nvanbenschoten)


pkg/storage/mvcc.go line 2674 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Has this always been a restriction or are you introducing it now? What is preventing inline increments from working correctly?

I introduced it. I was assuming that all inline data was stored as MVCCMetadata so wouldn't support increment anyway.
Should I remove this restriction?

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @sumeerbhola)


pkg/storage/mvcc.go line 2674 at r1 (raw file):

I was assuming that all inline data was stored as MVCCMetadata

This is true, but why would that preclude an increment operation over the value in MVCCMetadata.RawBytes?

Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @itsbilal and @nvanbenschoten)


pkg/storage/mvcc.go line 2674 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

I was assuming that all inline data was stored as MVCCMetadata

This is true, but why would that preclude an increment operation over the value in MVCCMetadata.RawBytes?

Ah, I missed that we are doing makeOptionalValue(roachpb.Value{RawBytes: meta.RawBytes}). Removed the restriction.

Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @itsbilal and @sumeerbhola)


pkg/storage/testdata/mvcc_histories/replicated_locks line 455 at r2 (raw file):

meta: "k4"/0,0 -> txn={id=00000001 key=/Min iso=Serializable pri=0.00000000 epo=1 ts=10.000000000,0 min=0,0 seq=2} ts=10.000000000,0 del=false klen=12 vlen=13 ih={{1 /BYTES/v4}} mergeTs=<nil> txnDidNotUpdateMeta=false
data: "k4"/10.000000000,0 -> /BYTES/v4_prime
error: (*withstack.withStack:) "k1"/0,0: put is inline=true, but existing value is inline=false

This error is unrelated to replicated locks, so it would be better if we tested this case and its inverse in mvcc_histories/inline, but I won't block the PR on that.

We had an optimization that avoided constructing an iterator over the
locktable key space when doing inline puts/deletes. This was lost in
cockroachdb@ac14ebb
that constructed a lockTableKeyScanner even for inline puts/deletes.
The optimization is restored here to fix the regression discussed in
cockroachdb#111919 (comment).

Informs cockroachdb#111919

Epic: none

Release note: None
Copy link
Collaborator Author

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @itsbilal)


pkg/storage/testdata/mvcc_histories/replicated_locks line 455 at r2 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

This error is unrelated to replicated locks, so it would be better if we tested this case and its inverse in mvcc_histories/inline, but I won't block the PR on that.

Done

@sumeerbhola
Copy link
Collaborator Author

TFTRs!

@sumeerbhola
Copy link
Collaborator Author

bors r=jbowens,nvanbenschoten

@craig
Copy link
Contributor

craig bot commented Oct 24, 2023

Build succeeded:

@craig craig bot merged commit cbbcefb into cockroachdb:master Oct 24, 2023
3 checks passed
@blathers-crl
Copy link

blathers-crl bot commented Oct 24, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 3b6e770 to blathers/backport-release-23.2-112586: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.2.x Flags PRs that need to be backported to 23.2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants