Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crl-release-23.2: internal/keyspan: obey Seek invariants in filteringIter #3049

Merged

Conversation

itsbilal
Copy link
Member

@itsbilal itsbilal commented Nov 9, 2023

23.2 Backport of #3046.


Previously if filteringIter's FilterFunc mutated the passed-in span to no longer be a valid return value for a SeekLT or SeekGE call, we would still return that span even though it could be >= the seek key (for SeekLT), or less than it (for SeekGE).

This change updates filteringIter to guard for this case before returning from a seek call.

Found with #3044.

Informs cockroachdb/cockroach#113973, cockroachdb/cockroach#114056.

@itsbilal itsbilal requested review from jbowens and a team November 9, 2023 16:21
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @itsbilal and @jbowens)


internal/keyspan/filter.go line 60 at r1 (raw file):

	// no knowledge of the seek key) mutated the span to start at a key greater
	// than or equal to `key`. Detect this case and prev/invalidate the iter.
	if span != nil && i.cmp(span.Start, key) >= 0 {

Should this be > 0? span.Start is inclusive, no?

@RaduBerinde
Copy link
Member

internal/keyspan/filter.go line 49 at r1 (raw file):

	// less than or equal to `key`. Detect this case and next/invalidate the iter.
	if span != nil && i.cmp(span.End, key) <= 0 {
		return i.Next()

Is it not possible for i.Next() to still be less than the key? Like if we are seeking to a0 and we have keys a1, a2 which the filter truncates to a?

Copy link
Member Author

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: 0 of 6 files reviewed, 2 unresolved discussions (waiting on @jbowens and @RaduBerinde)


internal/keyspan/filter.go line 49 at r1 (raw file):

Previously, RaduBerinde wrote…

Is it not possible for i.Next() to still be less than the key? Like if we are seeking to a0 and we have keys a1, a2 which the filter truncates to a?

We're relying on the invariant that the underlying iter returned a span that was the correct SeekGE/SeekLT span, but filter excluded it, so a Next would take us to something that's strictly forward. Right?


internal/keyspan/filter.go line 60 at r1 (raw file):

Previously, RaduBerinde wrote…

Should this be > 0? span.Start is inclusive, no?

SeekLT is supposed to return a span that covers a key strictly less than the seek key, so >= is correct to catch the case that's wrong.

@RaduBerinde
Copy link
Member

internal/keyspan/filter.go line 49 at r1 (raw file):

Previously, itsbilal (Bilal Akhtar) wrote…

We're relying on the invariant that the underlying iter returned a span that was the correct SeekGE/SeekLT span, but filter excluded it, so a Next would take us to something that's strictly forward. Right?

Can't the filter exclude the next one too? I guess I don't understand what's special about the first key returned by Seek that doesn't apply to the following key

Previously if filteringIter's FilterFunc mutated the passed-in span
to no longer be a valid return value for a SeekLT or SeekGE call,
we would still return that span even though it could be >=  the seek
key (for SeekLT), or less than it (for SeekGE).

This change updates filteringIter to guard for this case before
returning from a seek call.

Found with cockroachdb#3044.

Informs cockroachdb/cockroach#113973, cockroachdb/cockroach#114056.
@itsbilal itsbilal force-pushed the filtering-iter-fix-23.2 branch from 48360e6 to cb7ca3f Compare November 9, 2023 17:14
Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @itsbilal and @jbowens)


internal/keyspan/filter.go line 49 at r1 (raw file):

Previously, RaduBerinde wrote…

Can't the filter exclude the next one too? I guess I don't understand what's special about the first key returned by Seek that doesn't apply to the following key

I guess it's not possible with the truncate, because only the first span can intersect the upper bound.. but I'm less clear what is the filter contract in general, w.r.t modification of the span.

Copy link
Member Author

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @jbowens and @RaduBerinde)


internal/keyspan/filter.go line 49 at r1 (raw file):

Previously, RaduBerinde wrote…

I guess it's not possible with the truncate, because only the first span can intersect the upper bound.. but I'm less clear what is the filter contract in general, w.r.t modification of the span.

Yeah. You're right that this covers the Truncate case, but doesn't generally solve a FilterFunc case for some contracts of FilterFunc that are currently completely not spelled out. We've filed #3048 to separate the two implementations anyway. so I'm okay with a Truncate-only fix for now.

Copy link
Member

@RaduBerinde RaduBerinde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @itsbilal and @jbowens)


internal/keyspan/filter.go line 49 at r1 (raw file):

Previously, itsbilal (Bilal Akhtar) wrote…

Yeah. You're right that this covers the Truncate case, but doesn't generally solve a FilterFunc case for some contracts of FilterFunc that are currently completely not spelled out. We've filed #3048 to separate the two implementations anyway. so I'm okay with a Truncate-only fix for now.

Awesome! Makes sense now!

Copy link
Member Author

@itsbilal itsbilal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: 0 of 6 files reviewed, 1 unresolved discussion (waiting on @jbowens)

@itsbilal itsbilal merged commit c4f530d into cockroachdb:crl-release-23.2 Nov 9, 2023
10 checks passed
@itsbilal itsbilal deleted the filtering-iter-fix-23.2 branch November 9, 2023 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants