Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

builtins: new function crdb_internal.ranges_in_span #97537

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

THardy98
Copy link

Part of: #96011

This PR introduces a new SRF: crdb_internal.ranges_in_span(start_key, end_key) - returns the set of ranges encompassing this span in the form of (range_id, start_key, end_key)

Example response:

select * from crdb_internal.ranges_in_span('\xf670', '\xf671');
  range_id | start_key | end_key
-----------+-----------+----------
        91 | \xf66f    | \xffff

Release note (sql change): introduce new builtin function `crdb_internal.ranges_in_span(start_key, end_key), returns the set of ranges encompassing the given start and end key.

@THardy98 THardy98 requested a review from a team February 23, 2023 00:01
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Comment on lines +472 to +473
{Name: "start_key", Typ: types.Bytes},
{Name: "end_key", Typ: types.Bytes},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor feature request: If you add another overload that takes a single parameter that is a byte array and then do something like:

				arr := tree.MustBeDArray(args[0])
				if arr.Len() != 2 {
					return nil, errors.New("expected an array of two elements")
				}
				startKey := []byte(tree.MustBeDBytes(arr.Array[0]))
				endKey := []byte(tree.MustBeDBytes(arr.Array[1]))

Then this function composes well with crdb_internal.tenant_span, crdb_internal.table_span, and crdb_internal.index_span`.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added byte array overload

@THardy98 THardy98 force-pushed the ranges_in_span branch 2 times, most recently from 6c292b0 to b6756c3 Compare February 24, 2023 23:13
@THardy98 THardy98 requested a review from stevendanna February 27, 2023 14:45
@THardy98
Copy link
Author

Not sure what's causing the ranges_in_span logic test to fail...

@rafiss rafiss requested a review from ecwall February 27, 2023 16:18
Copy link
Contributor

@ecwall ecwall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @stevendanna and @THardy98)


pkg/sql/planner.go line 841 at r2 (raw file):

// GetRangeIteratorWithinSpan is part of the eval.Planner interface.
func (p *planner) GetRangeIteratorWithinSpan(

I would call this GetRangeDescIterator since the rangedesc.Iterator is different from kvcoord.RangeIterator. You can leave out WithinSpan because that is the only way the rangedesc.Iterator can be created anyways.


pkg/sql/planner.go line 844 at r2 (raw file):

	ctx context.Context, span roachpb.Span,
) (rangedesc.Iterator, error) {
	rangeDescIterator, err := p.execCfg.RangeDescIteratorFactory.NewIterator(ctx, span)

Can simply this to return p.execCfg.RangeDescIteratorFactory.NewIterator(ctx, span) since either rangeDescIterator or err will be nil in the result anyways.


pkg/sql/faketreeeval/evalctx.go line 482 at r2 (raw file):

// GetRangeIteratorWithinSpan is part of the eval.Planner interface.
func (ep *DummyEvalPlanner) GetRangeIteratorWithinSpan(
	ctx context.Context, span roachpb.Span,

These input params don't need names since they are not referenced (see GetRangeDescByID above).


pkg/sql/sem/builtins/generator_builtins.go line 2942 at r2 (raw file):

	var err error
	rs.rangeIter, err = rs.p.GetRangeIteratorWithinSpan(ctx, rs.span)
	if err != nil {

Similarly here you can just do return err in all cases.


pkg/sql/sem/builtins/generator_builtins.go line 3000 at r2 (raw file):

		endKey = []byte(tree.MustBeDBytes(arr.Array[1]))
	}
	if len(args) == 2 {

IMO easier to read as

switch len(args) {
  case 1:
    ...
  case 2:
    ...
  default:
    panic(...)
}

@ecwall
Copy link
Contributor

ecwall commented Feb 27, 2023

Not sure what's causing the ranges_in_span logic test to fail...

Is it failing when you run it locally? You should be able to debug TestTenantLogic_ranges_in_span in goland after running ./dev gen.

Part of: cockroachdb#96011

This PR introduces a new SRF: `crdb_internal.ranges_in_span(start_key,
end_key)` - returns the set of ranges encompassing this span in the form
of `(range_id, start_key, end_key)`

Release note (sql change): introduce new builtin function
`crdb_internal.ranges_in_span(start_key, end_key), returns the set of
ranges encompassing the given start and end key.
Copy link
Author

@THardy98 THardy98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @stevendanna)


pkg/sql/planner.go line 841 at r2 (raw file):

Previously, ecwall (Evan Wall) wrote…

I would call this GetRangeDescIterator since the rangedesc.Iterator is different from kvcoord.RangeIterator. You can leave out WithinSpan because that is the only way the rangedesc.Iterator can be created anyways.

Done


pkg/sql/planner.go line 844 at r2 (raw file):

Previously, ecwall (Evan Wall) wrote…

Can simply this to return p.execCfg.RangeDescIteratorFactory.NewIterator(ctx, span) since either rangeDescIterator or err will be nil in the result anyways.

Done


pkg/sql/faketreeeval/evalctx.go line 482 at r2 (raw file):

Previously, ecwall (Evan Wall) wrote…

These input params don't need names since they are not referenced (see GetRangeDescByID above).

Done


pkg/sql/sem/builtins/generator_builtins.go line 2942 at r2 (raw file):

Previously, ecwall (Evan Wall) wrote…

Similarly here you can just do return err in all cases.

Done


pkg/sql/sem/builtins/generator_builtins.go line 3000 at r2 (raw file):

Previously, ecwall (Evan Wall) wrote…

IMO easier to read as

switch len(args) {
  case 1:
    ...
  case 2:
    ...
  default:
    panic(...)
}

Agreed, I've switched it to use a switch.

Copy link
Author

@THardy98 THardy98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run it via ./dev testlogic --files=ranges_in_span --config=3node-tenant I just seem to be getting a time out:

panic: test timed out after 59m55s

but I don't see what would be causing this.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @stevendanna)

@THardy98 THardy98 requested a review from ecwall February 28, 2023 19:43
@ecwall
Copy link
Contributor

ecwall commented Feb 28, 2023

When I run it via ./dev testlogic --files=ranges_in_span --config=3node-tenant I just seem to be getting a time out:

panic: test timed out after 59m55s

but I don't see what would be causing this.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @stevendanna)

I would step through the new builtin code in the GoLand debugger to figure out where it's getting stuck. If you haven't debugged a logic test in GoLand before, the easiest way is to press the play button and select "Debug" next to the function that was created when you ran ./dev gen: https://github.com/cockroachdb/cockroach/pull/97537/files#diff-319c2c28c607644555fc077735cdaaf56407ec2e8f3fcba9da5edfaa46685b88R1371.

Copy link
Author

@THardy98 THardy98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been running the test via debug and unless I'm missing something (this is my first time doing this), it just seems to run indefinitely.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @stevendanna)

@ecwall
Copy link
Contributor

ecwall commented Mar 1, 2023

I've been running the test via debug and unless I'm missing something (this is my first time doing this), it just seems to run indefinitely.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ecwall and @stevendanna)

When you are free let's meet up so we can figure this out.

@rafiss rafiss added the X-noremind Bots won't notify about PRs with X-noremind label May 9, 2023
@stevendanna stevendanna removed their request for review October 3, 2023 09:33
@cockroach-teamcity
Copy link
Member

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Thomas Hardy seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
X-noremind Bots won't notify about PRs with X-noremind
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants