Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: fix TestRandomSyntaxFunctions related bugs/flakes #107491

Merged
merged 3 commits into from
Aug 1, 2023

Conversation

fqazi
Copy link
Collaborator

@fqazi fqazi commented Jul 24, 2023

Previously TestRandomSyntaxFunctions could flake due to the following bugs:

  1. crdb_internal.revalidate_unique_constraints_in_all_tables did not handle cancelled contexts properly which could lead to timeouts
  2. crdb_internal.pretty_value could fail on random values could panic decoding, so a recover should be used inside this builtin.
  3. TestRandomSyntaxFunctions was testing crdb_internal.gen_rand_ident, which can take a long time based on the input parameters (specifically when a large number of identifiers is requested)

Fixes: #107410
Fixes: #107411

@fqazi fqazi requested a review from a team as a code owner July 24, 2023 19:31
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@fqazi fqazi force-pushed the uniqueConstraintValidateCancel branch 2 times, most recently from ad7607b to d42c484 Compare July 24, 2023 19:33
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)


pkg/sql/sem/builtins/builtins.go line 5768 at r3 (raw file):

			Fn: func(_ context.Context, _ *eval.Context, args tree.Datums) (d tree.Datum, err error) {
				defer func() {
					if r := recover(); r != nil {

i'm curious about this one - i think it's unexpected that the PrettyPrint function would ever panic. how does that happen?

Copy link
Collaborator Author

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)


pkg/sql/sem/builtins/builtins.go line 5768 at r3 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

i'm curious about this one - i think it's unexpected that the PrettyPrint function would ever panic. how does that happen?

@rafiss DecodeUntaggedUUIDValue and DecodeUntaggedIPAddrValue do not validate the input, so that they will hit out-of-bounds errors. The other approach is to update those code paths to generate errors. Not sure if there are performance considerations for those code paths

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @fqazi)


pkg/sql/check.go line 439 at r1 (raw file):

		// If the context is cancelled, then we should bail out, since
		// the actual revalidate operation might not check anything.
		if err := ctx.Err(); err != nil {

i think sql-queries added this builtin so let's get them to sign off


pkg/sql/sem/builtins/builtins.go line 5768 at r3 (raw file):

Previously, fqazi (Faizan Qazi) wrote…

@rafiss DecodeUntaggedUUIDValue and DecodeUntaggedIPAddrValue do not validate the input, so that they will hit out-of-bounds errors. The other approach is to update those code paths to generate errors. Not sure if there are performance considerations for those code paths

ah i see. i think we should actually do the recover inside of the PrettyPrint function, that way the error reporting is consistent with other kinds of errors that can happen in that function.

fqazi added 2 commits July 25, 2023 14:34
Previously, the function: revalidate_unique_constraints_in_all_tables
did not properly check for cancellation when looping over all
descriptors. This could just lead to a delayed cancellation or
timeouts in tests. To address this, this patch updates
the logic to check for cancellations while loop over
descriptors.

Fixes: cockroachdb#107410, cockroachdb#107411

Release note: None
Previously, when crdb_internal.pretty_value was added,
we didn't handle cases where the decode functions can
run into runtime errors. This can lead to out of bounds
errors in certain cases. To address this, this patch makes
this function safe by adding a recover call with the
internally encountered error.

Release note: None
@fqazi fqazi force-pushed the uniqueConstraintValidateCancel branch from d42c484 to cc7e54e Compare July 25, 2023 18:35
@fqazi fqazi requested a review from a team as a code owner July 25, 2023 18:35
Copy link
Collaborator Author

@fqazi fqazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)


pkg/sql/sem/builtins/builtins.go line 5768 at r3 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

ah i see. i think we should actually do the recover inside of the PrettyPrint function, that way the error reporting is consistent with other kinds of errors that can happen in that function.

Done.

@fqazi fqazi requested review from a team, mgartner and rafiss July 25, 2023 19:50
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1, 2 of 2 files at r5, 1 of 1 files at r6, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @mgartner)

@fqazi
Copy link
Collaborator Author

fqazi commented Jul 31, 2023

bors r+

@fqazi
Copy link
Collaborator Author

fqazi commented Jul 31, 2023

bors r-

Previously, the TestRandomSyntaxFunctions could invoke
crdb_internal.gen_rand_ident with random arguments,
which could be giant number of random identifiers.
To address this, this patch will avoid validating
this function.

Release note: None
@fqazi fqazi force-pushed the uniqueConstraintValidateCancel branch from cc7e54e to 72ac7ea Compare July 31, 2023 21:18
@fqazi
Copy link
Collaborator Author

fqazi commented Aug 1, 2023

bors r+

@craig
Copy link
Contributor

craig bot commented Aug 1, 2023

Build succeeded:

@craig craig bot merged commit 05dd384 into cockroachdb:master Aug 1, 2023
@fqazi
Copy link
Collaborator Author

fqazi commented Aug 1, 2023

blathers backport 23.1

@blathers-crl
Copy link

blathers-crl bot commented Aug 1, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from cf8fb9c to blathers/backport-release-23.1-107491: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.1 failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql/tests: TestRandomSyntaxFunctions failed sql/tests: TestRandomSyntaxFunctions failed
3 participants