Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/sem: add check for interval for asof.DatumToHLC() #93146

Merged
merged 1 commit into from
Dec 10, 2022

Conversation

ZhouXing19
Copy link
Collaborator

@ZhouXing19 ZhouXing19 commented Dec 6, 2022

Currently, if the given interval for AS OF SYSTEM TIME interval is a small postive duration, the query can incorrectly pass. It's because when we call clock.Now(), it has passed the threashold ('statement timestamp + duration').

Now we add a check for the duration value. It has to be negative, with the absolute value greater than a nanosecond.

fixes #91021
link epic CRDB-17785

Release note (bug fix): add restriction for duration value for AS OF SYSTEM TIME statement.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@ZhouXing19 ZhouXing19 requested a review from a team December 6, 2022 18:36
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ZhouXing19)


pkg/sql/sem/asof/as_of.go line 253 at r1 (raw file):

		if iv, err := tree.ParseDInterval(evalCtx.GetIntervalStyle(), s); err == nil {
			if (iv.Duration == duration.Duration{}) {
				convErr = errors.Errorf("interval value %v too small, absolute value must be >= %v", d, time.Microsecond)

is time.Microsecond correct in this earlier message?

actually, we can just remove this if case if we change the below ones to iv.Duration.Compare(duration.Duration{}) >= 0 and iv.Duration.Compare(duration.Duration{}) <= 0


pkg/sql/sem/asof/as_of.go line 255 at r1 (raw file):

				convErr = errors.Errorf("interval value %v too small, absolute value must be >= %v", d, time.Microsecond)
			} else if (usage == AsOf && iv.Duration.Compare(duration.Duration{}) > 0 && !syn) {
				convErr = errors.Wrapf(

i think this should just be:

convErr = errors.Newf("interval value %v too large, AS OF interval must be <= %v", d, -time.Nanosecond)

pkg/sql/sem/asof/as_of.go line 261 at r1 (raw file):

				)
			} else if (usage == Split && iv.Duration.Compare(duration.Duration{}) < 0 && !syn) {
				convErr = errors.Wrapf(

i think this should just be:

convErr = errors.Newf("interval value %v too small, SPLIT AT interval must be >= %v", d, time.Nanosecond)

pkg/sql/sem/asof/as_of.go line 290 at r1 (raw file):

				errors.Newf("unsupported interval duration: %v", d),
				"expiration interval must be non-negative",
			)

same comment on error messages (and using <= 0 and >= 0 in the ifs

@ZhouXing19 ZhouXing19 force-pushed the fix-asot-1206 branch 4 times, most recently from 3232246 to 0f61337 Compare December 7, 2022 19:02
Copy link
Collaborator Author

@ZhouXing19 ZhouXing19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)


pkg/sql/sem/asof/as_of.go line 253 at r1 (raw file):

is time.Microsecond correct in this earlier message?

Ah yes, it was correct, I made a typo. Changed them back.

change the below ones to iv.Duration.Compare(duration.Duration{}) >= 0 and iv.Duration.Compare(duration.Duration{}) <= 0

I separated the == case because we always disallow it, but we actually allow > for AS OF and < for SPLIT if the interval is synthetic ,see here.


pkg/sql/sem/asof/as_of.go line 255 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

i think this should just be:

convErr = errors.Newf("interval value %v too large, AS OF interval must be <= %v", d, -time.Nanosecond)

Done.


pkg/sql/sem/asof/as_of.go line 261 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

i think this should just be:

convErr = errors.Newf("interval value %v too small, SPLIT AT interval must be >= %v", d, time.Nanosecond)

Done.


pkg/sql/sem/asof/as_of.go line 290 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

same comment on error messages (and using <= 0 and >= 0 in the ifs

Done.

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fix!

Reviewed 1 of 4 files at r1, 2 of 3 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ZhouXing19)


-- commits line 11 at r4:
nit: commit message still uses nanosecond instead of microsecond


pkg/sql/sem/asof/as_of.go line 256 at r4 (raw file):

			} else if (usage == AsOf && iv.Duration.Compare(duration.Duration{}) > 0 && !syn) {
				convErr = errors.Errorf("interval value %v too large, AS OF interval must be <= -%v", d, time.Microsecond)
			} else if (usage == Split && iv.Duration.Compare(duration.Duration{}) < 0 && !syn) {

are you sure the syn check is needed on this one? (i'm not too familiar with it)

  fixes cockroachdb#91021

  Currently, if the given interval for `AS OF SYSTEM TIME interval` is a small
  postive duration, the query will incorrectly pass. It's because when we call
  `clock.Now()`, it has passed the threashold ('statement timestamp + duration').

  Now we add a check for the duration value. It has to be negative, with the
  absolute value >= a microsecond.

  Similarly, when the logic is used under the SPLIT stmt, the interval's value has
  to be positive (for the experation duration), with an absolute value >= a
  microsecond.

  link epic CRDB-17785

  Release note (bug fix): add restriction for duration value for AS OF SYSTEM TIME
  and SPLIT statement.
Copy link
Collaborator Author

@ZhouXing19 ZhouXing19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss)


-- commits line 11 at r4:

Previously, rafiss (Rafi Shamim) wrote…

nit: commit message still uses nanosecond instead of microsecond

Done.


pkg/sql/sem/asof/as_of.go line 256 at r4 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

are you sure the syn check is needed on this one? (i'm not too familiar with it)

Good point, I'm not familiar either, and there wasn't check / test for synthetic timestamp previously for SPLIT AT. I removed it for now and added a comment.

@ZhouXing19
Copy link
Collaborator Author

TFTR!
bors r=rafiss

@ZhouXing19 ZhouXing19 changed the title sql/sem: error when ASOT with a positive interval sql/sem: add check for interval for asof.DatumToHLC() Dec 9, 2022
@craig
Copy link
Contributor

craig bot commented Dec 9, 2022

This PR was included in a batch that was canceled, it will be automatically retried

@craig
Copy link
Contributor

craig bot commented Dec 9, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Dec 10, 2022

Build succeeded:

@craig craig bot merged commit 072ec8d into cockroachdb:master Dec 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent AS OF SYSTEM TIME future value behavior
3 participants