Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coltypes: don't shallow copy decimals #40335

Merged
merged 2 commits into from
Sep 5, 2019
Merged

Conversation

jordanlewis
Copy link
Member

@jordanlewis jordanlewis commented Aug 29, 2019

This commit fixes a correctness bug caused by illegal shallow copy of
decimals. To copy a decimal, you must use the .Set method on it, and its
memory must already be distinct from the source decimal.

To facilitate this and generally simplify things, the "safe" GET method
is removed and replaced by COPYVAL, which copies a scalar value to
another scalar value safely. There were fewer than 5 users of GET, so
this wasn't particularly disruptive.

Closes #39777.
Closes #39540.

Release note: None

@jordanlewis jordanlewis requested a review from a team August 29, 2019 15:57
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@jordanlewis jordanlewis changed the title wip: coltypes: don't shallow copy decimals coltypes: don't shallow copy decimals Aug 29, 2019
Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @yuzefovich)


pkg/col/coltypes/types.go, line 201 at r1 (raw file):

tmp.Set(&%[1]s[%[2]s])
%[1]s[%[2]s].Set(&%[1]s[%[3]s])
%[1]s[%[3]s].Set(&%[1]s[%[2]s])`, target, i, j)

not sure if this is terribly readable -- you could consider using the inline templating idea that you have me for overloads.go

also, i think for the multiline ones it would be nicer to have some consistent indentation -- like:

return fmt.Sprintf(`
  var tmp apd.Decimal
  tmp.Set(&%[1]s[%[2]s])
  %[1]s[%[2]s].Set(&%[1]s[%[3]s])
`

pkg/col/coltypes/types.go, line 221 at r1 (raw file):

	case Decimal:
		return fmt.Sprintf(`
__tgt_slice := %[1]s[%[2]s:]

i guess it's fine that these variables share the outer scope? in overloads.go i had been introducing a new scope for all the cases where i defined a new variable, just to be safe, but maybe that's overkill?

@jordanlewis jordanlewis force-pushed the set-dec branch 4 times, most recently from c8de1a2 to 8fde2d8 Compare August 30, 2019 00:12
Copy link
Member Author

@jordanlewis jordanlewis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I redid this to be better with the templates.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @rafiss and @yuzefovich)


pkg/col/coltypes/types.go, line 201 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

not sure if this is terribly readable -- you could consider using the inline templating idea that you have me for overloads.go

also, i think for the multiline ones it would be nicer to have some consistent indentation -- like:

return fmt.Sprintf(`
  var tmp apd.Decimal
  tmp.Set(&%[1]s[%[2]s])
  %[1]s[%[2]s].Set(&%[1]s[%[3]s])
`

Done.


pkg/col/coltypes/types.go, line 221 at r1 (raw file):

Previously, rafiss (Rafi Shamim) wrote…

i guess it's fine that these variables share the outer scope? in overloads.go i had been introducing a new scope for all the cases where i defined a new variable, just to be safe, but maybe that's overkill?

Done.

Copy link
Member

@yuzefovich yuzefovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 8 of 9 files at r1, 2 of 2 files at r2.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)


pkg/col/coltypes/types.go, line 196 at r2 (raw file):

// Swap is a function that should only be used in templates.
func (t T) Swap(target, i, j string) string {

I'm proposing to remove this function in the cleanup-vec PR.


pkg/sql/logictest/testdata/logic_test/vectorize, line 705 at r2 (raw file):

	tab_426214._bool ASC
----
1296 values hashing to cad02075a867c3c0564bf80fe665eed6

Huh, what is this output?

@jordanlewis
Copy link
Member Author

Huh, what is this output?

You can have logic test compare the hash of the output to avoid having to write all of the rows, since in this case there are many.

rohany and others added 2 commits September 5, 2019 00:40
* Fixed a bug where the appropriate setters and getters were not being
generated for the casts.

* Fixed a bug where columns were not getting sliced, causing column
length mismatches.

Fixes cockroachdb#40461.

Release note: None
This commit fixes a correctness bug caused by illegal shallow copy of
decimals. To copy a decimal, you must use the .Set method on it, and its
memory must already be distinct from the source decimal.

To facilitate this and generally simplify things, the "safe" GET method
is removed and replaced by COPYVAL, which copies a scalar value to
another scalar value safely. There were fewer than 5 users of GET, so
this wasn't particularly disruptive.

Release note: None
@jordanlewis
Copy link
Member Author

PTAL - I think this is ready to go.

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 12 of 12 files at r4.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @jordanlewis and @rafiss)

@jordanlewis
Copy link
Member Author

Thanks for the reviews!

bors r+

craig bot pushed a commit that referenced this pull request Sep 5, 2019
40335: coltypes: don't shallow copy decimals r=jordanlewis a=jordanlewis

This commit fixes a correctness bug caused by illegal shallow copy of
decimals. To copy a decimal, you must use the .Set method on it, and its
memory must already be distinct from the source decimal.

To facilitate this and generally simplify things, the "safe" GET method
is removed and replaced by COPYVAL, which copies a scalar value to
another scalar value safely. There were fewer than 5 users of GET, so
this wasn't particularly disruptive.

Closes #39777.
Closes #39540.

Release note: None

40481: roachtest: update pgjdbc blacklist for 2.1 r=rafiss a=rafiss

The 2.1 blacklist was never filled in.

closes #39406

Release note: None

40492: Docs: Add SHOW JOBS diagram r=lhirata a=lhirata

Release note: none

Co-authored-by: Rohan Yadav <[email protected]>
Co-authored-by: Jordan Lewis <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
Co-authored-by: Lauren <[email protected]>
@craig
Copy link
Contributor

craig bot commented Sep 5, 2019

Build succeeded

@craig craig bot merged commit faa3109 into cockroachdb:master Sep 5, 2019
@jordanlewis jordanlewis deleted the set-dec branch September 5, 2019 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
6 participants