Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

execbuilder: re-enable skipped tests #75972

Merged
merged 1 commit into from
Feb 4, 2022

Conversation

mgartner
Copy link
Collaborator

@mgartner mgartner commented Feb 3, 2022

This commit de-flakes tests that were previously disabled in #75962 by
setting column families explicitly. I originally thought these
execbuilder tests were flaky because there were two query plans with the
same cost, but the flakiness was actually the result of randomized
families in logic tests.

Fixes #75960

Release note: None

This commit de-flakes tests that were previously disabled in cockroachdb#75962 by
setting column families explicitly. I originally thought these
execbuilder tests were flaky because there were two query plans with the
same cost, but the flakiness was actually the result of randomized
families in logic tests.

Fixes cockroachdb#75960

Release note: None
@mgartner mgartner requested review from RaduBerinde and a team February 3, 2022 19:27
@mgartner mgartner requested a review from a team as a code owner February 3, 2022 19:27
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Contributor

@cucaroach cucaroach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r1, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @RaduBerinde)

Copy link
Contributor

@cucaroach cucaroach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:LGTM:

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @RaduBerinde)

Copy link
Collaborator

@rharding6373 rharding6373 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @mgartner and @RaduBerinde)


pkg/sql/opt/exec/execbuilder/testdata/hash_sharded_index, line 203 at r1 (raw file):

  a INT8 PRIMARY KEY USING HASH WITH BUCKET_COUNT = 8,
  b INT8 NOT NULL,
  FAMILY (a, b)

Drive-by question: So without specifying the column family, we will store all columns as a single family but randomly decide whether to store them in (a,b) order or (b,a) order?

@mgartner
Copy link
Collaborator Author

mgartner commented Feb 3, 2022


pkg/sql/opt/exec/execbuilder/testdata/hash_sharded_index, line 203 at r1 (raw file):

Previously, rharding6373 (Rachael Harding) wrote…

Drive-by question: So without specifying the column family, we will store all columns as a single family but randomly decide whether to store them in (a,b) order or (b,a) order?

The logic tests use a column family mutator here:

execSQL, changed := randgen.ApplyString(t.rng, stmt.sql, randgen.ColumnFamilyMutator)

This mutates CREATE TABLE statements by randomly splitting up columns into a random number of families if there are no families explicitly specified in the CREATE TABLE statement:

func columnFamilyMutator(rng *rand.Rand, stmt tree.Statement) (changed bool) {
ast, ok := stmt.(*tree.CreateTable)
if !ok {
return false
}
var columns []tree.Name
for _, def := range ast.Defs {
switch def := def.(type) {
case *tree.FamilyTableDef:
return false
case *tree.ColumnTableDef:
if def.HasColumnFamily() {
return false
}
if !def.Computed.Virtual {
columns = append(columns, def.Name)
}
}
}
if len(columns) <= 1 {
return false
}
// Any columns not specified in column families
// are auto assigned to the first family, so
// there's no requirement to exhaust columns here.
rng.Shuffle(len(columns), func(i, j int) {
columns[i], columns[j] = columns[j], columns[i]
})
fd := &tree.FamilyTableDef{}
for {
if len(columns) == 0 {
if len(fd.Columns) > 0 {
ast.Defs = append(ast.Defs, fd)
}
break
}
fd.Columns = append(fd.Columns, columns[0])
columns = columns[1:]
// 50% chance to make a new column family.
if rng.Intn(2) != 0 {
ast.Defs = append(ast.Defs, fd)
fd = &tree.FamilyTableDef{}
}
}
return true
}

This is only happens for logic tests. In production, column families are always deterministic. I believe the default is to include all columns in a single family.

@mgartner
Copy link
Collaborator Author

mgartner commented Feb 3, 2022

bors r+

@craig
Copy link
Contributor

craig bot commented Feb 4, 2022

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Feb 4, 2022

Build failed:

@mgartner
Copy link
Collaborator Author

mgartner commented Feb 4, 2022

Looks like a flake.

bors r+

@craig
Copy link
Contributor

craig bot commented Feb 4, 2022

Build succeeded:

@craig craig bot merged commit b51c9b4 into cockroachdb:master Feb 4, 2022
@mgartner mgartner deleted the 75960-re-enable-flakes branch February 4, 2022 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql: TestExecBuild/local/hash_sharded_index/test_hash_index_unique_constraint_sec_key fails under stress
4 participants