Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/sem/builtins: TestSerialNormalizationWithUniqueUnorderedID failed #78075

Closed
cockroach-teamcity opened this issue Mar 18, 2022 · 7 comments · Fixed by #83809
Closed

sql/sem/builtins: TestSerialNormalizationWithUniqueUnorderedID failed #78075

cockroach-teamcity opened this issue Mar 18, 2022 · 7 comments · Fixed by #83809
Assignees
@ajwerner
Labels
branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 18, 2022 •
edited by cockroach-jira-scripts
Loading

sql/sem/builtins.TestSerialNormalizationWithUniqueUnorderedID failed with artifacts on release-22.1 @ d7252106b166f23d791a4d677afdbc46b6b059ae:

=== RUN   TestSerialNormalizationWithUniqueUnorderedID
    builtins_test.go:130: Key counts in each split range
    builtins_test.go:132: range 0: 25
    builtins_test.go:132: range 1: 12
    builtins_test.go:132: range 2: 12
    builtins_test.go:132: range 3: 5
    builtins_test.go:132: range 4: 19
    builtins_test.go:132: range 5: 22
    builtins_test.go:132: range 6: 3
    assertion_compare.go:332: 
        	Error Trace:	builtins_test.go:146
        	Error:      	"30" is not less than "19.5114"
        	Test:       	TestSerialNormalizationWithUniqueUnorderedID
        	Messages:   	[chiSquared value of %f must be less than criticalVal %f to guarantee distribution is relatively uniform 30 19.5114]
--- FAIL: TestSerialNormalizationWithUniqueUnorderedID (1.92s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • TAGS=bazel,gss

/cc @cockroachdb/sql-experience

This test on roachdash | Improve this report!

Jira issue: CRDB-13953
@cockroach-teamcity cockroach-teamcity added branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 18, 2022
@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Mar 18, 2022
@rafiss
Copy link
Collaborator

rafiss commented Mar 18, 2022

cc @ajwerner do you know if anything changed with unique_unordered_id?

@ajwerner
Copy link
Contributor

No could just be a fluke. We should probably increase N.

@ajwerner
Copy link
Contributor

At least this flake will be rare

@cockroach-teamcity cockroach-teamcity mentioned this issue Mar 24, 2022
Closed
@cockroach-teamcity
Copy link
Member Author

sql/sem/builtins.TestSerialNormalizationWithUniqueUnorderedID failed with artifacts on release-22.1 @ 2d9c7c0869392b757778ac87acc09b7678e9a41d:

=== RUN   TestSerialNormalizationWithUniqueUnorderedID
    builtins_test.go:130: Key counts in each split range
    builtins_test.go:132: range 0: 8
    builtins_test.go:132: range 1: 3
    builtins_test.go:132: range 2: 21
    builtins_test.go:132: range 3: 9
    builtins_test.go:132: range 4: 18
    builtins_test.go:132: range 5: 2
    builtins_test.go:132: range 6: 22
    assertion_compare.go:332: 
        	Error Trace:	builtins_test.go:146
        	Error:      	"35.66265060240964" is not less than "19.5114"
        	Test:       	TestSerialNormalizationWithUniqueUnorderedID
        	Messages:   	[chiSquared value of %f must be less than criticalVal %f to guarantee distribution is relatively uniform 35.66265060240964 19.5114]
--- FAIL: TestSerialNormalizationWithUniqueUnorderedID (1.94s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • TAGS=bazel,gss

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/sem/builtins.TestSerialNormalizationWithUniqueUnorderedID failed with artifacts on release-22.1 @ 3181b7faa2f3b41d6a15ab4b74d2c60bcfe5132d:

=== RUN   TestSerialNormalizationWithUniqueUnorderedID
    builtins_test.go:130: Key counts in each split range
    builtins_test.go:132: range 0: 19
    builtins_test.go:132: range 1: 6
    builtins_test.go:132: range 2: 21
    builtins_test.go:132: range 3: 3
    builtins_test.go:132: range 4: 21
    builtins_test.go:132: range 5: 15
    builtins_test.go:132: range 6: 10
    assertion_compare.go:332: 
        	Error Trace:	builtins_test.go:146
        	Error:      	"23.852631578947367" is not less than "19.5114"
        	Test:       	TestSerialNormalizationWithUniqueUnorderedID
        	Messages:   	[chiSquared value of %f must be less than criticalVal %f to guarantee distribution is relatively uniform 23.852631578947367 19.5114]
--- FAIL: TestSerialNormalizationWithUniqueUnorderedID (2.00s)
Help

See also: How To Investigate a Go Test Failure (internal)
Parameters in this failure:

  • TAGS=bazel,gss

Same failure on other branches

This test on roachdash | Improve this report!

@otan
Copy link
Contributor

otan commented Jun 13, 2022

cc @cockroachdb/sql-schema who reviewed #70338

@blathers-crl blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Jun 13, 2022
@ajwerner
Copy link
Contributor

Yeah, I'll take it. If the statistics made sense, this wouldn't happen.

@ajwerner ajwerner self-assigned this Jun 13, 2022
@postamar postamar removed the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Jun 21, 2022
@ajwerner ajwerner mentioned this issue Jul 5, 2022
Merged
craig bot pushed a commit that referenced this issue Jul 5, 2022
@rickystewart @AlexTalks @ajwerner
Merge #83711 #83733 #83809
9c5472e 
83711: bazel: delete bespoke `string_test` target r=mari-crl a=rickystewart

This is an artifact of when `rules_go` had worse support for "external"
tests, but this works out-of-the-box now.

Release note: None

83733: roachtest: unskip large decommissionBench test r=AlexTalks a=AlexTalks

This extends the timeout of the large, 3000 warehouse decommission
benchmark roachtest to 3 hours, since it can take up to an hour for the
test to import data, achieve range count balance, and ramp up its
workload. This was skipped in #83445 due to frequent timeouts at the 1hr
mark.

It also adds a `--max-rate` parameter to the workload generator in order
to ensure the cluster avoids overload.

Closes #82870

Release note: None

83809: sql/sem/builtins: deflake TestSerialNormalizationWithUniqueUnorderedID r=ajwerner a=ajwerner

The test had two problems leading to flakes:
1) The statistic value was chosen with the wrong number of degrees of freedom.
   As such, it represented a 0.62% p value as opposed to a 0.001% p value.
2) The test is only meaningful when collected from a large N, which we did
   not use for the race test.

This should deflake the test.

Fixes #78075.

Release note: None

Co-authored-by: Ricky Stewart <[email protected]>
Co-authored-by: Alex Sarkesian <[email protected]>
Co-authored-by: Andrew Werner <[email protected]>
@craig craig bot closed this as completed in 24d5340 Jul 5, 2022
blathers-crl bot pushed a commit that referenced this issue Jul 6, 2022
@ajwerner
sql/sem/builtins: deflake TestSerialNormalizationWithUniqueUnorderedID
96834e8 
The test had two problems leading to flakes:
1) The statistic value was chosen with the wrong number of degrees of freedom.
   As such, it represented a 0.62% p value as opposed to a 0.001% p value.
2) The test is only meaningful when collected from a large N, which we did
   not use for the race test.

This should deflake the test.

Fixes #78075.

Release note: None
@blathers-crl blathers-crl bot mentioned this issue Jul 6, 2022
Merged
rafiss pushed a commit that referenced this issue Jul 6, 2022
@ajwerner @rafiss
sql/sem/builtins: deflake TestSerialNormalizationWithUniqueUnorderedID
92c547b 
The test had two problems leading to flakes:
1) The statistic value was chosen with the wrong number of degrees of freedom.
   As such, it represented a 0.62% p value as opposed to a 0.001% p value.
2) The test is only meaningful when collected from a large N, which we did
   not use for the race test.

This should deflake the test.

Fixes #78075.

Release note: None
@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
branch-release-22.1 Used to mark GA and release blockers, technical advisories, and bugs for 22.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql/sem/builtins: deflake TestSerialNormalizationWithUniqueUnorderedID ajwerner/cockroach
5 participants
@rafiss @ajwerner @postamar @otan @cockroach-teamcity