Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ccl/multiregionccl: TestMrSystemDatabase failed #98039

Closed
cockroach-teamcity opened this issue Mar 5, 2023 · 6 comments · Fixed by #100516
Closed

ccl/multiregionccl: TestMrSystemDatabase failed #98039

cockroach-teamcity opened this issue Mar 5, 2023 · 6 comments · Fixed by #100516
Assignees
@ajwerner
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 5, 2023 •
edited by cockroach-jira-scripts
Loading

ccl/multiregionccl.TestMrSystemDatabase failed with artifacts on master @ cf14ad694ee562676f53e36fa8495206c3aed61f:

=== RUN   TestMrSystemDatabase
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/e558fc8050776f4c54ea39ba371b49da/logTestMrSystemDatabase3500678595
    test_log_scope.go:79: use -show-logs to present logs inline
=== CONT  TestMrSystemDatabase
    multiregion_system_table_test.go:252: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/e558fc8050776f4c54ea39ba371b49da/logTestMrSystemDatabase3500678595
--- FAIL: TestMrSystemDatabase (5.82s)
=== RUN   TestMrSystemDatabase/QueryByEnum
    multiregion_system_table_test.go:249: error scanning '&{0xc001d058c0 <nil>}': pq: unsupported comparison: bytes to crdb_internal_region
    --- FAIL: TestMrSystemDatabase/QueryByEnum (0.00s)

Parameters: TAGS=bazel,gss

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

Jira issue: CRDB-25037
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 5, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Mar 5, 2023
@blathers-crl blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Mar 5, 2023
@ajwerner
Copy link
Contributor

ajwerner commented Mar 6, 2023

@jeffswenson can you put this on your radar?

@mgartner
Copy link
Collaborator

I saw this flake here: https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_BazelEssentialCi/9008712?showRootCauses=true&expandBuildChangesSection=true&expandBuildProblemsSection=true&expandBuildTestsSection=true

mgartner added a commit to mgartner/cockroach that referenced this issue Mar 13, 2023
@mgartner
ccl/multiregion: skip TestMrSystemDatabase
d17c9b9 
This test is flaky. See cockroachdb#98039.

Epic: None

Release note: None
@mgartner mgartner mentioned this issue Mar 13, 2023
Merged
@mgartner
Copy link
Collaborator

Adding the GA-blocker label now that the test will be skipped once #98533 merges.

craig bot pushed a commit that referenced this issue Mar 13, 2023
@rafiss @mgartner
Merge #98528 #98533
2ac6e51 
98528: logictest: use a retry loop when starting testserver r=rafiss a=rafiss

A similar change was already made for upgrading the node.

informs #94956

Release note: None

98533: ccl/multiregion: skip TestMrSystemDatabase r=mgartner a=mgartner

This test is flaky. See #98039.

Epic: None

Release note: None

Co-authored-by: Rafi Shamim <[email protected]>
Co-authored-by: Marcus Gartner <[email protected]>
@jeffswenson
Copy link
Collaborator

I'm removing the GA-blocker label from this issue. I think there is a bug here that needs to be fixed, but it only impacts multi-region Serverless.

@ajwerner
Copy link
Contributor

ajwerner commented Apr 3, 2023

I don't know that there's a bug here anymore. I've stressed the crap out of this with roachprod stress on 160 cores and can't get it to fail. I'll re-enable it.

@ajwerner ajwerner mentioned this issue Apr 3, 2023
Merged
craig bot pushed a commit that referenced this issue Apr 3, 2023
@Xiang-Gu @rytaft
@jbowens @ajwerner @yuzefovich
Merge #99858 #100357 #100489 #100516 #100527
cc9e0c6 
99858: screl: Add IndexID as a attr of UniqueWithoutIndex element r=Xiang-Gu a=Xiang-Gu

Previously, ALTER TABLE stmt where we add column/drop column/alter PK and adding a unique without index is problematic in that the it can succeed even when there are duplicate values. We already had a dep rule that enforces the new primary index to be backfilled before we validate the constraint against it. Unfortunately, this rule is not enforced on unique without index constraint because IndexID was not a attr of it. This commit fixes this.

Fixes #99281
Epic: None
Release note (bug fix): Fixed a bug in v23.1 in the declarative schema changer where unique without index can be incorrectly added in tables with duplicate values if it's with a ADD/DROP COLUMN in one ALTER TABLE statement.


100357: sql: allow changing the number of histogram samples and buckets r=rytaft a=rytaft

Informs #72418
Informs #97701

Release note (sql change): Added two new cluster settings that enable users to change the number of histogram samples and buckets collected when building histograms as part of table statistics collection: `sql.stats.histogram_samples.count` and `sql.stats.histogram_buckets.count`. While the default values should work for most cases, it may be beneficial to increase the number of samples and buckets for very large tables to avoid creating a histogram that misses important values.

100489: go.mod: bump Pebble to b84a7ec7d8dc r=RaduBerinde a=jbowens

```
b84a7ec7 db: populate return statistics for flushable ingests
5fd58365 objstorage: implement tracing
7f7451f2 db,record: add BatchCommitStats to measure total and component durations for commit
295aaab0 objstorage: implement basic refcounting
```

Epic: None
Release note: None

100516: multiregionccl: reenable TestMrSystemDatabase r=ajwerner a=ajwerner

I stressed this for a long time on many cores and it did not fail.

Epic: none

Fixes: #98039

Release note: None

100527: roachtest: skip multitenant/distsql for now r=yuzefovich a=yuzefovich

Informs: #100260.

Epic: None

Release note: None

Co-authored-by: Xiang Gu <[email protected]>
Co-authored-by: Rebecca Taft <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
Co-authored-by: Andrew Werner <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
@craig craig bot closed this as completed in 1740018 Apr 3, 2023
@jeffswenson
Copy link
Collaborator

I also ran this heavily under stress on the same commit from the original flake. I was unable to reproduce it. I'll keep an eye out for reflakes. Maybe there is some random test parameter that was required to trigger the flake.

@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

multiregionccl: reenable TestMrSystemDatabase ajwerner/cockroach
4 participants
@mgartner @ajwerner @jeffswenson @cockroach-teamcity