Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/tests: TestRandomSyntaxFunctions failed #99182

Closed
cockroach-teamcity opened this issue Mar 22, 2023 · 95 comments · Fixed by #107233
Closed

sql/tests: TestRandomSyntaxFunctions failed #99182

cockroach-teamcity opened this issue Mar 22, 2023 · 95 comments · Fixed by #107233
Assignees
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Milestone

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 22, 2023

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 80c4895c566a7eaa6f16c4098980509dd3795ad7:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14744262 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 184539 executions, 114327 successful
    rsg_test.go:828: 50s of 5m0s: 203901 executions, 126321 successful
    rsg_test.go:828: 55s of 5m0s: 224124 executions, 138962 successful
    rsg_test.go:828: 1m0s of 5m0s: 244942 executions, 151887 successful
    rsg_test.go:828: 1m5s of 5m0s: 264489 executions, 164027 successful
    rsg_test.go:828: 1m10s of 5m0s: 286231 executions, 177561 successful
    rsg_test.go:828: 1m15s of 5m0s: 306708 executions, 190169 successful
    rsg_test.go:828: 1m20s of 5m0s: 328477 executions, 203704 successful
    rsg_test.go:828: 1m25s of 5m0s: 350914 executions, 217589 successful
    rsg_test.go:828: 1m30s of 5m0s: 369611 executions, 229264 successful
    rsg_test.go:828: 1m35s of 5m0s: 391949 executions, 243109 successful
    rsg_test.go:828: 1m40s of 5m0s: 411239 executions, 255088 successful
    rsg_test.go:828: 1m45s of 5m0s: 431015 executions, 267258 successful
    rsg_test.go:828: 1m50s of 5m0s: 453401 executions, 281275 successful
    rsg_test.go:828: 1m55s of 5m0s: 470645 executions, 291943 successful
    rsg_test.go:828: 2m0s of 5m0s: 490494 executions, 304159 successful
    rsg_test.go:828: 2m5s of 5m0s: 510413 executions, 316515 successful
    rsg_test.go:828: 2m10s of 5m0s: 532347 executions, 329995 successful
    rsg_test.go:828: 2m15s of 5m0s: 551421 executions, 341800 successful
    rsg_test.go:828: 2m20s of 5m0s: 574984 executions, 356342 successful
    rsg_test.go:828: 2m25s of 5m0s: 596910 executions, 370015 successful
    rsg_test.go:828: 2m30s of 5m0s: 617486 executions, 382749 successful
    rsg_test.go:828: 2m35s of 5m0s: 640619 executions, 397074 successful
    rsg_test.go:828: 2m40s of 5m0s: 664202 executions, 411572 successful
    rsg_test.go:828: 2m45s of 5m0s: 682912 executions, 423226 successful
    rsg_test.go:828: 2m50s of 5m0s: 704889 executions, 436710 successful
    rsg_test.go:828: 2m55s of 5m0s: 726256 executions, 449956 successful
    rsg_test.go:828: 3m0s of 5m0s: 745851 executions, 462135 successful
    rsg_test.go:828: 3m5s of 5m0s: 765690 executions, 474382 successful
    rsg_test.go:828: 3m10s of 5m0s: 788136 executions, 488352 successful
    rsg_test.go:828: 3m15s of 5m0s: 811683 executions, 502956 successful
    rsg_test.go:828: 3m20s of 5m0s: 833731 executions, 516672 successful
    rsg_test.go:828: 3m25s of 5m0s: 854843 executions, 529714 successful
    rsg_test.go:828: 3m30s of 5m0s: 875625 executions, 542602 successful
    rsg_test.go:828: 3m35s of 5m0s: 896712 executions, 555671 successful
    rsg_test.go:828: 3m40s of 5m0s: 920582 executions, 570273 successful
    rsg_test.go:828: 3m45s of 5m0s: 944048 executions, 584776 successful
    rsg_test.go:828: 3m50s of 5m0s: 964197 executions, 597197 successful
    rsg_test.go:828: 3m55s of 5m0s: 988363 executions, 612375 successful
    rsg_test.go:828: 4m0s of 5m0s: 1008301 executions, 624692 successful
    rsg_test.go:828: 4m5s of 5m0s: 1028220 executions, 637050 successful
    rsg_test.go:828: 4m10s of 5m0s: 1049754 executions, 650421 successful
    rsg_test.go:828: 4m15s of 5m0s: 1072363 executions, 664443 successful
    rsg_test.go:828: 4m20s of 5m0s: 1094347 executions, 678049 successful
    rsg_test.go:828: 4m25s of 5m0s: 1116985 executions, 692049 successful
    rsg_test.go:828: 4m30s of 5m0s: 1138611 executions, 705447 successful
    rsg_test.go:828: 4m35s of 5m0s: 1162669 executions, 720398 successful
    rsg_test.go:828: 4m40s of 5m0s: 1183353 executions, 733226 successful
    rsg_test.go:828: 4m45s of 5m0s: 1205810 executions, 747037 successful
    rsg_test.go:828: 4m50s of 5m0s: 1227922 executions, 760707 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

/cc @cockroachdb/sql-sessions

This test on roachdash | Improve this report!

Jira issue: CRDB-25738

@cockroach-teamcity cockroach-teamcity added branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 22, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Mar 22, 2023
@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Mar 22, 2023
@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 52e55d2ef172b7cfec14e8a0a954f8864b2be779:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 12922748 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 4m45s of 5m0s: 1101969 executions, 683528 successful
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'\x14l%*|':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'W$$\n':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery('"':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'\x1e\x01jz':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery('^':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'e\x02W\'':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'*\f$|\x1c':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'cWT|\x06\x12#':::STRING) ;
        
        Stack trace:
    rsg_test.go:828: 4m50s of 5m0s: 1121294 executions, 695496 successful
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'\x0e0wk\x03\x1d/{':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery('F:H-a':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e',\x17[N\x0e|Vcc':::STRING) ;
        
        Stack trace:
    rsg_test.go:855: Crash detected: server panic: pq: internal error: interface conversion: tree.FnOverload is func(*eval.Context, tree.Datums) (tree.Datum, error), not func(context.Context, *eval.Context, tree.Datums) (tree.Datum, error)
        SELECT phraseto_tsquery(e'|b\tg':::STRING) ;
        
        Stack trace:

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ f351747ed97862fc037717cadec23f18073fb6be:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 13101667 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 154890 executions, 96052 successful
    rsg_test.go:828: 50s of 5m0s: 173367 executions, 107542 successful
    rsg_test.go:828: 55s of 5m0s: 190273 executions, 118007 successful
    rsg_test.go:828: 1m0s of 5m0s: 210249 executions, 130255 successful
    rsg_test.go:828: 1m5s of 5m0s: 228356 executions, 141490 successful
    rsg_test.go:828: 1m10s of 5m0s: 246297 executions, 152674 successful
    rsg_test.go:828: 1m15s of 5m0s: 264454 executions, 163960 successful
    rsg_test.go:828: 1m20s of 5m0s: 283593 executions, 175849 successful
    rsg_test.go:828: 1m25s of 5m0s: 298900 executions, 185404 successful
    rsg_test.go:828: 1m30s of 5m0s: 318340 executions, 197445 successful
    rsg_test.go:828: 1m35s of 5m0s: 335492 executions, 208107 successful
    rsg_test.go:828: 1m40s of 5m0s: 356934 executions, 221372 successful
    rsg_test.go:828: 1m45s of 5m0s: 374720 executions, 232350 successful
    rsg_test.go:828: 1m50s of 5m0s: 392635 executions, 243399 successful
    rsg_test.go:828: 1m55s of 5m0s: 410133 executions, 254361 successful
    rsg_test.go:828: 2m0s of 5m0s: 427800 executions, 265293 successful
    rsg_test.go:828: 2m5s of 5m0s: 445940 executions, 276596 successful
    rsg_test.go:828: 2m10s of 5m0s: 463288 executions, 287390 successful
    rsg_test.go:828: 2m15s of 5m0s: 479347 executions, 297338 successful
    rsg_test.go:828: 2m20s of 5m0s: 498508 executions, 309274 successful
    rsg_test.go:828: 2m25s of 5m0s: 517864 executions, 321211 successful
    rsg_test.go:828: 2m30s of 5m0s: 536530 executions, 332724 successful
    rsg_test.go:828: 2m35s of 5m0s: 557223 executions, 345625 successful
    rsg_test.go:828: 2m40s of 5m0s: 574076 executions, 356079 successful
    rsg_test.go:828: 2m45s of 5m0s: 590786 executions, 366399 successful
    rsg_test.go:828: 2m50s of 5m0s: 603754 executions, 374395 successful
    rsg_test.go:828: 2m55s of 5m0s: 620571 executions, 384875 successful
    rsg_test.go:828: 3m0s of 5m0s: 637741 executions, 395580 successful
    rsg_test.go:828: 3m5s of 5m0s: 653953 executions, 405616 successful
    rsg_test.go:828: 3m10s of 5m0s: 672287 executions, 417078 successful
    rsg_test.go:828: 3m15s of 5m0s: 691962 executions, 429242 successful
    rsg_test.go:828: 3m20s of 5m0s: 708757 executions, 439764 successful
    rsg_test.go:828: 3m25s of 5m0s: 727153 executions, 451112 successful
    rsg_test.go:828: 3m30s of 5m0s: 746010 executions, 462834 successful
    rsg_test.go:828: 3m35s of 5m0s: 765124 executions, 474661 successful
    rsg_test.go:828: 3m40s of 5m0s: 786301 executions, 487669 successful
    rsg_test.go:828: 3m45s of 5m0s: 802706 executions, 497863 successful
    rsg_test.go:828: 3m50s of 5m0s: 819416 executions, 508168 successful
    rsg_test.go:828: 3m55s of 5m0s: 836625 executions, 518774 successful
    rsg_test.go:828: 4m0s of 5m0s: 853157 executions, 529002 successful
    rsg_test.go:828: 4m5s of 5m0s: 868794 executions, 538762 successful
    rsg_test.go:828: 4m10s of 5m0s: 884533 executions, 548529 successful
    rsg_test.go:828: 4m15s of 5m0s: 900960 executions, 558687 successful
    rsg_test.go:828: 4m20s of 5m0s: 917558 executions, 569077 successful
    rsg_test.go:828: 4m25s of 5m0s: 935041 executions, 579957 successful
    rsg_test.go:828: 4m30s of 5m0s: 954285 executions, 591843 successful
    rsg_test.go:828: 4m35s of 5m0s: 972137 executions, 602799 successful
    rsg_test.go:828: 4m40s of 5m0s: 987727 executions, 612516 successful
    rsg_test.go:828: 4m45s of 5m0s: 1004839 executions, 623037 successful
    rsg_test.go:828: 4m50s of 5m0s: 1023483 executions, 634662 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 186218edb86b5974501b2976cfd0704d4d6d1beb:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 13746966 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 169998 executions, 105336 successful
    rsg_test.go:828: 50s of 5m0s: 189680 executions, 117640 successful
    rsg_test.go:828: 55s of 5m0s: 209489 executions, 130034 successful
    rsg_test.go:828: 1m0s of 5m0s: 230882 executions, 143272 successful
    rsg_test.go:828: 1m5s of 5m0s: 251020 executions, 155705 successful
    rsg_test.go:828: 1m10s of 5m0s: 272728 executions, 169121 successful
    rsg_test.go:828: 1m15s of 5m0s: 293370 executions, 181848 successful
    rsg_test.go:828: 1m20s of 5m0s: 314475 executions, 194926 successful
    rsg_test.go:828: 1m25s of 5m0s: 334045 executions, 207085 successful
    rsg_test.go:828: 1m30s of 5m0s: 353959 executions, 219381 successful
    rsg_test.go:828: 1m35s of 5m0s: 374526 executions, 232182 successful
    rsg_test.go:828: 1m40s of 5m0s: 390957 executions, 242277 successful
    rsg_test.go:828: 1m45s of 5m0s: 411332 executions, 254975 successful
    rsg_test.go:828: 1m50s of 5m0s: 431948 executions, 267764 successful
    rsg_test.go:828: 1m55s of 5m0s: 452863 executions, 280737 successful
    rsg_test.go:828: 2m0s of 5m0s: 473634 executions, 293670 successful
    rsg_test.go:828: 2m5s of 5m0s: 495485 executions, 307253 successful
    rsg_test.go:828: 2m10s of 5m0s: 513817 executions, 318581 successful
    rsg_test.go:828: 2m15s of 5m0s: 535127 executions, 331782 successful
    rsg_test.go:828: 2m20s of 5m0s: 556388 executions, 345008 successful
    rsg_test.go:828: 2m25s of 5m0s: 573051 executions, 355313 successful
    rsg_test.go:828: 2m30s of 5m0s: 593372 executions, 368041 successful
    rsg_test.go:828: 2m35s of 5m0s: 614333 executions, 381006 successful
    rsg_test.go:828: 2m40s of 5m0s: 634976 executions, 393766 successful
    rsg_test.go:828: 2m45s of 5m0s: 655466 executions, 406527 successful
    rsg_test.go:828: 2m50s of 5m0s: 673227 executions, 417574 successful
    rsg_test.go:828: 2m55s of 5m0s: 691068 executions, 428722 successful
    rsg_test.go:828: 3m0s of 5m0s: 710614 executions, 440838 successful
    rsg_test.go:828: 3m5s of 5m0s: 730885 executions, 453340 successful
    rsg_test.go:828: 3m10s of 5m0s: 751852 executions, 466430 successful
    rsg_test.go:828: 3m15s of 5m0s: 774414 executions, 480362 successful
    rsg_test.go:828: 3m20s of 5m0s: 793203 executions, 492055 successful
    rsg_test.go:828: 3m25s of 5m0s: 814142 executions, 505020 successful
    rsg_test.go:828: 3m30s of 5m0s: 836927 executions, 519126 successful
    rsg_test.go:828: 3m35s of 5m0s: 858558 executions, 532559 successful
    rsg_test.go:828: 3m40s of 5m0s: 881253 executions, 546664 successful
    rsg_test.go:828: 3m45s of 5m0s: 899677 executions, 558020 successful
    rsg_test.go:828: 3m50s of 5m0s: 921609 executions, 571600 successful
    rsg_test.go:828: 3m55s of 5m0s: 942734 executions, 584802 successful
    rsg_test.go:828: 4m0s of 5m0s: 964614 executions, 598264 successful
    rsg_test.go:828: 4m5s of 5m0s: 984449 executions, 610561 successful
    rsg_test.go:828: 4m10s of 5m0s: 1006333 executions, 624215 successful
    rsg_test.go:828: 4m15s of 5m0s: 1028514 executions, 638009 successful
    rsg_test.go:828: 4m20s of 5m0s: 1049599 executions, 651102 successful
    rsg_test.go:828: 4m25s of 5m0s: 1071912 executions, 664893 successful
    rsg_test.go:828: 4m30s of 5m0s: 1093575 executions, 678287 successful
    rsg_test.go:828: 4m35s of 5m0s: 1113100 executions, 690436 successful
    rsg_test.go:828: 4m40s of 5m0s: 1132901 executions, 702730 successful
    rsg_test.go:828: 4m45s of 5m0s: 1153128 executions, 715163 successful
    rsg_test.go:828: 4m50s of 5m0s: 1171045 executions, 726311 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 3551dc089ac3efcb3772df2109164dcb01e79d8f:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14224171 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

=== RUN   TestRandomSyntaxFunctions
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/d437d2c847dfedbc4972f231c3331c8e/logTestRandomSyntaxFunctions778170626
    test_log_scope.go:79: use -show-logs to present logs inline
    rsg_test.go:828: 5s of 5m0s: 20537 executions, 12696 successful
    rsg_test.go:828: 10s of 5m0s: 38321 executions, 23685 successful
    rsg_test.go:828: 15s of 5m0s: 60313 executions, 37344 successful
    rsg_test.go:828: 20s of 5m0s: 79296 executions, 49137 successful
    rsg_test.go:828: 25s of 5m0s: 97434 executions, 60301 successful
    rsg_test.go:828: 30s of 5m0s: 118644 executions, 73443 successful
    rsg_test.go:828: 35s of 5m0s: 140019 executions, 86690 successful
    rsg_test.go:828: 40s of 5m0s: 159839 executions, 99056 successful
    rsg_test.go:828: 45s of 5m0s: 180117 executions, 111623 successful
    rsg_test.go:828: 50s of 5m0s: 200904 executions, 124559 successful
    rsg_test.go:828: 55s of 5m0s: 222889 executions, 138282 successful
    rsg_test.go:828: 1m0s of 5m0s: 243107 executions, 150836 successful
    rsg_test.go:828: 1m5s of 5m0s: 264578 executions, 164112 successful
    rsg_test.go:828: 1m10s of 5m0s: 282741 executions, 175406 successful
    rsg_test.go:828: 1m15s of 5m0s: 303107 executions, 187960 successful
    rsg_test.go:828: 1m20s of 5m0s: 323130 executions, 200353 successful
    rsg_test.go:828: 1m25s of 5m0s: 346186 executions, 214686 successful
    rsg_test.go:828: 1m30s of 5m0s: 366911 executions, 227596 successful
    rsg_test.go:828: 1m35s of 5m0s: 384288 executions, 238372 successful
    rsg_test.go:828: 1m40s of 5m0s: 407918 executions, 252847 successful
    rsg_test.go:828: 1m45s of 5m0s: 431975 executions, 267810 successful
    rsg_test.go:828: 1m50s of 5m0s: 454381 executions, 281721 successful
    rsg_test.go:828: 1m55s of 5m0s: 475234 executions, 294612 successful
    rsg_test.go:828: 2m0s of 5m0s: 494221 executions, 306336 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ aec78f33d45a8376a0ecec885688bae60dbfb85c:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14633906 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 169800 executions, 105473 successful
    rsg_test.go:828: 50s of 5m0s: 190513 executions, 118278 successful
    rsg_test.go:828: 55s of 5m0s: 210486 executions, 130548 successful
    rsg_test.go:828: 1m0s of 5m0s: 233381 executions, 144769 successful
    rsg_test.go:828: 1m5s of 5m0s: 251962 executions, 156400 successful
    rsg_test.go:828: 1m10s of 5m0s: 271153 executions, 168288 successful
    rsg_test.go:828: 1m15s of 5m0s: 289851 executions, 179836 successful
    rsg_test.go:828: 1m20s of 5m0s: 307780 executions, 191084 successful
    rsg_test.go:828: 1m25s of 5m0s: 324932 executions, 201706 successful
    rsg_test.go:828: 1m30s of 5m0s: 342258 executions, 212442 successful
    rsg_test.go:828: 1m35s of 5m0s: 364317 executions, 226004 successful
    rsg_test.go:828: 1m40s of 5m0s: 382800 executions, 237528 successful
    rsg_test.go:828: 1m45s of 5m0s: 401920 executions, 249283 successful
    rsg_test.go:828: 1m50s of 5m0s: 422870 executions, 262345 successful
    rsg_test.go:828: 1m55s of 5m0s: 440942 executions, 273461 successful
    rsg_test.go:828: 2m0s of 5m0s: 460187 executions, 285401 successful
    rsg_test.go:828: 2m5s of 5m0s: 481068 executions, 298399 successful
    rsg_test.go:828: 2m10s of 5m0s: 499976 executions, 310204 successful
    rsg_test.go:828: 2m15s of 5m0s: 519712 executions, 322381 successful
    rsg_test.go:828: 2m20s of 5m0s: 537844 executions, 333713 successful
    rsg_test.go:828: 2m25s of 5m0s: 557301 executions, 345779 successful
    rsg_test.go:828: 2m30s of 5m0s: 577820 executions, 358402 successful
    rsg_test.go:828: 2m35s of 5m0s: 597694 executions, 370706 successful
    rsg_test.go:828: 2m40s of 5m0s: 616949 executions, 382652 successful
    rsg_test.go:828: 2m45s of 5m0s: 636863 executions, 395004 successful
    rsg_test.go:828: 2m50s of 5m0s: 657782 executions, 407880 successful
    rsg_test.go:828: 2m55s of 5m0s: 676670 executions, 419618 successful
    rsg_test.go:828: 3m0s of 5m0s: 697764 executions, 432723 successful
    rsg_test.go:828: 3m5s of 5m0s: 716282 executions, 444219 successful
    rsg_test.go:828: 3m10s of 5m0s: 735601 executions, 456217 successful
    rsg_test.go:828: 3m15s of 5m0s: 754460 executions, 467870 successful
    rsg_test.go:828: 3m20s of 5m0s: 773521 executions, 479722 successful
    rsg_test.go:828: 3m25s of 5m0s: 790837 executions, 490466 successful
    rsg_test.go:828: 3m30s of 5m0s: 811917 executions, 503465 successful
    rsg_test.go:828: 3m35s of 5m0s: 827984 executions, 513484 successful
    rsg_test.go:828: 3m40s of 5m0s: 847508 executions, 525575 successful
    rsg_test.go:828: 3m45s of 5m0s: 868651 executions, 538649 successful
    rsg_test.go:828: 3m50s of 5m0s: 887456 executions, 550369 successful
    rsg_test.go:828: 3m55s of 5m0s: 907781 executions, 562825 successful
    rsg_test.go:828: 4m0s of 5m0s: 927900 executions, 575328 successful
    rsg_test.go:828: 4m5s of 5m0s: 947164 executions, 587258 successful
    rsg_test.go:828: 4m10s of 5m0s: 968253 executions, 600324 successful
    rsg_test.go:828: 4m15s of 5m0s: 985977 executions, 611237 successful
    rsg_test.go:828: 4m20s of 5m0s: 1008614 executions, 625374 successful
    rsg_test.go:828: 4m25s of 5m0s: 1027903 executions, 637324 successful
    rsg_test.go:828: 4m30s of 5m0s: 1047083 executions, 649125 successful
    rsg_test.go:828: 4m35s of 5m0s: 1067357 executions, 661578 successful
    rsg_test.go:828: 4m40s of 5m0s: 1086179 executions, 673270 successful
    rsg_test.go:828: 4m45s of 5m0s: 1102513 executions, 683397 successful
    rsg_test.go:828: 4m50s of 5m0s: 1118504 executions, 693139 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 7e72aae900c3ff4b44f1643c2d7ba55fbb2cbe23:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14067578 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 186650 executions, 115779 successful
    rsg_test.go:828: 50s of 5m0s: 207395 executions, 128530 successful
    rsg_test.go:828: 55s of 5m0s: 226905 executions, 140696 successful
    rsg_test.go:828: 1m0s of 5m0s: 249354 executions, 154514 successful
    rsg_test.go:828: 1m5s of 5m0s: 266671 executions, 165196 successful
    rsg_test.go:828: 1m10s of 5m0s: 285770 executions, 176990 successful
    rsg_test.go:828: 1m15s of 5m0s: 304412 executions, 188528 successful
    rsg_test.go:828: 1m20s of 5m0s: 323010 executions, 200213 successful
    rsg_test.go:828: 1m25s of 5m0s: 342381 executions, 212243 successful
    rsg_test.go:828: 1m30s of 5m0s: 361167 executions, 223850 successful
    rsg_test.go:828: 1m35s of 5m0s: 381326 executions, 236289 successful
    rsg_test.go:828: 1m40s of 5m0s: 399512 executions, 247616 successful
    rsg_test.go:828: 1m45s of 5m0s: 419678 executions, 260101 successful
    rsg_test.go:828: 1m50s of 5m0s: 438219 executions, 271722 successful
    rsg_test.go:828: 1m55s of 5m0s: 458228 executions, 283993 successful
    rsg_test.go:828: 2m0s of 5m0s: 472318 executions, 292730 successful
    rsg_test.go:828: 2m5s of 5m0s: 493348 executions, 305777 successful
    rsg_test.go:828: 2m10s of 5m0s: 513175 executions, 318170 successful
    rsg_test.go:828: 2m15s of 5m0s: 532668 executions, 330235 successful
    rsg_test.go:828: 2m20s of 5m0s: 554847 executions, 343993 successful
    rsg_test.go:828: 2m25s of 5m0s: 575040 executions, 356515 successful
    rsg_test.go:828: 2m30s of 5m0s: 596075 executions, 369622 successful
    rsg_test.go:828: 2m35s of 5m0s: 615413 executions, 381630 successful
    rsg_test.go:828: 2m40s of 5m0s: 636954 executions, 394954 successful
    rsg_test.go:828: 2m45s of 5m0s: 653483 executions, 405281 successful
    rsg_test.go:828: 2m50s of 5m0s: 672729 executions, 417178 successful
    rsg_test.go:828: 2m55s of 5m0s: 693537 executions, 430103 successful
    rsg_test.go:828: 3m0s of 5m0s: 715821 executions, 443827 successful
    rsg_test.go:828: 3m5s of 5m0s: 735503 executions, 456097 successful
    rsg_test.go:828: 3m10s of 5m0s: 756526 executions, 469116 successful
    rsg_test.go:828: 3m15s of 5m0s: 779090 executions, 483043 successful
    rsg_test.go:828: 3m20s of 5m0s: 797487 executions, 494453 successful
    rsg_test.go:828: 3m25s of 5m0s: 814376 executions, 504914 successful
    rsg_test.go:828: 3m30s of 5m0s: 833022 executions, 516393 successful
    rsg_test.go:828: 3m35s of 5m0s: 853665 executions, 529201 successful
    rsg_test.go:828: 3m40s of 5m0s: 872047 executions, 540651 successful
    rsg_test.go:828: 3m45s of 5m0s: 892198 executions, 553130 successful
    rsg_test.go:828: 3m50s of 5m0s: 910802 executions, 564758 successful
    rsg_test.go:828: 3m55s of 5m0s: 930731 executions, 577129 successful
    rsg_test.go:828: 4m0s of 5m0s: 951245 executions, 589951 successful
    rsg_test.go:828: 4m5s of 5m0s: 969821 executions, 601438 successful
    rsg_test.go:828: 4m10s of 5m0s: 990461 executions, 614175 successful
    rsg_test.go:828: 4m15s of 5m0s: 1011658 executions, 627292 successful
    rsg_test.go:828: 4m20s of 5m0s: 1030658 executions, 639048 successful
    rsg_test.go:828: 4m25s of 5m0s: 1049762 executions, 650875 successful
    rsg_test.go:828: 4m30s of 5m0s: 1067814 executions, 662137 successful
    rsg_test.go:828: 4m35s of 5m0s: 1088339 executions, 674803 successful
    rsg_test.go:828: 4m40s of 5m0s: 1109215 executions, 687798 successful
    rsg_test.go:828: 4m45s of 5m0s: 1132592 executions, 702271 successful
    rsg_test.go:828: 4m50s of 5m0s: 1151209 executions, 713801 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 5d5abe704b0551556c35d51aa673d3162f493aa8:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 10387569 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 145262 executions, 90073 successful
    rsg_test.go:828: 50s of 5m0s: 163139 executions, 101149 successful
    rsg_test.go:828: 55s of 5m0s: 178131 executions, 110437 successful
    rsg_test.go:828: 1m0s of 5m0s: 192021 executions, 119055 successful
    rsg_test.go:828: 1m5s of 5m0s: 205020 executions, 127165 successful
    rsg_test.go:828: 1m10s of 5m0s: 219830 executions, 136234 successful
    rsg_test.go:828: 1m15s of 5m0s: 234548 executions, 145430 successful
    rsg_test.go:828: 1m20s of 5m0s: 251959 executions, 156209 successful
    rsg_test.go:828: 1m25s of 5m0s: 268782 executions, 166500 successful
    rsg_test.go:828: 1m30s of 5m0s: 286005 executions, 177124 successful
    rsg_test.go:828: 1m35s of 5m0s: 299248 executions, 185360 successful
    rsg_test.go:828: 1m40s of 5m0s: 313633 executions, 194412 successful
    rsg_test.go:828: 1m45s of 5m0s: 327986 executions, 203254 successful
    rsg_test.go:828: 1m50s of 5m0s: 341746 executions, 211782 successful
    rsg_test.go:828: 1m55s of 5m0s: 357022 executions, 221219 successful
    rsg_test.go:828: 2m0s of 5m0s: 373148 executions, 231279 successful
    rsg_test.go:828: 2m5s of 5m0s: 386789 executions, 239715 successful
    rsg_test.go:828: 2m10s of 5m0s: 403764 executions, 250213 successful
    rsg_test.go:828: 2m15s of 5m0s: 420410 executions, 260443 successful
    rsg_test.go:828: 2m20s of 5m0s: 437427 executions, 271015 successful
    rsg_test.go:828: 2m25s of 5m0s: 452852 executions, 280516 successful
    rsg_test.go:828: 2m30s of 5m0s: 468030 executions, 289958 successful
    rsg_test.go:828: 2m35s of 5m0s: 484149 executions, 299979 successful
    rsg_test.go:828: 2m40s of 5m0s: 498132 executions, 308621 successful
    rsg_test.go:828: 2m45s of 5m0s: 513060 executions, 317942 successful
    rsg_test.go:828: 2m50s of 5m0s: 527686 executions, 326867 successful
    rsg_test.go:828: 2m55s of 5m0s: 541799 executions, 335696 successful
    rsg_test.go:828: 3m0s of 5m0s: 558675 executions, 346058 successful
    rsg_test.go:828: 3m5s of 5m0s: 574671 executions, 356001 successful
    rsg_test.go:828: 3m10s of 5m0s: 590811 executions, 365956 successful
    rsg_test.go:828: 3m15s of 5m0s: 608258 executions, 376748 successful
    rsg_test.go:828: 3m20s of 5m0s: 622217 executions, 385353 successful
    rsg_test.go:828: 3m25s of 5m0s: 637688 executions, 394975 successful
    rsg_test.go:828: 3m30s of 5m0s: 652243 executions, 403976 successful
    rsg_test.go:828: 3m35s of 5m0s: 666282 executions, 412712 successful
    rsg_test.go:828: 3m40s of 5m0s: 683192 executions, 423223 successful
    rsg_test.go:828: 3m45s of 5m0s: 700429 executions, 433935 successful
    rsg_test.go:828: 3m50s of 5m0s: 716191 executions, 443708 successful
    rsg_test.go:828: 3m55s of 5m0s: 731184 executions, 453002 successful
    rsg_test.go:828: 4m0s of 5m0s: 745106 executions, 461715 successful
    rsg_test.go:828: 4m5s of 5m0s: 761031 executions, 471598 successful
    rsg_test.go:828: 4m10s of 5m0s: 776891 executions, 481301 successful
    rsg_test.go:828: 4m15s of 5m0s: 793005 executions, 491329 successful
    rsg_test.go:828: 4m20s of 5m0s: 810704 executions, 502329 successful
    rsg_test.go:828: 4m25s of 5m0s: 826238 executions, 512000 successful
    rsg_test.go:828: 4m30s of 5m0s: 840828 executions, 520976 successful
    rsg_test.go:828: 4m35s of 5m0s: 857643 executions, 531367 successful
    rsg_test.go:828: 4m40s of 5m0s: 874065 executions, 541578 successful
    rsg_test.go:828: 4m45s of 5m0s: 890278 executions, 551501 successful
    rsg_test.go:828: 4m50s of 5m0s: 907625 executions, 562273 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 63b683eab3e0ebfff812bb54fdac936c101d9383:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14634800 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 182470 executions, 113072 successful
    rsg_test.go:828: 50s of 5m0s: 203399 executions, 126062 successful
    rsg_test.go:828: 55s of 5m0s: 225971 executions, 140123 successful
    rsg_test.go:828: 1m0s of 5m0s: 245436 executions, 152191 successful
    rsg_test.go:828: 1m5s of 5m0s: 265704 executions, 164626 successful
    rsg_test.go:828: 1m10s of 5m0s: 286205 executions, 177333 successful
    rsg_test.go:828: 1m15s of 5m0s: 306708 executions, 189979 successful
    rsg_test.go:828: 1m20s of 5m0s: 328186 executions, 203240 successful
    rsg_test.go:828: 1m25s of 5m0s: 348485 executions, 215738 successful
    rsg_test.go:828: 1m30s of 5m0s: 369701 executions, 228868 successful
    rsg_test.go:828: 1m35s of 5m0s: 390961 executions, 242081 successful
    rsg_test.go:828: 1m40s of 5m0s: 413367 executions, 256045 successful
    rsg_test.go:828: 1m45s of 5m0s: 437547 executions, 270936 successful
    rsg_test.go:828: 1m50s of 5m0s: 458544 executions, 283885 successful
    rsg_test.go:828: 1m55s of 5m0s: 478311 executions, 296181 successful
    rsg_test.go:828: 2m0s of 5m0s: 499623 executions, 309425 successful
    rsg_test.go:828: 2m5s of 5m0s: 522431 executions, 323514 successful
    rsg_test.go:828: 2m10s of 5m0s: 538106 executions, 333240 successful
    rsg_test.go:828: 2m15s of 5m0s: 556774 executions, 344731 successful
    rsg_test.go:828: 2m20s of 5m0s: 575995 executions, 356654 successful
    rsg_test.go:828: 2m25s of 5m0s: 595077 executions, 368433 successful
    rsg_test.go:828: 2m30s of 5m0s: 615086 executions, 380929 successful
    rsg_test.go:828: 2m35s of 5m0s: 637079 executions, 394487 successful
    rsg_test.go:828: 2m40s of 5m0s: 655394 executions, 405861 successful
    rsg_test.go:828: 2m45s of 5m0s: 676309 executions, 418874 successful
    rsg_test.go:828: 2m50s of 5m0s: 699271 executions, 433194 successful
    rsg_test.go:828: 2m55s of 5m0s: 720155 executions, 446115 successful
    rsg_test.go:828: 3m0s of 5m0s: 742423 executions, 459958 successful
    rsg_test.go:828: 3m5s of 5m0s: 765604 executions, 474347 successful
    rsg_test.go:828: 3m10s of 5m0s: 788003 executions, 488306 successful
    rsg_test.go:828: 3m15s of 5m0s: 807126 executions, 500194 successful
    rsg_test.go:828: 3m20s of 5m0s: 829220 executions, 513893 successful
    rsg_test.go:828: 3m25s of 5m0s: 849100 executions, 526319 successful
    rsg_test.go:828: 3m30s of 5m0s: 870434 executions, 539480 successful
    rsg_test.go:828: 3m35s of 5m0s: 891092 executions, 552318 successful
    rsg_test.go:828: 3m40s of 5m0s: 912396 executions, 565576 successful
    rsg_test.go:828: 3m45s of 5m0s: 933257 executions, 578543 successful
    rsg_test.go:828: 3m50s of 5m0s: 954243 executions, 591530 successful
    rsg_test.go:828: 3m55s of 5m0s: 975888 executions, 605082 successful
    rsg_test.go:828: 4m0s of 5m0s: 999068 executions, 619401 successful
    rsg_test.go:828: 4m5s of 5m0s: 1019191 executions, 631844 successful
    rsg_test.go:828: 4m10s of 5m0s: 1043209 executions, 646753 successful
    rsg_test.go:828: 4m15s of 5m0s: 1067971 executions, 662101 successful
    rsg_test.go:828: 4m20s of 5m0s: 1089375 executions, 675352 successful
    rsg_test.go:828: 4m25s of 5m0s: 1110058 executions, 688185 successful
    rsg_test.go:828: 4m30s of 5m0s: 1132673 executions, 702064 successful
    rsg_test.go:828: 4m35s of 5m0s: 1153665 executions, 715100 successful
    rsg_test.go:828: 4m40s of 5m0s: 1174356 executions, 727936 successful
    rsg_test.go:828: 4m45s of 5m0s: 1196325 executions, 741617 successful
    rsg_test.go:828: 4m50s of 5m0s: 1216478 executions, 754072 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 35b724947ac464e6af45ce5193e34035fcacb1ef:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 15330553 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 186730 executions, 115962 successful
    rsg_test.go:828: 50s of 5m0s: 204669 executions, 127002 successful
    rsg_test.go:828: 55s of 5m0s: 225987 executions, 140283 successful
    rsg_test.go:828: 1m0s of 5m0s: 246745 executions, 153107 successful
    rsg_test.go:828: 1m5s of 5m0s: 269495 executions, 167344 successful
    rsg_test.go:828: 1m10s of 5m0s: 288685 executions, 179260 successful
    rsg_test.go:828: 1m15s of 5m0s: 307826 executions, 191102 successful
    rsg_test.go:828: 1m20s of 5m0s: 328231 executions, 203659 successful
    rsg_test.go:828: 1m25s of 5m0s: 348924 executions, 216448 successful
    rsg_test.go:828: 1m30s of 5m0s: 369877 executions, 229385 successful
    rsg_test.go:828: 1m35s of 5m0s: 390228 executions, 242093 successful
    rsg_test.go:828: 1m40s of 5m0s: 409874 executions, 254180 successful
    rsg_test.go:828: 1m45s of 5m0s: 432370 executions, 268207 successful
    rsg_test.go:828: 1m50s of 5m0s: 451842 executions, 280221 successful
    rsg_test.go:828: 1m55s of 5m0s: 472084 executions, 292668 successful
    rsg_test.go:828: 2m0s of 5m0s: 490768 executions, 304234 successful
    rsg_test.go:828: 2m5s of 5m0s: 511278 executions, 317056 successful
    rsg_test.go:828: 2m10s of 5m0s: 534161 executions, 331271 successful
    rsg_test.go:828: 2m15s of 5m0s: 556323 executions, 344919 successful
    rsg_test.go:828: 2m20s of 5m0s: 576143 executions, 357198 successful
    rsg_test.go:828: 2m25s of 5m0s: 598956 executions, 371331 successful
    rsg_test.go:828: 2m30s of 5m0s: 616918 executions, 382476 successful
    rsg_test.go:828: 2m35s of 5m0s: 637217 executions, 394971 successful
    rsg_test.go:828: 2m40s of 5m0s: 657054 executions, 407352 successful
    rsg_test.go:828: 2m45s of 5m0s: 678514 executions, 420683 successful
    rsg_test.go:828: 2m50s of 5m0s: 699322 executions, 433597 successful
    rsg_test.go:828: 2m55s of 5m0s: 722759 executions, 448097 successful
    rsg_test.go:828: 3m0s of 5m0s: 742973 executions, 460633 successful
    rsg_test.go:828: 3m5s of 5m0s: 765944 executions, 474790 successful
    rsg_test.go:828: 3m10s of 5m0s: 786536 executions, 487541 successful
    rsg_test.go:828: 3m15s of 5m0s: 811578 executions, 503063 successful
    rsg_test.go:828: 3m20s of 5m0s: 831918 executions, 515717 successful
    rsg_test.go:828: 3m25s of 5m0s: 853128 executions, 528830 successful
    rsg_test.go:828: 3m30s of 5m0s: 872796 executions, 540915 successful
    rsg_test.go:828: 3m35s of 5m0s: 895156 executions, 554803 successful
    rsg_test.go:828: 3m40s of 5m0s: 917481 executions, 568592 successful
    rsg_test.go:828: 3m45s of 5m0s: 938413 executions, 581583 successful
    rsg_test.go:828: 3m50s of 5m0s: 960231 executions, 595169 successful
    rsg_test.go:828: 3m55s of 5m0s: 982316 executions, 608751 successful
    rsg_test.go:828: 4m0s of 5m0s: 1002739 executions, 621490 successful
    rsg_test.go:828: 4m5s of 5m0s: 1020011 executions, 632227 successful
    rsg_test.go:828: 4m10s of 5m0s: 1040195 executions, 644683 successful
    rsg_test.go:828: 4m15s of 5m0s: 1059747 executions, 656772 successful
    rsg_test.go:828: 4m20s of 5m0s: 1082143 executions, 670676 successful
    rsg_test.go:828: 4m25s of 5m0s: 1104264 executions, 684496 successful
    rsg_test.go:828: 4m30s of 5m0s: 1124344 executions, 696817 successful
    rsg_test.go:828: 4m35s of 5m0s: 1143343 executions, 708627 successful
    rsg_test.go:828: 4m40s of 5m0s: 1165104 executions, 722146 successful
    rsg_test.go:828: 4m45s of 5m0s: 1186493 executions, 735378 successful
    rsg_test.go:828: 4m50s of 5m0s: 1209207 executions, 749482 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ e847e4358c004d4a8977ae26749768ad1d417fba:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14288694 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 165750 executions, 102717 successful
    rsg_test.go:828: 50s of 5m0s: 183519 executions, 113660 successful
    rsg_test.go:828: 55s of 5m0s: 204170 executions, 126523 successful
    rsg_test.go:828: 1m0s of 5m0s: 220374 executions, 136519 successful
    rsg_test.go:828: 1m5s of 5m0s: 238836 executions, 148059 successful
    rsg_test.go:828: 1m10s of 5m0s: 258706 executions, 160338 successful
    rsg_test.go:828: 1m15s of 5m0s: 275867 executions, 170904 successful
    rsg_test.go:828: 1m20s of 5m0s: 295828 executions, 183354 successful
    rsg_test.go:828: 1m25s of 5m0s: 314089 executions, 194534 successful
    rsg_test.go:828: 1m30s of 5m0s: 331695 executions, 205520 successful
    rsg_test.go:828: 1m35s of 5m0s: 349518 executions, 216507 successful
    rsg_test.go:828: 1m40s of 5m0s: 370152 executions, 229260 successful
    rsg_test.go:828: 1m45s of 5m0s: 388319 executions, 240588 successful
    rsg_test.go:828: 1m50s of 5m0s: 405712 executions, 251355 successful
    rsg_test.go:828: 1m55s of 5m0s: 425777 executions, 263657 successful
    rsg_test.go:828: 2m0s of 5m0s: 446736 executions, 276642 successful
    rsg_test.go:828: 2m5s of 5m0s: 466326 executions, 288738 successful
    rsg_test.go:828: 2m10s of 5m0s: 485599 executions, 300739 successful
    rsg_test.go:828: 2m15s of 5m0s: 505549 executions, 313158 successful
    rsg_test.go:828: 2m20s of 5m0s: 525719 executions, 325548 successful
    rsg_test.go:828: 2m25s of 5m0s: 543896 executions, 336841 successful
    rsg_test.go:828: 2m30s of 5m0s: 564240 executions, 349361 successful
    rsg_test.go:828: 2m35s of 5m0s: 584251 executions, 361805 successful
    rsg_test.go:828: 2m40s of 5m0s: 604592 executions, 374360 successful
    rsg_test.go:828: 2m45s of 5m0s: 626012 executions, 387628 successful
    rsg_test.go:828: 2m50s of 5m0s: 644795 executions, 399226 successful
    rsg_test.go:828: 2m55s of 5m0s: 664678 executions, 411603 successful
    rsg_test.go:828: 3m0s of 5m0s: 684170 executions, 423671 successful
    rsg_test.go:828: 3m5s of 5m0s: 702753 executions, 435050 successful
    rsg_test.go:828: 3m10s of 5m0s: 722991 executions, 447640 successful
    rsg_test.go:828: 3m15s of 5m0s: 744171 executions, 460794 successful
    rsg_test.go:828: 3m20s of 5m0s: 761014 executions, 471310 successful
    rsg_test.go:828: 3m25s of 5m0s: 782742 executions, 484722 successful
    rsg_test.go:828: 3m30s of 5m0s: 803184 executions, 497455 successful
    rsg_test.go:828: 3m35s of 5m0s: 821864 executions, 508968 successful
    rsg_test.go:828: 3m40s of 5m0s: 839817 executions, 520064 successful
    rsg_test.go:828: 3m45s of 5m0s: 860534 executions, 532882 successful
    rsg_test.go:828: 3m50s of 5m0s: 880009 executions, 544907 successful
    rsg_test.go:828: 3m55s of 5m0s: 900762 executions, 557806 successful
    rsg_test.go:828: 4m0s of 5m0s: 919713 executions, 569506 successful
    rsg_test.go:828: 4m5s of 5m0s: 939990 executions, 582051 successful
    rsg_test.go:828: 4m10s of 5m0s: 958763 executions, 593722 successful
    rsg_test.go:828: 4m15s of 5m0s: 980264 executions, 607010 successful
    rsg_test.go:828: 4m20s of 5m0s: 1001007 executions, 619960 successful
    rsg_test.go:828: 4m25s of 5m0s: 1019261 executions, 631218 successful
    rsg_test.go:828: 4m30s of 5m0s: 1037427 executions, 642576 successful
    rsg_test.go:828: 4m35s of 5m0s: 1056448 executions, 654403 successful
    rsg_test.go:828: 4m40s of 5m0s: 1075861 executions, 666390 successful
    rsg_test.go:828: 4m45s of 5m0s: 1094095 executions, 677648 successful
    rsg_test.go:828: 4m50s of 5m0s: 1116130 executions, 691383 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ b432e8c20339de5cfa7c811a9ee6f5dc98d15a1e:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14608934 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

=== RUN   TestRandomSyntaxFunctions
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/d437d2c847dfedbc4972f231c3331c8e/logTestRandomSyntaxFunctions119133282
    test_log_scope.go:79: use -show-logs to present logs inline
    rsg_test.go:828: 5s of 5m0s: 18384 executions, 11359 successful
    rsg_test.go:828: 10s of 5m0s: 36593 executions, 22721 successful
    rsg_test.go:828: 15s of 5m0s: 57199 executions, 35450 successful
    rsg_test.go:828: 20s of 5m0s: 77640 executions, 48119 successful
    rsg_test.go:828: 25s of 5m0s: 96092 executions, 59574 successful
    rsg_test.go:828: 30s of 5m0s: 116328 executions, 72093 successful
    rsg_test.go:828: 35s of 5m0s: 137006 executions, 85029 successful
    rsg_test.go:828: 40s of 5m0s: 154143 executions, 95660 successful
    rsg_test.go:828: 45s of 5m0s: 169666 executions, 105232 successful
    rsg_test.go:828: 50s of 5m0s: 186078 executions, 115426 successful
    rsg_test.go:828: 55s of 5m0s: 206009 executions, 127845 successful
    rsg_test.go:828: 1m0s of 5m0s: 226237 executions, 140293 successful
    rsg_test.go:828: 1m5s of 5m0s: 247309 executions, 153378 successful
    rsg_test.go:828: 1m10s of 5m0s: 267344 executions, 165841 successful
    rsg_test.go:828: 1m15s of 5m0s: 287211 executions, 178026 successful
    rsg_test.go:828: 1m20s of 5m0s: 307197 executions, 190538 successful
    rsg_test.go:828: 1m25s of 5m0s: 330675 executions, 205222 successful
    rsg_test.go:828: 1m30s of 5m0s: 351032 executions, 217774 successful
    rsg_test.go:828: 1m35s of 5m0s: 370093 executions, 229515 successful
    rsg_test.go:828: 1m40s of 5m0s: 390262 executions, 242071 successful
    rsg_test.go:828: 1m45s of 5m0s: 411994 executions, 255395 successful
    rsg_test.go:828: 1m50s of 5m0s: 433683 executions, 268850 successful
    rsg_test.go:828: 1m55s of 5m0s: 452332 executions, 280375 successful
    rsg_test.go:828: 2m0s of 5m0s: 469481 executions, 290996 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ af5e6abf7efbb1361252839703e3ef8e067c9410:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 13131021 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 163335 executions, 101216 successful
    rsg_test.go:828: 50s of 5m0s: 183936 executions, 113933 successful
    rsg_test.go:828: 55s of 5m0s: 203871 executions, 126251 successful
    rsg_test.go:828: 1m0s of 5m0s: 219052 executions, 135676 successful
    rsg_test.go:828: 1m5s of 5m0s: 237307 executions, 146931 successful
    rsg_test.go:828: 1m10s of 5m0s: 254789 executions, 157842 successful
    rsg_test.go:828: 1m15s of 5m0s: 269467 executions, 167023 successful
    rsg_test.go:828: 1m20s of 5m0s: 286519 executions, 177579 successful
    rsg_test.go:828: 1m25s of 5m0s: 304071 executions, 188468 successful
    rsg_test.go:828: 1m30s of 5m0s: 322629 executions, 199950 successful
    rsg_test.go:828: 1m35s of 5m0s: 341121 executions, 211378 successful
    rsg_test.go:828: 1m40s of 5m0s: 357662 executions, 221650 successful
    rsg_test.go:828: 1m45s of 5m0s: 376166 executions, 233069 successful
    rsg_test.go:828: 1m50s of 5m0s: 393558 executions, 243872 successful
    rsg_test.go:828: 1m55s of 5m0s: 409675 executions, 253984 successful
    rsg_test.go:828: 2m0s of 5m0s: 431594 executions, 267490 successful
    rsg_test.go:828: 2m5s of 5m0s: 448865 executions, 278239 successful
    rsg_test.go:828: 2m10s of 5m0s: 466260 executions, 288889 successful
    rsg_test.go:828: 2m15s of 5m0s: 486129 executions, 301353 successful
    rsg_test.go:828: 2m20s of 5m0s: 503527 executions, 312191 successful
    rsg_test.go:828: 2m25s of 5m0s: 519459 executions, 321939 successful
    rsg_test.go:828: 2m30s of 5m0s: 539007 executions, 334120 successful
    rsg_test.go:828: 2m35s of 5m0s: 559361 executions, 346737 successful
    rsg_test.go:828: 2m40s of 5m0s: 577399 executions, 357860 successful
    rsg_test.go:828: 2m45s of 5m0s: 595887 executions, 369307 successful
    rsg_test.go:828: 2m50s of 5m0s: 614044 executions, 380557 successful
    rsg_test.go:828: 2m55s of 5m0s: 629689 executions, 390248 successful
    rsg_test.go:828: 3m0s of 5m0s: 646243 executions, 400432 successful
    rsg_test.go:828: 3m5s of 5m0s: 662805 executions, 410813 successful
    rsg_test.go:828: 3m10s of 5m0s: 680899 executions, 421878 successful
    rsg_test.go:828: 3m15s of 5m0s: 699846 executions, 433651 successful
    rsg_test.go:828: 3m20s of 5m0s: 715893 executions, 443684 successful
    rsg_test.go:828: 3m25s of 5m0s: 736718 executions, 456662 successful
    rsg_test.go:828: 3m30s of 5m0s: 755711 executions, 468490 successful
    rsg_test.go:828: 3m35s of 5m0s: 774810 executions, 480309 successful
    rsg_test.go:828: 3m40s of 5m0s: 794861 executions, 492767 successful
    rsg_test.go:828: 3m45s of 5m0s: 814808 executions, 505145 successful
    rsg_test.go:828: 3m50s of 5m0s: 833317 executions, 516524 successful
    rsg_test.go:828: 3m55s of 5m0s: 851976 executions, 528070 successful
    rsg_test.go:828: 4m0s of 5m0s: 871562 executions, 540298 successful
    rsg_test.go:828: 4m5s of 5m0s: 888891 executions, 551025 successful
    rsg_test.go:828: 4m10s of 5m0s: 908836 executions, 563384 successful
    rsg_test.go:828: 4m15s of 5m0s: 926030 executions, 574000 successful
    rsg_test.go:828: 4m20s of 5m0s: 943420 executions, 584768 successful
    rsg_test.go:828: 4m25s of 5m0s: 962485 executions, 596624 successful
    rsg_test.go:828: 4m30s of 5m0s: 977101 executions, 605612 successful
    rsg_test.go:828: 4m35s of 5m0s: 995187 executions, 616835 successful
    rsg_test.go:828: 4m40s of 5m0s: 1011879 executions, 627107 successful
    rsg_test.go:828: 4m45s of 5m0s: 1028309 executions, 637310 successful
    rsg_test.go:828: 4m50s of 5m0s: 1046748 executions, 648868 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ de239a7438f44d382c9aefceb65d9c39911dabd2:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14761221 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 183477 executions, 113716 successful
    rsg_test.go:828: 50s of 5m0s: 203302 executions, 126014 successful
    rsg_test.go:828: 55s of 5m0s: 221838 executions, 137459 successful
    rsg_test.go:828: 1m0s of 5m0s: 237333 executions, 147087 successful
    rsg_test.go:828: 1m5s of 5m0s: 253354 executions, 156951 successful
    rsg_test.go:828: 1m10s of 5m0s: 275195 executions, 170532 successful
    rsg_test.go:828: 1m15s of 5m0s: 296389 executions, 183643 successful
    rsg_test.go:828: 1m20s of 5m0s: 318870 executions, 197552 successful
    rsg_test.go:828: 1m25s of 5m0s: 341893 executions, 211838 successful
    rsg_test.go:828: 1m30s of 5m0s: 358382 executions, 222050 successful
    rsg_test.go:828: 1m35s of 5m0s: 379969 executions, 235439 successful
    rsg_test.go:828: 1m40s of 5m0s: 397443 executions, 246291 successful
    rsg_test.go:828: 1m45s of 5m0s: 418490 executions, 259417 successful
    rsg_test.go:828: 1m50s of 5m0s: 434763 executions, 269520 successful
    rsg_test.go:828: 1m55s of 5m0s: 455847 executions, 282655 successful
    rsg_test.go:828: 2m0s of 5m0s: 476149 executions, 295215 successful
    rsg_test.go:828: 2m5s of 5m0s: 495262 executions, 307025 successful
    rsg_test.go:828: 2m10s of 5m0s: 515196 executions, 319348 successful
    rsg_test.go:828: 2m15s of 5m0s: 533393 executions, 330616 successful
    rsg_test.go:828: 2m20s of 5m0s: 554644 executions, 343654 successful
    rsg_test.go:828: 2m25s of 5m0s: 574683 executions, 356192 successful
    rsg_test.go:828: 2m30s of 5m0s: 595145 executions, 368832 successful
    rsg_test.go:828: 2m35s of 5m0s: 611812 executions, 379155 successful
    rsg_test.go:828: 2m40s of 5m0s: 631437 executions, 391317 successful
    rsg_test.go:828: 2m45s of 5m0s: 651823 executions, 403938 successful
    rsg_test.go:828: 2m50s of 5m0s: 670926 executions, 415845 successful
    rsg_test.go:828: 2m55s of 5m0s: 691446 executions, 428540 successful
    rsg_test.go:828: 3m0s of 5m0s: 713044 executions, 441902 successful
    rsg_test.go:828: 3m5s of 5m0s: 733414 executions, 454441 successful
    rsg_test.go:828: 3m10s of 5m0s: 752520 executions, 466201 successful
    rsg_test.go:828: 3m15s of 5m0s: 774726 executions, 479840 successful
    rsg_test.go:828: 3m20s of 5m0s: 796413 executions, 493340 successful
    rsg_test.go:828: 3m25s of 5m0s: 816479 executions, 505759 successful
    rsg_test.go:828: 3m30s of 5m0s: 838206 executions, 519183 successful
    rsg_test.go:828: 3m35s of 5m0s: 856808 executions, 530804 successful
    rsg_test.go:828: 3m40s of 5m0s: 879107 executions, 544527 successful
    rsg_test.go:828: 3m45s of 5m0s: 899421 executions, 557131 successful
    rsg_test.go:828: 3m50s of 5m0s: 919436 executions, 569516 successful
    rsg_test.go:828: 3m55s of 5m0s: 938144 executions, 581144 successful
    rsg_test.go:828: 4m0s of 5m0s: 957584 executions, 593194 successful
    rsg_test.go:828: 4m5s of 5m0s: 978942 executions, 606350 successful
    rsg_test.go:828: 4m10s of 5m0s: 996953 executions, 617556 successful
    rsg_test.go:828: 4m15s of 5m0s: 1017104 executions, 629954 successful
    rsg_test.go:828: 4m20s of 5m0s: 1037453 executions, 642596 successful
    rsg_test.go:828: 4m25s of 5m0s: 1057893 executions, 655217 successful
    rsg_test.go:828: 4m30s of 5m0s: 1078465 executions, 667999 successful
    rsg_test.go:828: 4m35s of 5m0s: 1098519 executions, 680395 successful
    rsg_test.go:828: 4m40s of 5m0s: 1119455 executions, 693333 successful
    rsg_test.go:828: 4m45s of 5m0s: 1140659 executions, 706411 successful
    rsg_test.go:828: 4m50s of 5m0s: 1161574 executions, 719324 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 2f96695f75b07c872ec5f146acc1fa198135768f:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 14596195 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 186628 executions, 115585 successful
    rsg_test.go:828: 50s of 5m0s: 204660 executions, 126714 successful
    rsg_test.go:828: 55s of 5m0s: 226001 executions, 139939 successful
    rsg_test.go:828: 1m0s of 5m0s: 242066 executions, 149935 successful
    rsg_test.go:828: 1m5s of 5m0s: 262802 executions, 162692 successful
    rsg_test.go:828: 1m10s of 5m0s: 286890 executions, 177682 successful
    rsg_test.go:828: 1m15s of 5m0s: 306410 executions, 189721 successful
    rsg_test.go:828: 1m20s of 5m0s: 325665 executions, 201679 successful
    rsg_test.go:828: 1m25s of 5m0s: 346323 executions, 214478 successful
    rsg_test.go:828: 1m30s of 5m0s: 364478 executions, 225672 successful
    rsg_test.go:828: 1m35s of 5m0s: 387448 executions, 239887 successful
    rsg_test.go:828: 1m40s of 5m0s: 408652 executions, 252949 successful
    rsg_test.go:828: 1m45s of 5m0s: 428698 executions, 265429 successful
    rsg_test.go:828: 1m50s of 5m0s: 449082 executions, 277912 successful
    rsg_test.go:828: 1m55s of 5m0s: 470504 executions, 291137 successful
    rsg_test.go:828: 2m0s of 5m0s: 489914 executions, 303302 successful
    rsg_test.go:828: 2m5s of 5m0s: 509470 executions, 315467 successful
    rsg_test.go:828: 2m10s of 5m0s: 528788 executions, 327474 successful
    rsg_test.go:828: 2m15s of 5m0s: 549496 executions, 340336 successful
    rsg_test.go:828: 2m20s of 5m0s: 569007 executions, 352351 successful
    rsg_test.go:828: 2m25s of 5m0s: 590505 executions, 365575 successful
    rsg_test.go:828: 2m30s of 5m0s: 608275 executions, 376561 successful
    rsg_test.go:828: 2m35s of 5m0s: 626299 executions, 387795 successful
    rsg_test.go:828: 2m40s of 5m0s: 646107 executions, 400004 successful
    rsg_test.go:828: 2m45s of 5m0s: 665690 executions, 412134 successful
    rsg_test.go:828: 2m50s of 5m0s: 684030 executions, 423593 successful
    rsg_test.go:828: 2m55s of 5m0s: 705079 executions, 436613 successful
    rsg_test.go:828: 3m0s of 5m0s: 726421 executions, 449804 successful
    rsg_test.go:828: 3m5s of 5m0s: 744957 executions, 461348 successful
    rsg_test.go:828: 3m10s of 5m0s: 766946 executions, 474959 successful
    rsg_test.go:828: 3m15s of 5m0s: 784445 executions, 485790 successful
    rsg_test.go:828: 3m20s of 5m0s: 803635 executions, 497718 successful
    rsg_test.go:828: 3m25s of 5m0s: 825795 executions, 511492 successful
    rsg_test.go:828: 3m30s of 5m0s: 847337 executions, 524788 successful
    rsg_test.go:828: 3m35s of 5m0s: 868456 executions, 537902 successful
    rsg_test.go:828: 3m40s of 5m0s: 888456 executions, 550178 successful
    rsg_test.go:828: 3m45s of 5m0s: 909722 executions, 563386 successful
    rsg_test.go:828: 3m50s of 5m0s: 929931 executions, 575942 successful
    rsg_test.go:828: 3m55s of 5m0s: 952698 executions, 590051 successful
    rsg_test.go:828: 4m0s of 5m0s: 971152 executions, 601377 successful
    rsg_test.go:828: 4m5s of 5m0s: 993132 executions, 615023 successful
    rsg_test.go:828: 4m10s of 5m0s: 1013287 executions, 627440 successful
    rsg_test.go:828: 4m15s of 5m0s: 1031473 executions, 638807 successful
    rsg_test.go:828: 4m20s of 5m0s: 1052493 executions, 651799 successful
    rsg_test.go:828: 4m25s of 5m0s: 1073034 executions, 664399 successful
    rsg_test.go:828: 4m30s of 5m0s: 1092984 executions, 676806 successful
    rsg_test.go:828: 4m35s of 5m0s: 1114877 executions, 690290 successful
    rsg_test.go:828: 4m40s of 5m0s: 1137030 executions, 704082 successful
    rsg_test.go:828: 4m45s of 5m0s: 1155417 executions, 715365 successful
    rsg_test.go:828: 4m50s of 5m0s: 1175534 executions, 727841 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ a9eb6d220fc0957d4f6f8261d447991e9d03b4ad:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 15905970 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 172596 executions, 106959 successful
    rsg_test.go:828: 50s of 5m0s: 189223 executions, 117229 successful
    rsg_test.go:828: 55s of 5m0s: 207035 executions, 128292 successful
    rsg_test.go:828: 1m0s of 5m0s: 229049 executions, 141949 successful
    rsg_test.go:828: 1m5s of 5m0s: 247163 executions, 153120 successful
    rsg_test.go:828: 1m10s of 5m0s: 267959 executions, 166003 successful
    rsg_test.go:828: 1m15s of 5m0s: 287429 executions, 178087 successful
    rsg_test.go:828: 1m20s of 5m0s: 306015 executions, 189628 successful
    rsg_test.go:828: 1m25s of 5m0s: 326428 executions, 202280 successful
    rsg_test.go:828: 1m30s of 5m0s: 349083 executions, 216298 successful
    rsg_test.go:828: 1m35s of 5m0s: 369028 executions, 228718 successful
    rsg_test.go:828: 1m40s of 5m0s: 388339 executions, 240645 successful
    rsg_test.go:828: 1m45s of 5m0s: 408632 executions, 253049 successful
    rsg_test.go:828: 1m50s of 5m0s: 427774 executions, 264992 successful
    rsg_test.go:828: 1m55s of 5m0s: 448067 executions, 277594 successful
    rsg_test.go:828: 2m0s of 5m0s: 468945 executions, 290530 successful
    rsg_test.go:828: 2m5s of 5m0s: 488197 executions, 302404 successful
    rsg_test.go:828: 2m10s of 5m0s: 507984 executions, 314557 successful
    rsg_test.go:828: 2m15s of 5m0s: 527693 executions, 326726 successful
    rsg_test.go:828: 2m20s of 5m0s: 548925 executions, 339934 successful
    rsg_test.go:828: 2m25s of 5m0s: 568331 executions, 352033 successful
    rsg_test.go:828: 2m30s of 5m0s: 588223 executions, 364401 successful
    rsg_test.go:828: 2m35s of 5m0s: 606505 executions, 375705 successful
    rsg_test.go:828: 2m40s of 5m0s: 625594 executions, 387625 successful
    rsg_test.go:828: 2m45s of 5m0s: 645760 executions, 400129 successful
    rsg_test.go:828: 2m50s of 5m0s: 665937 executions, 412560 successful
    rsg_test.go:828: 2m55s of 5m0s: 685451 executions, 424597 successful
    rsg_test.go:828: 3m0s of 5m0s: 705304 executions, 436999 successful
    rsg_test.go:828: 3m5s of 5m0s: 724929 executions, 449076 successful
    rsg_test.go:828: 3m10s of 5m0s: 744095 executions, 460934 successful
    rsg_test.go:828: 3m15s of 5m0s: 761824 executions, 471933 successful
    rsg_test.go:828: 3m20s of 5m0s: 781945 executions, 484478 successful
    rsg_test.go:828: 3m25s of 5m0s: 802290 executions, 496985 successful
    rsg_test.go:828: 3m30s of 5m0s: 823628 executions, 510284 successful
    rsg_test.go:828: 3m35s of 5m0s: 846824 executions, 524757 successful
    rsg_test.go:828: 3m40s of 5m0s: 867709 executions, 537691 successful
    rsg_test.go:828: 3m45s of 5m0s: 888414 executions, 550539 successful
    rsg_test.go:828: 3m50s of 5m0s: 909987 executions, 563898 successful
    rsg_test.go:828: 3m55s of 5m0s: 933314 executions, 578464 successful
    rsg_test.go:828: 4m0s of 5m0s: 953108 executions, 590780 successful
    rsg_test.go:828: 4m5s of 5m0s: 972655 executions, 602831 successful
    rsg_test.go:828: 4m10s of 5m0s: 992579 executions, 615237 successful
    rsg_test.go:828: 4m15s of 5m0s: 1014885 executions, 629084 successful
    rsg_test.go:828: 4m20s of 5m0s: 1033207 executions, 640518 successful
    rsg_test.go:828: 4m25s of 5m0s: 1055372 executions, 654136 successful
    rsg_test.go:828: 4m30s of 5m0s: 1075402 executions, 666614 successful
    rsg_test.go:828: 4m35s of 5m0s: 1095989 executions, 679322 successful
    rsg_test.go:828: 4m40s of 5m0s: 1116186 executions, 691832 successful
    rsg_test.go:828: 4m45s of 5m0s: 1136643 executions, 704397 successful
    rsg_test.go:828: 4m50s of 5m0s: 1158255 executions, 717830 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ c8a4703f9853a442ac676b44d074b43eb387f60c:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 15116313 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 177977 executions, 110190 successful
    rsg_test.go:828: 50s of 5m0s: 197379 executions, 122271 successful
    rsg_test.go:828: 55s of 5m0s: 218521 executions, 135338 successful
    rsg_test.go:828: 1m0s of 5m0s: 240563 executions, 149137 successful
    rsg_test.go:828: 1m5s of 5m0s: 261794 executions, 162283 successful
    rsg_test.go:828: 1m10s of 5m0s: 282106 executions, 174907 successful
    rsg_test.go:828: 1m15s of 5m0s: 304659 executions, 188901 successful
    rsg_test.go:828: 1m20s of 5m0s: 322542 executions, 199986 successful
    rsg_test.go:828: 1m25s of 5m0s: 343477 executions, 212962 successful
    rsg_test.go:828: 1m30s of 5m0s: 363962 executions, 225628 successful
    rsg_test.go:828: 1m35s of 5m0s: 384657 executions, 238507 successful
    rsg_test.go:828: 1m40s of 5m0s: 403330 executions, 250014 successful
    rsg_test.go:828: 1m45s of 5m0s: 421559 executions, 261193 successful
    rsg_test.go:828: 1m50s of 5m0s: 441904 executions, 273884 successful
    rsg_test.go:828: 1m55s of 5m0s: 464063 executions, 287518 successful
    rsg_test.go:828: 2m0s of 5m0s: 485214 executions, 300637 successful
    rsg_test.go:828: 2m5s of 5m0s: 503604 executions, 312093 successful
    rsg_test.go:828: 2m10s of 5m0s: 520133 executions, 322181 successful
    rsg_test.go:828: 2m15s of 5m0s: 539758 executions, 334382 successful
    rsg_test.go:828: 2m20s of 5m0s: 560803 executions, 347365 successful
    rsg_test.go:828: 2m25s of 5m0s: 581843 executions, 360376 successful
    rsg_test.go:828: 2m30s of 5m0s: 602619 executions, 373268 successful
    rsg_test.go:828: 2m35s of 5m0s: 621909 executions, 385279 successful
    rsg_test.go:828: 2m40s of 5m0s: 643649 executions, 398907 successful
    rsg_test.go:828: 2m45s of 5m0s: 663540 executions, 411098 successful
    rsg_test.go:828: 2m50s of 5m0s: 682387 executions, 422843 successful
    rsg_test.go:828: 2m55s of 5m0s: 703386 executions, 435887 successful
    rsg_test.go:828: 3m0s of 5m0s: 724483 executions, 449007 successful
    rsg_test.go:828: 3m5s of 5m0s: 745450 executions, 462043 successful
    rsg_test.go:828: 3m10s of 5m0s: 766518 executions, 475143 successful
    rsg_test.go:828: 3m15s of 5m0s: 790813 executions, 490146 successful
    rsg_test.go:828: 3m20s of 5m0s: 812102 executions, 503407 successful
    rsg_test.go:828: 3m25s of 5m0s: 832780 executions, 516278 successful
    rsg_test.go:828: 3m30s of 5m0s: 854947 executions, 529943 successful
    rsg_test.go:828: 3m35s of 5m0s: 873902 executions, 541607 successful
    rsg_test.go:828: 3m40s of 5m0s: 890084 executions, 551543 successful
    rsg_test.go:828: 3m45s of 5m0s: 911803 executions, 565002 successful
    rsg_test.go:828: 3m50s of 5m0s: 931830 executions, 577538 successful
    rsg_test.go:828: 3m55s of 5m0s: 948554 executions, 587875 successful
    rsg_test.go:828: 4m0s of 5m0s: 969196 executions, 600708 successful
    rsg_test.go:828: 4m5s of 5m0s: 990721 executions, 613975 successful
    rsg_test.go:828: 4m10s of 5m0s: 1009136 executions, 625311 successful
    rsg_test.go:828: 4m15s of 5m0s: 1028596 executions, 637448 successful
    rsg_test.go:828: 4m20s of 5m0s: 1049080 executions, 650186 successful
    rsg_test.go:828: 4m25s of 5m0s: 1069315 executions, 662691 successful
    rsg_test.go:828: 4m30s of 5m0s: 1089099 executions, 675055 successful
    rsg_test.go:828: 4m35s of 5m0s: 1107617 executions, 686493 successful
    rsg_test.go:828: 4m40s of 5m0s: 1126950 executions, 698487 successful
    rsg_test.go:828: 4m45s of 5m0s: 1150267 executions, 712957 successful
    rsg_test.go:828: 4m50s of 5m0s: 1169577 executions, 724933 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ cf84c1c11f2f9c1a4a979033fa0af3befb2ccbcf:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 12932073 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 148441 executions, 91974 successful
    rsg_test.go:828: 50s of 5m0s: 166982 executions, 103467 successful
    rsg_test.go:828: 55s of 5m0s: 185055 executions, 114733 successful
    rsg_test.go:828: 1m0s of 5m0s: 203789 executions, 126400 successful
    rsg_test.go:828: 1m5s of 5m0s: 223138 executions, 138348 successful
    rsg_test.go:828: 1m10s of 5m0s: 241023 executions, 149289 successful
    rsg_test.go:828: 1m15s of 5m0s: 260918 executions, 161653 successful
    rsg_test.go:828: 1m20s of 5m0s: 279827 executions, 173381 successful
    rsg_test.go:828: 1m25s of 5m0s: 295900 executions, 183415 successful
    rsg_test.go:828: 1m30s of 5m0s: 313587 executions, 194369 successful
    rsg_test.go:828: 1m35s of 5m0s: 332626 executions, 206211 successful
    rsg_test.go:828: 1m40s of 5m0s: 351836 executions, 218104 successful
    rsg_test.go:828: 1m45s of 5m0s: 370493 executions, 229541 successful
    rsg_test.go:828: 1m50s of 5m0s: 389956 executions, 241660 successful
    rsg_test.go:828: 1m55s of 5m0s: 406816 executions, 252066 successful
    rsg_test.go:828: 2m0s of 5m0s: 423004 executions, 262236 successful
    rsg_test.go:828: 2m5s of 5m0s: 440459 executions, 273007 successful
    rsg_test.go:828: 2m10s of 5m0s: 455746 executions, 282532 successful
    rsg_test.go:828: 2m15s of 5m0s: 475245 executions, 294543 successful
    rsg_test.go:828: 2m20s of 5m0s: 493191 executions, 305729 successful
    rsg_test.go:828: 2m25s of 5m0s: 513596 executions, 318355 successful
    rsg_test.go:828: 2m30s of 5m0s: 531714 executions, 329590 successful
    rsg_test.go:828: 2m35s of 5m0s: 550305 executions, 341035 successful
    rsg_test.go:828: 2m40s of 5m0s: 567524 executions, 351688 successful
    rsg_test.go:828: 2m45s of 5m0s: 588001 executions, 364492 successful
    rsg_test.go:828: 2m50s of 5m0s: 609680 executions, 377860 successful
    rsg_test.go:828: 2m55s of 5m0s: 626550 executions, 388336 successful
    rsg_test.go:828: 3m0s of 5m0s: 647180 executions, 401101 successful
    rsg_test.go:828: 3m5s of 5m0s: 670689 executions, 415683 successful
    rsg_test.go:828: 3m10s of 5m0s: 690990 executions, 428271 successful
    rsg_test.go:828: 3m15s of 5m0s: 714148 executions, 442628 successful
    rsg_test.go:828: 3m20s of 5m0s: 732084 executions, 453819 successful
    rsg_test.go:828: 3m25s of 5m0s: 753319 executions, 466933 successful
    rsg_test.go:828: 3m30s of 5m0s: 774915 executions, 480203 successful
    rsg_test.go:828: 3m35s of 5m0s: 798688 executions, 494937 successful
    rsg_test.go:828: 3m40s of 5m0s: 820419 executions, 508492 successful
    rsg_test.go:828: 3m45s of 5m0s: 840117 executions, 520638 successful
    rsg_test.go:828: 3m50s of 5m0s: 862315 executions, 534538 successful
    rsg_test.go:828: 3m55s of 5m0s: 885190 executions, 548656 successful
    rsg_test.go:828: 4m0s of 5m0s: 904804 executions, 560853 successful
    rsg_test.go:828: 4m5s of 5m0s: 923285 executions, 572203 successful
    rsg_test.go:828: 4m10s of 5m0s: 942953 executions, 584495 successful
    rsg_test.go:828: 4m15s of 5m0s: 964042 executions, 597656 successful
    rsg_test.go:828: 4m20s of 5m0s: 986019 executions, 611192 successful
    rsg_test.go:828: 4m25s of 5m0s: 1006781 executions, 624162 successful
    rsg_test.go:828: 4m30s of 5m0s: 1029732 executions, 638395 successful
    rsg_test.go:828: 4m35s of 5m0s: 1048748 executions, 650180 successful
    rsg_test.go:828: 4m40s of 5m0s: 1072207 executions, 664655 successful
    rsg_test.go:828: 4m45s of 5m0s: 1093621 executions, 677949 successful
    rsg_test.go:828: 4m50s of 5m0s: 1113403 executions, 690229 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 84905b8396e7cd7ef999b264faaafe13110d631a:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 16192664 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 178820 executions, 110870 successful
    rsg_test.go:828: 50s of 5m0s: 200969 executions, 124543 successful
    rsg_test.go:828: 55s of 5m0s: 220520 executions, 136648 successful
    rsg_test.go:828: 1m0s of 5m0s: 240202 executions, 148939 successful
    rsg_test.go:828: 1m5s of 5m0s: 260078 executions, 161182 successful
    rsg_test.go:828: 1m10s of 5m0s: 279914 executions, 173515 successful
    rsg_test.go:828: 1m15s of 5m0s: 302881 executions, 187733 successful
    rsg_test.go:828: 1m20s of 5m0s: 324052 executions, 200915 successful
    rsg_test.go:828: 1m25s of 5m0s: 343713 executions, 213125 successful
    rsg_test.go:828: 1m30s of 5m0s: 366511 executions, 227268 successful
    rsg_test.go:828: 1m35s of 5m0s: 388969 executions, 241188 successful
    rsg_test.go:828: 1m40s of 5m0s: 411030 executions, 254942 successful
    rsg_test.go:828: 1m45s of 5m0s: 434637 executions, 269640 successful
    rsg_test.go:828: 1m50s of 5m0s: 457520 executions, 283748 successful
    rsg_test.go:828: 1m55s of 5m0s: 475685 executions, 294997 successful
    rsg_test.go:828: 2m0s of 5m0s: 496963 executions, 308153 successful
    rsg_test.go:828: 2m5s of 5m0s: 518708 executions, 321655 successful
    rsg_test.go:828: 2m10s of 5m0s: 537523 executions, 333330 successful
    rsg_test.go:828: 2m15s of 5m0s: 560407 executions, 347533 successful
    rsg_test.go:828: 2m20s of 5m0s: 580258 executions, 359905 successful
    rsg_test.go:828: 2m25s of 5m0s: 603517 executions, 374281 successful
    rsg_test.go:828: 2m30s of 5m0s: 625994 executions, 388199 successful
    rsg_test.go:828: 2m35s of 5m0s: 648394 executions, 402119 successful
    rsg_test.go:828: 2m40s of 5m0s: 668525 executions, 414654 successful
    rsg_test.go:828: 2m45s of 5m0s: 690226 executions, 427984 successful
    rsg_test.go:828: 2m50s of 5m0s: 710891 executions, 440946 successful
    rsg_test.go:828: 2m55s of 5m0s: 730321 executions, 452956 successful
    rsg_test.go:828: 3m0s of 5m0s: 753262 executions, 467214 successful
    rsg_test.go:828: 3m5s of 5m0s: 776895 executions, 481796 successful
    rsg_test.go:828: 3m10s of 5m0s: 798058 executions, 494901 successful
    rsg_test.go:828: 3m15s of 5m0s: 817924 executions, 507187 successful
    rsg_test.go:828: 3m20s of 5m0s: 840939 executions, 521471 successful
    rsg_test.go:828: 3m25s of 5m0s: 861347 executions, 534177 successful
    rsg_test.go:828: 3m30s of 5m0s: 882868 executions, 547542 successful
    rsg_test.go:828: 3m35s of 5m0s: 905379 executions, 561475 successful
    rsg_test.go:828: 3m40s of 5m0s: 923988 executions, 573045 successful
    rsg_test.go:828: 3m45s of 5m0s: 945937 executions, 586513 successful
    rsg_test.go:828: 3m50s of 5m0s: 970288 executions, 601636 successful
    rsg_test.go:828: 3m55s of 5m0s: 990341 executions, 613946 successful
    rsg_test.go:828: 4m0s of 5m0s: 1013960 executions, 628600 successful
    rsg_test.go:828: 4m5s of 5m0s: 1034149 executions, 641084 successful
    rsg_test.go:828: 4m10s of 5m0s: 1056246 executions, 654831 successful
    rsg_test.go:828: 4m15s of 5m0s: 1075906 executions, 666993 successful
    rsg_test.go:828: 4m20s of 5m0s: 1097006 executions, 680047 successful
    rsg_test.go:828: 4m25s of 5m0s: 1121590 executions, 695316 successful
    rsg_test.go:828: 4m30s of 5m0s: 1145960 executions, 710348 successful
    rsg_test.go:828: 4m35s of 5m0s: 1163556 executions, 721266 successful
    rsg_test.go:828: 4m40s of 5m0s: 1186436 executions, 735433 successful
    rsg_test.go:828: 4m45s of 5m0s: 1207834 executions, 748730 successful
    rsg_test.go:828: 4m50s of 5m0s: 1228820 executions, 761673 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 6cce6c746150307a9ecf3b529dfd633a6985c110:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 15806050 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

    rsg_test.go:828: 45s of 5m0s: 195576 executions, 121340 successful
    rsg_test.go:828: 50s of 5m0s: 216655 executions, 134543 successful
    rsg_test.go:828: 55s of 5m0s: 237971 executions, 147752 successful
    rsg_test.go:828: 1m0s of 5m0s: 259295 executions, 160980 successful
    rsg_test.go:828: 1m5s of 5m0s: 279042 executions, 173251 successful
    rsg_test.go:828: 1m10s of 5m0s: 300607 executions, 186507 successful
    rsg_test.go:828: 1m15s of 5m0s: 319927 executions, 198479 successful
    rsg_test.go:828: 1m20s of 5m0s: 339156 executions, 210360 successful
    rsg_test.go:828: 1m25s of 5m0s: 357855 executions, 222032 successful
    rsg_test.go:828: 1m30s of 5m0s: 377067 executions, 233954 successful
    rsg_test.go:828: 1m35s of 5m0s: 399436 executions, 247800 successful
    rsg_test.go:828: 1m40s of 5m0s: 421036 executions, 261230 successful
    rsg_test.go:828: 1m45s of 5m0s: 441471 executions, 273858 successful
    rsg_test.go:828: 1m50s of 5m0s: 465769 executions, 288877 successful
    rsg_test.go:828: 1m55s of 5m0s: 486375 executions, 301757 successful
    rsg_test.go:828: 2m0s of 5m0s: 508527 executions, 315451 successful
    rsg_test.go:828: 2m5s of 5m0s: 529936 executions, 328688 successful
    rsg_test.go:828: 2m10s of 5m0s: 550635 executions, 341505 successful
    rsg_test.go:828: 2m15s of 5m0s: 569712 executions, 353332 successful
    rsg_test.go:828: 2m20s of 5m0s: 588132 executions, 364713 successful
    rsg_test.go:828: 2m25s of 5m0s: 607344 executions, 376620 successful
    rsg_test.go:828: 2m30s of 5m0s: 626783 executions, 388639 successful
    rsg_test.go:828: 2m35s of 5m0s: 650379 executions, 403201 successful
    rsg_test.go:828: 2m40s of 5m0s: 670519 executions, 415776 successful
    rsg_test.go:828: 2m45s of 5m0s: 692514 executions, 429323 successful
    rsg_test.go:828: 2m50s of 5m0s: 715368 executions, 443561 successful
    rsg_test.go:828: 2m55s of 5m0s: 737631 executions, 457491 successful
    rsg_test.go:828: 3m0s of 5m0s: 757497 executions, 469697 successful
    rsg_test.go:828: 3m5s of 5m0s: 779255 executions, 483268 successful
    rsg_test.go:828: 3m10s of 5m0s: 797199 executions, 494403 successful
    rsg_test.go:828: 3m15s of 5m0s: 818975 executions, 507850 successful
    rsg_test.go:828: 3m20s of 5m0s: 839446 executions, 520692 successful
    rsg_test.go:828: 3m25s of 5m0s: 861111 executions, 534048 successful
    rsg_test.go:828: 3m30s of 5m0s: 882344 executions, 547193 successful
    rsg_test.go:828: 3m35s of 5m0s: 901963 executions, 559432 successful
    rsg_test.go:828: 3m40s of 5m0s: 924007 executions, 573133 successful
    rsg_test.go:828: 3m45s of 5m0s: 945155 executions, 586307 successful
    rsg_test.go:828: 3m50s of 5m0s: 966875 executions, 599564 successful
    rsg_test.go:828: 3m55s of 5m0s: 988918 executions, 613331 successful
    rsg_test.go:828: 4m0s of 5m0s: 1011112 executions, 627040 successful
    rsg_test.go:828: 4m5s of 5m0s: 1027470 executions, 637217 successful
    rsg_test.go:828: 4m10s of 5m0s: 1048471 executions, 650254 successful
    rsg_test.go:828: 4m15s of 5m0s: 1068881 executions, 662781 successful
    rsg_test.go:828: 4m20s of 5m0s: 1089250 executions, 675400 successful
    rsg_test.go:828: 4m25s of 5m0s: 1113701 executions, 690532 successful
    rsg_test.go:828: 4m30s of 5m0s: 1137073 executions, 705016 successful
    rsg_test.go:828: 4m35s of 5m0s: 1158440 executions, 718204 successful
    rsg_test.go:828: 4m40s of 5m0s: 1179069 executions, 731008 successful
    rsg_test.go:828: 4m45s of 5m0s: 1201234 executions, 744661 successful
    rsg_test.go:828: 4m50s of 5m0s: 1222480 executions, 757900 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 2f86475f2e3d713b4c8c976c34dba99acce7c4c7:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 13640417 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

=== RUN   TestRandomSyntaxFunctions
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/d437d2c847dfedbc4972f231c3331c8e/logTestRandomSyntaxFunctions2596570674
    test_log_scope.go:79: use -show-logs to present logs inline
    rsg_test.go:828: 5s of 5m0s: 19106 executions, 11802 successful
    rsg_test.go:828: 10s of 5m0s: 40787 executions, 25350 successful
    rsg_test.go:828: 15s of 5m0s: 64115 executions, 39785 successful
    rsg_test.go:828: 20s of 5m0s: 85855 executions, 53291 successful
    rsg_test.go:828: 25s of 5m0s: 107120 executions, 66507 successful
    rsg_test.go:828: 30s of 5m0s: 128383 executions, 79633 successful
    rsg_test.go:828: 35s of 5m0s: 151056 executions, 93678 successful
    rsg_test.go:828: 40s of 5m0s: 170763 executions, 105881 successful
    rsg_test.go:828: 45s of 5m0s: 188294 executions, 116793 successful
    rsg_test.go:828: 50s of 5m0s: 209229 executions, 129708 successful
    rsg_test.go:828: 55s of 5m0s: 232320 executions, 143980 successful
    rsg_test.go:828: 1m0s of 5m0s: 252605 executions, 156502 successful
    rsg_test.go:828: 1m5s of 5m0s: 274513 executions, 170143 successful
    rsg_test.go:828: 1m10s of 5m0s: 295499 executions, 183174 successful
    rsg_test.go:828: 1m15s of 5m0s: 312179 executions, 193478 successful
    rsg_test.go:828: 1m20s of 5m0s: 334765 executions, 207560 successful
    rsg_test.go:828: 1m25s of 5m0s: 357311 executions, 221570 successful
    rsg_test.go:828: 1m30s of 5m0s: 378662 executions, 234816 successful
    rsg_test.go:828: 1m35s of 5m0s: 399083 executions, 247378 successful
    rsg_test.go:828: 1m40s of 5m0s: 418530 executions, 259413 successful
    rsg_test.go:828: 1m45s of 5m0s: 440121 executions, 272879 successful
    rsg_test.go:828: 1m50s of 5m0s: 461183 executions, 285861 successful
    rsg_test.go:828: 1m55s of 5m0s: 485037 executions, 300611 successful
    rsg_test.go:828: 2m0s of 5m0s: 505711 executions, 313516 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ ad6ce866ea3b5c5bb47ba9a0ac19b721a0c98add:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 15018462 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

=== RUN   TestRandomSyntaxFunctions
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/d437d2c847dfedbc4972f231c3331c8e/logTestRandomSyntaxFunctions3386607560
    test_log_scope.go:79: use -show-logs to present logs inline
    rsg_test.go:828: 5s of 5m0s: 20997 executions, 13067 successful
    rsg_test.go:828: 10s of 5m0s: 44421 executions, 27637 successful
    rsg_test.go:828: 15s of 5m0s: 63165 executions, 39192 successful
    rsg_test.go:828: 20s of 5m0s: 85243 executions, 52897 successful
    rsg_test.go:828: 25s of 5m0s: 103698 executions, 64502 successful
    rsg_test.go:828: 30s of 5m0s: 125569 executions, 78007 successful
    rsg_test.go:828: 35s of 5m0s: 148329 executions, 92071 successful
    rsg_test.go:828: 40s of 5m0s: 175513 executions, 109021 successful
    rsg_test.go:828: 45s of 5m0s: 198341 executions, 123192 successful
    rsg_test.go:828: 50s of 5m0s: 222198 executions, 138070 successful
    rsg_test.go:828: 55s of 5m0s: 242438 executions, 150537 successful
    rsg_test.go:828: 1m0s of 5m0s: 263052 executions, 163295 successful
    rsg_test.go:828: 1m5s of 5m0s: 284140 executions, 176464 successful
    rsg_test.go:828: 1m10s of 5m0s: 306458 executions, 190312 successful
    rsg_test.go:828: 1m15s of 5m0s: 328014 executions, 203663 successful
    rsg_test.go:828: 1m20s of 5m0s: 353341 executions, 219543 successful
    rsg_test.go:828: 1m25s of 5m0s: 373286 executions, 232017 successful
    rsg_test.go:828: 1m30s of 5m0s: 394928 executions, 245387 successful
    rsg_test.go:828: 1m35s of 5m0s: 416891 executions, 259063 successful
    rsg_test.go:828: 1m40s of 5m0s: 439503 executions, 273122 successful
    rsg_test.go:828: 1m45s of 5m0s: 459936 executions, 285786 successful
    rsg_test.go:828: 1m50s of 5m0s: 481432 executions, 299234 successful
    rsg_test.go:828: 1m55s of 5m0s: 505442 executions, 314165 successful
    rsg_test.go:828: 2m0s of 5m0s: 525665 executions, 326702 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@rafiss
Copy link
Collaborator

rafiss commented Jul 18, 2023

@fqazi will check if the timeout logic in this test can be improved. We might add a statement_timeout, or else make the statement runner more intelligent.

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 6ccfa2ca599092e742151ea7017695e99567d6f1:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 5688392 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

        	GOROOT/src/crypto/tls/key_schedule.go:63 +0x486
        crypto/tls.(*cipherSuiteTLS13).deriveSecret(0xa56c1e0, {0xc02b6ca820, 0x20, 0x20}, {0x5e7c2b8, 0xc}, {0x7499460?, 0xc012346200?})
        	GOROOT/src/crypto/tls/key_schedule.go:75 +0xd3
        crypto/tls.(*clientHandshakeStateTLS13).establishHandshakeKeys(0xc00ac4d550)
        	GOROOT/src/crypto/tls/handshake_client_tls13.go:382 +0x1fa
        crypto/tls.(*clientHandshakeStateTLS13).handshake(0xc00ac4d550)
        	GOROOT/src/crypto/tls/handshake_client_tls13.go:90 +0x21f
        crypto/tls.(*Conn).clientHandshake(0xc01e982a80, {0x7477218?, 0xc013b3f2c0})
        	GOROOT/src/crypto/tls/handshake_client.go:228 +0x5bb
        crypto/tls.(*Conn).handshakeContext(0xc01e982a80, {0x7477250, 0xc0001b4000})
        	GOROOT/src/crypto/tls/conn.go:1490 +0x32f
        crypto/tls.(*Conn).HandshakeContext(...)
        	GOROOT/src/crypto/tls/conn.go:1433
        crypto/tls.(*Conn).Handshake(...)
        	GOROOT/src/crypto/tls/conn.go:1417
        crypto/tls.(*Conn).Write(0x55fc940?, {0xc021691521, 0x63, 0x1ff})
        	GOROOT/src/crypto/tls/conn.go:1146 +0xf2
        github.com/lib/pq.(*conn).sendStartupPacket(0xc00ac4da10?, 0xc00ac4d988?)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:965 +0x85
        github.com/lib/pq.(*conn).startup(0xc021691500, 0xc0001b4000?)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:1170 +0x69e
        github.com/lib/pq.(*Connector).open(0xc0258ab518, {0x7477250, 0xc0001b4000})
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:378 +0x4c5
        github.com/lib/pq.DialOpen({0x74515e0?, 0xc015a4dd40}, {0xc009034000?, 0x7447680?})
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:328 +0x7b
        github.com/lib/pq.Open(...)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:318
        github.com/lib/pq.Driver.Open({}, {0xc009034000, 0x169})
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:56 +0x85
        database/sql.dsnConnector.Connect(...)
        	GOROOT/src/database/sql/sql.go:759
        database/sql.(*DB).conn(0xc002984c30, {0x7477218, 0xc00623f740}, 0x1)
        	GOROOT/src/database/sql/sql.go:1393 +0x763
        database/sql.(*DB).exec(0xc009056228?, {0x7477218, 0xc00623f740}, {0xc026dcb9e0, 0x53}, {0x0, 0x0, 0x0}, 0xc0?)
        	GOROOT/src/database/sql/sql.go:1655 +0x5d
        database/sql.(*DB).ExecContext(0xc005194b00?, {0x7477218, 0xc00623f740}, {0xc026dcb9e0, 0x53}, {0x0, 0x0, 0x0})
        	GOROOT/src/database/sql/sql.go:1633 +0xe5
        github.com/cockroachdb/cockroach/pkg/sql/tests_test.(*verifyFormatDB).execWithResettableTimeout.func2()
        	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:173 +0x48
        created by github.com/cockroachdb/cockroach/pkg/sql/tests_test.(*verifyFormatDB).execWithResettableTimeout
        	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:172 +0x19d
        
    rsg_test.go:252: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:854: Crash detected: server panic: statement exec timeout
        SELECT crdb_internal.reset_activity_tables() ;
        
        Stack trace:
        timeout: "SELECT crdb_internal.reset_activity_tables() ". currently executing: map[SELECT crdb_internal.gen_rand_ident('kw':::STRING, 3163918212908756977:::INT8) :1 SELECT crdb_internal.gen_rand_ident(e'V\bx\x1e[\x07\x1f4j':::STRING, 6799294187757613353:::INT8) :1 SELECT crdb_internal.reset_activity_tables() :1 SELECT st_mpointfromwkb('\x42d5e08dec4dc0':::BYTES, (-8247629302832732689):::INT8) :1 SELECT st_mpolyfromtext('':::STRING, (-3623053846072590258):::INT8) :1 SELECT st_mpolyfromwkb('\x':::BYTES) :1 SELECT st_mpolyfromwkb('\xee035896aae1ea7e':::BYTES, 393795857011715116:::INT8) :1 SELECT st_multi('010700008007000000010500008003000000010200008003000000280BE8221864EDC13897967CFA9DF6C184978167DBDDF4414ACD58730A6AF841964BD2FA7F2CF64100F2533F1760C1C1409BFD27291FAE41F02B0702ED4AC04160E93AA9F95BB74101020000800A00000000641C7E154E93C1C0BF8AB2AA56CAC1649DA2217AB4FF41D0EFCA3354ABDB419A1524081BF6E6C11072E64DE281FF41587E248EE195DE41247BFC9BAA52EFC1E4193E1B088AF0413ECCDD9EC443024226B43D9710C5E3C170F40FEA0EC0F0C10060D1CB539BF0417870A621C5DAE041D8B276F8C95301425C673D700332EF41B40BA5392220E341849B6508596A0242649AD9CD2A0BF641D8B8BEE11A6502421A50FA348553F441BC3BD111001AEC41203A8AC9985DFB414A61C20499DDFCC114225B77C894E04126B9572FDF620142F84048D94062EA414C6E3E7B6E18F5C13024693439B6E541E4CE05CD3207E141010200008005000000EC68EAD6E4FFFD41B8605F143BEBE3C17EADE1DD1AEFECC18655CB5183E2F741FCEAF3C3A97DEC41508987A828C4FB4178BE89D292BAD8C1E655891B8BAE0042761D16694294F541472DF09BB6FDF3C1A043F330FD4AE541084A6ADC981ED34176554565FD1BEDC140E34EE0E3B2DA41C0E642913D3CCEC10105000080060000000102000080030000001029D8057B1DFA412893D77AE65BE3C1C0149B4EF686C741D246158DC0A3F441A0575882243CE4C15E474B6325F2EFC19C2685503108E6C1F6C1457C467DE9C1169D212074A2F54101020000800200000004E8B993F321FD4186412B14778DE1C1FB8A5277EFD0FDC160C3008BD625D0C1A4DACBB84C70F541DC31AA79DA73FF4101020000800200000044704B20DC47E741AE1718421D1100C2727FBCEA6BD2F9C1048F57A39872F2417ADC11F35DA700C2C055E48D5826B241010200008002000000EC137F1EFEDDE541D4D66EE9FD70FE41F0FADC57A47CF841D4833B7170CFE041363B078EA8F2014208FCC6C2FFBAFC4101020000800800000088C3991FA899E5C17AA58798F38EF8C1682040368441D8418F0F792FD54002C217C021A23ECFFFC104734C3EE53400426481727AA069E9C11C841E86E62701C25CDDE4243490F0414CA87528B9E4E341D2688152181F01C20E85A32EB6A8F6C1CCFE342443AB0042D0D21537311102C23C3D7B22D8ECF041801177876207E2417A308CBDFB1DF8C13ED398A286F7F9C1D8E62545C929FE41B86A1A673418D2412095D3345578F4C1A9C8D92016EDFDC182B99F17877CE9C1B81326506A0EFC41010200008002000000A77CB5D528C9F3C18AF6245F1911F641B07E5B77A71DE8419F873938BE0DF5C14CA399EFB181F0414C361097FB96FEC10102000080070000009C1BAB779A31F6C19C3012B45975FFC1954A12ECDE0401C29CA44F615284E341C1BAEF7B30C8FEC126EDE9A9ED2EECC1B0D18B85FB5EFD419B6E5306347DF0C130E3DEC7BA9DE44108533B221F44E64144473EC91151E2C1D02DB5778ED4CDC1445A4F6FB2C7E3410A2A49503A87F0418FA5E26B862DF9C180D8E03003EFD0C11C1CFEE2FB07E141574442DFC799FCC12C1567370876F6C19CF58ADB4765F74196C86E844664F6410106000080040000000103000080010000000400000014BD7F6E2319FCC1BC596B2C48A1EAC1F87986C6A33DFDC10E0D2030ED02E6C1A48D01164F21E641B0038FCE7A52ECC1B4BF0629559BF5C1FC043C45524CE6417003F2383E7DFC4114BD7F6E2319FCC1BC596B2C48A1EAC1F87986C6A33DFDC101030000800100000006000000B01CD427CCFBD041B4E4249E8D79E141782F03A737D3F2C14024D776B7DAD941005759563E41B041AC7A918A65BAF14180D3D8C5C746FC41B4E65E193CDFD9C1A6D4DA31C8B4F14134716A29E16CF44130E6D8E4A01EFD4190C5BA407196F041B544D1ABF8EBFCC1C86049E6E5B9F1419717CAD6F35AF0C1B01CD427CCFBD041B4E4249E8D79E141782F03A737D3F2C101030000800100000008000000F4B7F9647C9BF5C1087462C64C32F8C104DFF2B07283FD414C49969624DDF4C1C2BCB278524901C2D27ACDA9E205F541E0CD0DD7A8C6E34190A108F6C7B6C9C17DC45F6C582CFAC1C2917F6652C0F94124806139FDF2E8411097013BEB5FF7C1004D17376706AAC17C3635EB049BF641E0F45C2D2825C4C18CA60ACB4619E4C11872DF81ADB4E94146EA1F9E894BF4C1F9490070D282F6C10827EDF9979CD941960A218FD75DF341F4B7F9647C9BF5C1087462C64C32F8C104DFF2B07283FD410103000080010000000B00000002FCB029BA4DE5C1FE7AC8993054F7C10EA45EA1164B0142C07071F859ADDC41B0381DCEF3DDFFC1B03F313B7E5CC74110C3ADB20C77EB418CDC7956969AFBC1702859BDDF8AE441622EAB5557F5F5413766AE8002D4F5C1EB39D6DECFD8FFC1D0C6904BEDB9FD41F8C2E23D915BD4C1AC780DE57EBDE441BCCC1C8E23F0FE4100FF0F5FF582B241A0427E642E63D141767829D63E1FFA4160EF3ADEECCEE541707661291030FCC1EC799AC22F80EA419A1B7D52F50D024280A668C36875D741685282DE839DD341F23C41BD4A99F341D2E261C5F09EF1C14A43BAAD421FEFC118A729924D21FE41DC61EE9C01EEFD4102FCB029BA4DE5C1FE7AC8993054F7C10EA45EA1164B01420106000080050000000103000080010000000D00000000D0B88AC9528DC1E830015145ABE4C13C574BD64438FCC12AD8979FD51A02C279E1939292DA01C2725C0B7FBFE8FEC1F0B05C24C65DD7411C093792A37CE4C198EC3A23E11CF6414C31F9582BABEA41B8047B2C7E5CD7C1CA0675489D12EAC1108E7E52A9A8FA418E3C5F7C0471EAC18AE964AFC5D0EAC10496F51894D1F641B8BD0381A758E1C1E054A66401940242C2D3479AE353F141008A602F73E0D4C1381B62ED9274FCC1103555E334140142DC5AEE8AD170ED411E62FA09094B00C226CBD9EF6C6B0242849069D773A1F1417466D254641AE8414CE48854BBC1F14138FF150A48E0FA4138FA0720DB99DA41E0F6C228F7B1E24100182AE19E0EA9C1364BA3AAE33AF54196BA71CF8D1CF0C12061238A8B17C1C1F29E0FA75B0BF24100D0B88AC9528DC1E830015145ABE4C13C574BD64438FCC10103000080010000000600000028636F02E30DF6C11105F8380C03F5C1D24695D784340142CBF828C729C3F6C17D3A7106CB3A02C28C59DEC1CA6CEAC10CFC86416B750242B7FCCBBA7AA0FCC1083D04DDFA2CF0C1A80C0BB52D6501425049A6C52793FE41E0AA58C779A5C6C16CF11BAA78C5EF419C909D4B50E30042004BD21E3B60AF4128636F02E30DF6C11105F8380C03F5C1D24695D7843401420103000080010000000B00000050028B75DD84C8C19D1A217E580DF2C1AD5AC4EBC1B4F2C1AC6DE9C6C491EC41DA72D35D69B001C2A026478588B3C441B08D31FB3DC0D641C7BC14774F0DF2C1FC4752A9B1A3E9C1EC6259DD74F3F3412ACF14CAED9DFCC1A0DD67BD476201C214996EB815E1F74116FE0ED46B0AF741A0CA3DB84943C041F64C60C55E88FDC1CAB97444D8530142942E7A0E09A0F2C1AA4B2AF4B4B7FAC1A06BCB0ABE0FFB4114859F1B59B900429E04BD5EC69DEEC1840ACF392CD2E741C28262378E88E8C18C602D021F1D01C25C9BEFD5F1EEF24126ED5186967BECC167CA9FEE14C9F7C1D0A76FEDE9F8D44126665E12C6E700C250028B75DD84C8C19D1A217E580DF2C1AD5AC4EBC1B4F2C10103000080010000000A000000CE57CA27BCB5FBC18666201FD073E6C10AEED32C0866FCC1639D399FCAC7F0C1BFFA922FE7A301C21D9598355D7CF8C18867A2C94C97E0410C206A866583FFC180349218A7C0AEC100C0E4A8CB856A41501D27C08BC9EF41B455BD694E5F00C27401B822F7F0D6C1D83339F9CA7AF54178259AE2359DFDC1E4024B428585E4C1F6F1BA1D400FF1417782769529A4F6C1A47969358ACCEBC1788E2F29EC43DD4178A9E3DA2F54D841C32B6EA36077F2C10061A9814BCBF2418AF8A4FDFCAAF24141B134CE0FDF00C290EB8533235CF441D07663A38D1AD5C1CE57CA27BCB5FBC18666201FD073E6C10AEED32C0866FCC10103000080010000000400000098249D485414E4415C05B0151E9CF8C1285A1CFBCD86D041B05A2AB0BCA7EF41E4C528447779D6C144798DA980B5FC413825FD820B6CDD41CC1C68E87292F241D8C0AD2364C2D34198249D485414E4415C05B0151E9CF8C1285A1CFBCD86D04101060000800100000001030000800100000009000000C6C03ADC1DFBE6C1DEC269AAFF78FAC1A053C78030D4B141881E33408855E3C1CEE65863208AE7C10039682B47BAC2C10694B6EE5C44F94137E3A81B90FA01C2AE898C981A520142ECA4F5A4AA29F7419F0DECDFCA57F0C13E0F6654803FEFC1FAA39B45CA74F4C1B86BFFB6C9CBE841500A843E4426E541D20D57E1B90502C2A8A8F39C6ED8EE411E6B1B57DB11F5C120B6A9F7D75BF4C1B00585A0271FD9C1FC77B559EB6FF541B2D381A8D2FAFEC1901653D99039DAC1401602143349B641C6C03ADC1DFBE6C1DEC269AAFF78FAC1A053C78030D4B141010200008005000000202DC8795414F2C1EAE30DF59205E9C1B62B3714D1B9F641C8F8D462CFC5EA41DAEB07FBD58901C2C46956FD6F96FE416EDA71FF1EDCF141F21B44031E1B00C2F2622B77410DEBC1462A3119802DF9413C517FA5F11CF7C15EBACEC02F89F041B0DBF80A5450FF41EE0C28F5A75CF7C1E0843E1A200AE541':::GEOMETRY) :1]
    rsg_test.go:827: 4m45s of 5m0s: 562275 executions, 349799 successful
    rsg_test.go:827: 4m50s of 5m0s: 572877 executions, 356384 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@fqazi
Copy link
Collaborator

fqazi commented Jul 19, 2023

Resettable timeout isn't at play with these tests, and the latest failure is a low truncate. Let me see if I can spot anything here

@fqazi
Copy link
Collaborator

fqazi commented Jul 19, 2023

Ah adopting a resettable timeout for these tests would generally improve their reliability. I'll get a PR up shortly, the execution times for individual statements are reasonable and we can get away from special casing statements out.

@rafiss rafiss added T-disaster-recovery and removed T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Jul 20, 2023
@blathers-crl
Copy link

blathers-crl bot commented Jul 20, 2023

cc @cockroachdb/disaster-recovery

@rafiss rafiss added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed A-disaster-recovery T-disaster-recovery labels Jul 20, 2023
fqazi added a commit to fqazi/cockroach that referenced this issue Jul 20, 2023
Previously, TestRandomSyntaxFunctions could flake since
some operations could take a long time. A resettable timeout
makes things more dynamic, and allows timeouts to be extended
while other connections make progress for a limited time, which
can help with certain builtins that are schema changes (i.e. they
will wait on other connections).

Fixes: cockroachdb#99182

Release note: None
fqazi added a commit to fqazi/cockroach that referenced this issue Jul 20, 2023
Previously this builtin would crash if a bad job ID was passed in.
This patch, adds check to confirm that a job can be found preventing
the crash.

Fixes: cockroachdb#99182

Release note: None
fqazi added a commit to fqazi/cockroach that referenced this issue Jul 20, 2023
Previously, crdb_internal.tenant_span_stats would crash
if the number of arguments were incorrect. This patch,
adds a check for the parameter count.

Fixes: cockroachdb#99182

Release note: None
@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ d62a2f3ab1dff86bd80cac037b215121537a2a9a:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 5885154 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

        github.com/cockroachdb/cockroach/pkg/server.startServeSQL.func1.1({0x7478e60?, 0xc004027aa0?}, {0x74c4210, 0xc03074ee00})
        	github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1766 +0x24e
        github.com/cockroachdb/cockroach/pkg/util/netutil.(*TCPServer).ServeWith.func1({0x7478e60, 0xc004027aa0})
        	github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:188 +0xc7
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2()
        	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470 +0x146
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx
        	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:461 +0x43b
        
    rsg_test.go:252: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:854: Crash detected: server panic: statement exec timeout
        SELECT crdb_internal.reset_activity_tables() ;
        
        Stack trace:
        timeout: "SELECT crdb_internal.reset_activity_tables() ". currently executing: map[SELECT crdb_internal.gen_rand_ident(e'\'':::STRING, 4603276908638047356:::INT8) :1 SELECT crdb_internal.gen_rand_ident(e'\x15h[':::STRING, 175505127776447869:::INT8, 'null':::JSONB) :1 SELECT crdb_internal.gen_rand_ident(e'o\x1ax?':::STRING, 7799035776469511505:::INT8) :1 SELECT crdb_internal.reset_activity_tables() :2 SELECT crdb_internal.show_create_all_tables('':::STRING) :1 SELECT has_function_privilege('NR|':::STRING, 2682103681:::OID, e'z\n\x01<Is\x1c\x0f':::STRING) :1 SELECT has_function_privilege(e'^FS?S;\x11\b':::STRING, e'j#B\x02':::STRING, e'2 \\p':::STRING) :1]
    rsg_test.go:827: 2m0s of 5m0s: 238999 executions, 148681 successful
    rsg_test.go:827: 2m5s of 5m0s: 246891 executions, 153620 successful
    rsg_test.go:827: 2m10s of 5m0s: 260550 executions, 162099 successful
    rsg_test.go:827: 2m15s of 5m0s: 275035 executions, 171091 successful
    rsg_test.go:827: 2m20s of 5m0s: 285224 executions, 177483 successful
    rsg_test.go:827: 2m25s of 5m0s: 293753 executions, 182851 successful
    rsg_test.go:827: 2m30s of 5m0s: 301454 executions, 187659 successful
    rsg_test.go:827: 2m35s of 5m0s: 313369 executions, 195027 successful
    rsg_test.go:827: 2m40s of 5m0s: 324630 executions, 202118 successful
    rsg_test.go:827: 2m45s of 5m0s: 336167 executions, 209257 successful
    rsg_test.go:827: 2m50s of 5m0s: 348253 executions, 216753 successful
    rsg_test.go:827: 2m55s of 5m0s: 360363 executions, 224283 successful
    rsg_test.go:827: 3m0s of 5m0s: 370977 executions, 230894 successful
    rsg_test.go:827: 3m5s of 5m0s: 379854 executions, 236539 successful
    rsg_test.go:827: 3m10s of 5m0s: 391337 executions, 243682 successful
    rsg_test.go:827: 3m15s of 5m0s: 403493 executions, 251147 successful
    rsg_test.go:827: 3m20s of 5m0s: 417398 executions, 259832 successful
    rsg_test.go:827: 3m25s of 5m0s: 430006 executions, 267700 successful
    rsg_test.go:827: 3m30s of 5m0s: 442697 executions, 275480 successful
    rsg_test.go:827: 3m35s of 5m0s: 453397 executions, 282104 successful
    rsg_test.go:827: 3m40s of 5m0s: 464630 executions, 289184 successful
    rsg_test.go:827: 3m45s of 5m0s: 472140 executions, 293860 successful
    rsg_test.go:827: 3m50s of 5m0s: 483634 executions, 300937 successful
    rsg_test.go:827: 3m55s of 5m0s: 494744 executions, 307788 successful
    rsg_test.go:827: 4m0s of 5m0s: 509187 executions, 316725 successful
    rsg_test.go:827: 4m5s of 5m0s: 522566 executions, 325031 successful
    rsg_test.go:827: 4m10s of 5m0s: 533463 executions, 331924 successful
    rsg_test.go:827: 4m15s of 5m0s: 544398 executions, 338636 successful
    rsg_test.go:827: 4m20s of 5m0s: 554967 executions, 345164 successful
    rsg_test.go:827: 4m25s of 5m0s: 566628 executions, 352468 successful
    rsg_test.go:827: 4m30s of 5m0s: 576147 executions, 358398 successful
    rsg_test.go:827: 4m35s of 5m0s: 584455 executions, 363542 successful
    rsg_test.go:827: 4m40s of 5m0s: 594042 executions, 369540 successful
    rsg_test.go:827: 4m45s of 5m0s: 601144 executions, 373949 successful
    rsg_test.go:827: 4m50s of 5m0s: 608364 executions, 378525 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

sql/tests.TestRandomSyntaxFunctions failed with artifacts on release-23.1 @ 043024b889af86e0b8f23f09619d8d3c2c9acd13:

Fatal error:

panic: test timed out after 14m55s

Stack:

goroutine 5717808 [running]:
testing.(*M).startAlarm.func1()
	GOROOT/src/testing/testing.go:2036 +0x8e
created by time.goFunc
	GOROOT/src/time/sleep.go:176 +0x32
Log preceding fatal error

        	GOROOT/src/bufio/bufio.go:237 +0x1bb
        io.ReadAtLeast({0x742aa60, 0xc01c28cae0}, {0xc0045e4920, 0x5, 0x200}, 0x5)
        	GOROOT/src/io/io.go:332 +0x9a
        io.ReadFull(...)
        	GOROOT/src/io/io.go:351
        github.com/lib/pq.(*conn).recvMessage(0xc0045e4900, 0xc008209958)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:1004 +0xca
        github.com/lib/pq.(*conn).recv1Buf(0xc0045e4900, 0xc0128b2cc0?)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:1059 +0x2e
        github.com/lib/pq.(*conn).recv1(...)
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:1086
        github.com/lib/pq.(*conn).simpleExec(0xc0045e4900, {0xc0075f84d0?, 0x6e})
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:664 +0x1ff
        github.com/lib/pq.(*conn).Exec(0xc0045e4900, {0xc0075f84d0, 0x6e}, {0xacbb438, 0x0, 0x0})
        	github.com/lib/pq/external/com_github_lib_pq/conn.go:921 +0x229
        github.com/lib/pq.(*conn).ExecContext(0x0?, {0x7479978, 0xc02861e300}, {0xc0075f84d0, 0x6e}, {0xacbb438, 0x0, 0x469ce5?})
        	github.com/lib/pq/external/com_github_lib_pq/conn_go18.go:46 +0x109
        database/sql.ctxDriverExec({0x7479978?, 0xc02861e300?}, {0x7f64567e5618?, 0xc0045e4900?}, {0x0?, 0x0?}, {0xc0075f84d0?, 0x0?}, {0xacbb438, 0x0, ...})
        	GOROOT/src/database/sql/ctxutil.go:31 +0xe5
        database/sql.(*DB).execDC.func2()
        	GOROOT/src/database/sql/sql.go:1679 +0x175
        database/sql.withLock({0x7449de0, 0xc02c6d79e0}, 0xc005cfddd8)
        	GOROOT/src/database/sql/sql.go:3439 +0x8c
        database/sql.(*DB).execDC(0x74ad101?, {0x7479978, 0xc02861e300}, 0xc02c6d79e0, 0x1773cac52ab0cdb3?, {0xc0075f84d0, 0x6e}, {0x0, 0x0, 0x0})
        	GOROOT/src/database/sql/sql.go:1674 +0x266
        database/sql.(*DB).exec(0xc000180000?, {0x7479978, 0xc02861e300}, {0xc0075f84d0, 0x6e}, {0x0, 0x0, 0x0}, 0xf8?)
        	GOROOT/src/database/sql/sql.go:1659 +0xe8
        database/sql.(*DB).ExecContext(0x744b758?, {0x7479978, 0xc02861e300}, {0xc0075f84d0, 0x6e}, {0x0, 0x0, 0x0})
        	GOROOT/src/database/sql/sql.go:1633 +0xe5
        github.com/cockroachdb/cockroach/pkg/sql/tests_test.(*verifyFormatDB).execWithResettableTimeout.func2()
        	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:173 +0x48
        created by github.com/cockroachdb/cockroach/pkg/sql/tests_test.(*verifyFormatDB).execWithResettableTimeout
        	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:172 +0x19d
        
    rsg_test.go:252: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:854: Crash detected: server panic: statement exec timeout
        SELECT crdb_internal.reset_activity_tables() ;
        
        Stack trace:
        timeout: "SELECT crdb_internal.reset_activity_tables() ". currently executing: map[SELECT box2d_send('BOX(0.761450393963791 -0.3319501372923972,1.3596064213509762 0.8062944585759774)':::BOX2D) :1 SELECT bpcharin(ARRAY[(-31039):::INT8,6529:::INT8]) :1 SELECT bpcharout('X':::STRING) :1 SELECT crdb_internal.gen_rand_ident('Q':::STRING, 8108908307947576502:::INT8) :1 SELECT crdb_internal.gen_rand_ident(e'\f':::STRING, 5159733409929104524:::INT8) :1 SELECT crdb_internal.gen_rand_ident(e'\x19':::STRING, 8396600530797976056:::INT8) :1 SELECT crdb_internal.reset_activity_tables() :1 SELECT crdb_internal.show_create_all_tables('':::STRING) :1]
    rsg_test.go:827: 4m5s of 5m0s: 519556 executions, 323020 successful
    rsg_test.go:827: 4m10s of 5m0s: 531711 executions, 330630 successful
    rsg_test.go:827: 4m15s of 5m0s: 543208 executions, 337776 successful
    rsg_test.go:827: 4m20s of 5m0s: 551470 executions, 342844 successful
    rsg_test.go:827: 4m25s of 5m0s: 562411 executions, 349681 successful
    rsg_test.go:827: 4m30s of 5m0s: 574499 executions, 357157 successful
    rsg_test.go:827: 4m35s of 5m0s: 585879 executions, 364232 successful
    rsg_test.go:827: 4m40s of 5m0s: 594721 executions, 369800 successful
    rsg_test.go:827: 4m45s of 5m0s: 605783 executions, 376590 successful
    rsg_test.go:827: 4m50s of 5m0s: 615366 executions, 382530 successful

Help

See also: How To Investigate a Go Test Failure (internal)

Same failure on other branches

This test on roachdash | Improve this report!

fqazi added a commit to fqazi/cockroach that referenced this issue Jul 21, 2023
Previously, crdb_internal.tenant_span_stats would crash
if the number of arguments were incorrect. This patch,
adds a check for the parameter count.

Fixes: cockroachdb#99182

Release note: None
craig bot pushed a commit that referenced this issue Jul 21, 2023
107233: sql: TestRandomSyntaxFunctions fix flakes r=fqazi a=fqazi

Previously, this test could flake due to the following:

1. The timeout is used for specific functions, so moving to a resettable timeout is beneficial, which allows these to be extended as long as connections make progress
2. Panics inside crdb_internal.tenant_span_stats due to nil pointer dereferences.

Fixes: #99182
Fixes: #107237

107358: sql: handle mixed-case for pg_get_serial_sequence r=rafiss a=rafiss

fixes #107234
Release note (bug fix): The pg_get_serial_sequence builtin function can now handle mixed-case names correctly.

Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
@craig craig bot closed this as completed in 657455e Jul 21, 2023
fqazi added a commit to fqazi/cockroach that referenced this issue Jul 21, 2023
Previously, TestRandomSyntaxFunctions could flake since
some operations could take a long time. A resettable timeout
makes things more dynamic, and allows timeouts to be extended
while other connections make progress for a limited time, which
can help with certain builtins that are schema changes (i.e. they
will wait on other connections).

Fixes: cockroachdb#99182

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants