Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/tests: TestRandomSyntaxSQLSmith failed [no locations to init GEOS] #110780

Closed
cockroach-teamcity opened this issue Sep 17, 2023 · 7 comments · Fixed by #110926
Closed

sql/tests: TestRandomSyntaxSQLSmith failed [no locations to init GEOS] #110780

cockroach-teamcity opened this issue Sep 17, 2023 · 7 comments · Fixed by #110926
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. O-rsg Random Syntax Generator release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Sep 17, 2023 •
edited by cockroach-jira-scripts
Loading

sql/tests.TestRandomSyntaxSQLSmith failed with artifacts on release-23.1 @ 12a0fdf76785787a3a7e83198f1adfd7184ea910:

	panic: geos: no locations to init GEOS [recovered]
	panic: geos: no locations to init GEOS

goroutine 9204926 [running]:
testing.tRunner.func1.2({0x58d63c0, 0xc0457c6420})
	GOROOT/src/testing/testing.go:1396 +0x24e
testing.tRunner.func1()
	GOROOT/src/testing/testing.go:1399 +0x39f
panic({0x58d63c0, 0xc0457c6420})
	GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/util/leaktest.AfterTest.func2()
	github.com/cockroachdb/cockroach/pkg/util/leaktest/leaktest.go:133 +0x2f5
panic({0x58d63c0, 0xc0457c6420})
	GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).recover(0x7508640?, {0x74bd6d0, 0xc0000c2048})
	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:229 +0x6a
panic({0x58d63c0, 0xc0457c6420})
	GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Stop(0xc065866880, {0x74bd6d0, 0xc0000c2048})
	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:507 +0x1dd
panic({0x58d63c0, 0xc0457c6420})
	GOROOT/src/runtime/panic.go:884 +0x212
github.com/cockroachdb/cockroach/pkg/sql/randgen.encodeInvertedIndexHistogramUpperBounds(0xa612680?, {0x7528388?, 0xc018d4a9c0?})
	github.com/cockroachdb/cockroach/pkg/sql/randgen/mutator.go:377 +0x445
github.com/cockroachdb/cockroach/pkg/sql/randgen.randHistogram(0x747a240?, 0xa612680)
	github.com/cockroachdb/cockroach/pkg/sql/randgen/mutator.go:310 +0x13c
github.com/cockroachdb/cockroach/pkg/sql/randgen.statisticsMutator.func1(0xc047b09b00)
	github.com/cockroachdb/cockroach/pkg/sql/randgen/mutator.go:227 +0x96
github.com/cockroachdb/cockroach/pkg/sql/randgen.statisticsMutator(0xc04c19c420, {0xc07694ecd0?, 0x5, 0xc04c19c420?})
	github.com/cockroachdb/cockroach/pkg/sql/randgen/mutator.go:260 +0x71f
github.com/cockroachdb/cockroach/pkg/sql/randgen.MultiStatementMutation.Mutate(0xc069d42a30?, 0x5e7bd2e?, {0xc07694ecd0?, 0x539cd60?, 0x1?})
	github.com/cockroachdb/cockroach/pkg/sql/randgen/mutator.go:98 +0x2e
github.com/cockroachdb/cockroach/pkg/sql/randgen.RandCreateTables(0x539cd60?, {0x5e7bd2e, 0x5}, 0x5, 0x60?, {0xc069d42b10, 0x3, 0xf200000000000000?})
	github.com/cockroachdb/cockroach/pkg/sql/randgen/schema.go:93 +0x16a
github.com/cockroachdb/cockroach/pkg/internal/sqlsmith.randTablesN(0x41?, 0xc069d42ba0?, {0x0, 0x0})
	github.com/cockroachdb/cockroach/pkg/internal/sqlsmith/setup.go:104 +0x194
github.com/cockroachdb/cockroach/pkg/internal/sqlsmith.randTables(0xc069d42cb8?)
	github.com/cockroachdb/cockroach/pkg/internal/sqlsmith/setup.go:84 +0x37
github.com/cockroachdb/cockroach/pkg/internal/sqlsmith.wrapCommonSetup.func1(0x5558f40?)
	github.com/cockroachdb/cockroach/pkg/internal/sqlsmith/setup.go:51 +0x22
github.com/cockroachdb/cockroach/pkg/sql/tests_test.TestRandomSyntaxSQLSmith.func1({0x74bd6d0, 0xc0000c2048}, 0xc020259240, 0xc04551a580)
	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:659 +0x157
github.com/cockroachdb/cockroach/pkg/sql/tests_test.testRandomSyntax(0xc02efc8d00, 0x0?, {0x5e9287d, 0x9}, 0xc069d43f28, 0xc03ec666a8)
	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:836 +0x62b
github.com/cockroachdb/cockroach/pkg/sql/tests_test.TestRandomSyntaxSQLSmith(0xc02efc8d00)
	github.com/cockroachdb/cockroach/pkg/sql/tests_test/pkg/sql/tests/rsg_test.go:653 +0x212
testing.tRunner(0xc02efc8d00, 0x625cd00)
	GOROOT/src/testing/testing.go:1446 +0x10b
created by testing.(*T).Run
	GOROOT/src/testing/testing.go:1493 +0x35f
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-foundations

This test on roachdash | Improve this report!

Jira issue: CRDB-31609
@cockroach-teamcity cockroach-teamcity added branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Sep 17, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Sep 17, 2023
@rafiss rafiss changed the title sql/tests: TestRandomSyntaxSQLSmith failed sql/tests: TestRandomSyntaxSQLSmith failed [no locations to init GEOS] Sep 17, 2023
@rafiss rafiss added O-rsg Random Syntax Generator T-sql-queries SQL Queries Team and removed T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) labels Sep 17, 2023
@github-project-automation github-project-automation bot moved this to Triage in SQL Queries Sep 17, 2023
@DrewKimball
Copy link
Collaborator

This failure included be0bb2c, so there must still be something missing for this test.

@rharding6373
Copy link
Collaborator

Reproducible at commit 12a0fdf767 with:

./dev test pkg/sql/tests -f=TestRandomSyntaxSQLSmith -- --test_arg -rsg=1m --test_env=COCKROACH_RANDOM_SEED=3036286256701247767

@rharding6373
Copy link
Collaborator

rharding6373 commented Sep 19, 2023 •
edited
Loading

@cockroach-dev-inf could we have your help on this again to figure out why the random syntax generator tests aren't getting libgeos? Thanks in advance!

@rharding6373
Copy link
Collaborator

@cockroachdb/test-eng could we have your help on this again to figure out why the random syntax generator tests aren't getting libgeos? Thanks in advance!

@srosenberg
Copy link
Member

srosenberg commented Sep 19, 2023 •
edited
Loading

On master, via the above reproduction, the test actually passes, but the same issue remains,

rsg_test.go:690: ERROR: pq: st_isvalid(): geos: no locations to init GEOS

Despite running with the same seed, the panic on 12a0fdf is caused by randgen.encodeInvertedIndexHistogramUpperBounds whereas on master it doesn't seem to be invoked. Any idea why?

In any case, I am sending a PR to use a runfile, this is how other unit tests with libgeos dependency work. The change in be0bb2c builds libgeos in a directory that's not reachable from a test sandbox.

@srosenberg srosenberg mentioned this issue Sep 19, 2023
Merged
craig bot pushed a commit that referenced this issue Sep 19, 2023
@srosenberg
Merge #110850 #110926
a499165 
110850: codeowners: add `test-eng-prs` to `pkg/clusterversion` r=healthy-pod a=healthy-pod

part of: DEVINF-509
Release note: None

110926: sql: add libgeos runfile dependency for RSG tests r=rickystewart,herkolategan a=srosenberg

In [1], we updated CI (nightly) to require a build step for `//c-deps:libgeos`. That didn't fix `rsg_test`'s implicit dependency on libgeos, since the produced artifacts are not reachable from the test sandbox.

Instead, we use a runfile dependency--same way other test packages specify libgeos dependency. This enables dynamic loading via `bazel.Runfile`.

[1] #110129

Epic: none
Fixes: #110780

Release note: None

Co-authored-by: healthy-pod <[email protected]>
Co-authored-by: Stan Rosenberg <[email protected]>
@craig craig bot closed this as completed in 4a65030 Sep 19, 2023
This was referenced Sep 20, 2023
Merged
Merged
srosenberg added a commit to srosenberg/cockroach that referenced this issue Sep 20, 2023
@srosenberg
sql: add libgeos runfile dependency for RSG tests
66fdde6 
In [1], we updated CI (nightly) to require a build
step for `//c-deps:libgeos`. That didn't fix `rsg_test`'s
implicit dependency on libgeos, since the produced artifacts
are not reachable from the test sandbox.

Instead, we use a runfile dependency--same way other
test packages specify libgeos dependency. This enables
dynamic loading via `bazel.Runfile`.

[1] cockroachdb#110129

Epic: none
Fixes: cockroachdb#110780

Release note: None
srosenberg added a commit to srosenberg/cockroach that referenced this issue Sep 20, 2023
@srosenberg
sql: add libgeos runfile dependency for RSG tests
88be618 
In [1], we updated CI (nightly) to require a build
step for `//c-deps:libgeos`. That didn't fix `rsg_test`'s
implicit dependency on libgeos, since the produced artifacts
are not reachable from the test sandbox.

Instead, we use a runfile dependency--same way other
test packages specify libgeos dependency. This enables
dynamic loading via `bazel.Runfile`.

[1] cockroachdb#110129

Epic: none
Fixes: cockroachdb#110780

Release note: None
@rharding6373
Copy link
Collaborator

Did you try the same seed on master? I'm wary of relying on random seeds to repro issues at different commits, since intervening code changes could change the number of calls to the RNG. I found that at the commit the issue was reported it was reproducible.

@rharding6373 rharding6373 moved this from Triage to Done in SQL Queries Sep 21, 2023
@srosenberg
Copy link
Member

Did you try the same seed on master?

Yep.

I'm wary of relying on random seeds to repro issues at different commits, since intervening code changes could change the number of calls to the RNG.

That's most likely the case. The two (RNG) sequences are not in 1-1 correspondence, hence the sequence on master didn't produce randgen.encodeInvertedIndexHistogramUpperBounds.

I found that at the commit the issue was reported it was reproducible.

Yep, the issue was reproducible at the commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. O-rsg Random Syntax Generator release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

sql: add libgeos runfile dependency for RSG tests srosenberg/cockroach
5 participants
@rafiss @srosenberg @rharding6373 @cockroach-teamcity @DrewKimball