Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: crdb_internal.reset_multi_region_zone_configs_for_database causes concurrent txn use detected #80023

Closed
cockroach-teamcity opened this issue Apr 16, 2022 · 3 comments · Fixed by #84653
Closed

sql: crdb_internal.reset_multi_region_zone_configs_for_database causes concurrent txn use detected #80023

cockroach-teamcity opened this issue Apr 16, 2022 · 3 comments · Fixed by #84653
Assignees
@ajwerner
Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Apr 16, 2022 •
edited by cockroach-jira-scripts
Loading

roachtest.sqlsmith/setup=rand-tables/setting=no-mutations failed with artifacts on release-21.2 @ 0f0029653c25772d09adae5be308ce0c45f84f0a:

The test failed on branch=release-21.2, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/sqlsmith/setup=rand-tables/setting=no-mutations/run_1
	sqlsmith.go:223,sqlsmith.go:256,test_runner.go:777: error: pq: internal error: crdb_internal.reset_multi_region_zone_configs_for_database(): concurrent txn use detected. ba: Get [/Table/3/1/2107461607/2/1,/Min), [txn: 34c72339], [can-forward-ts]
		stmt:
		WITH
			with_34293 (col_190415)
				AS (
					SELECT
						*
					FROM
						(
							VALUES
								(NULL),
								('00:16:32.649786+04:54:00':::TIMETZ),
								(('10:19:57.591189-07:27:00':::TIMETZ::TIMETZ - '290 years':::INTERVAL::INTERVAL)::TIMETZ)
						)
							AS tab_87357 (col_190415)
					UNION
						SELECT
							*
						FROM
							(VALUES ('06:21:22.028838+00:04:00':::TIMETZ), (NULL), ('21:03:02.373314+12:30:00':::TIMETZ))
								AS tab_87358 (col_190416)
				),
			with_34294 (col_190417) AS (SELECT * FROM (VALUES (NULL)) AS tab_87359 (col_190417))
		SELECT
			tab_87360.col3_4 AS col_190418,
			tab_87360.col3_2 AS col_190419,
			NULL AS col_190420,
			tab_87360.col3_17 AS col_190421,
			tab_87360.col3_11 AS col_190422
		FROM
			defaultdb.public.table3@table3_col3_8_key AS tab_87360, with_34293 AS cte_ref_9835
		WHERE
			(NOT crdb_internal.reset_multi_region_zone_configs_for_database(4010021785958174695:::INT8::INT8)::BOOL)
		LIMIT
			93:::INT8;
Reproduce

See: roachtest README

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-15677
@cockroach-teamcity cockroach-teamcity added branch-release-21.2 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Apr 16, 2022
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Apr 16, 2022
@rytaft rytaft mentioned this issue Apr 17, 2022
Closed
@rytaft rytaft changed the title roachtest: sqlsmith/setup=rand-tables/setting=no-mutations failed sql: crdb_internal.reset_multi_region_zone_configs_for_database causes concurrent txn use detected Apr 17, 2022
@rytaft rytaft removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Apr 17, 2022
@rytaft
Copy link
Collaborator

rytaft commented Apr 17, 2022

This should not be a release blocker.

@blathers-crl blathers-crl bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Apr 17, 2022
rytaft added a commit to rytaft/cockroach that referenced this issue Apr 18, 2022
@rytaft
sqlsmith: skip crdb_internal.reset_multi_region_zone_configs_for_data…
c1a959a 
…base

Don't use crdb_internal.reset_multi_region_zone_configs_for_database when
generating sqlsmith queries, since it currently causes an internal error.

Informs cockroachdb#80023

Release note: None
@rytaft rytaft mentioned this issue Apr 18, 2022
Merged
craig bot pushed a commit that referenced this issue Apr 18, 2022
@yuzefovich @rickystewart @rytaft
Merge #80106 #80108 #80111
8ab32d0 
80106: roachtest: silence known sqlsmith error r=yuzefovich a=yuzefovich

SQLSmith keeps on hitting a known "Constraint needs a valid start or end
key" error, so let's silence it for now.

Informs: #79886

Release note: None

80108: dev: avoid running `bazel` commands silently in `doctor` r=mari-crl a=rickystewart

Any `bazel build` we run during `doctor` can take unexpectedly long to
run for a variety of reasons. This is especially true if the dev has
turned on `lintonbuild` which can drastically increase the time for a
cold build. Long-running silent Bazel commands sometimes fool people
into thinking `dev doctor` is hanging indefinitely; in order to avoid
this, attach the std streams so the dev can see the ongoing progress.

Ideally in the past `dev doctor` has printed out the `you are ready to
build` message and nothing else if the machine passes all checks. Since
this is no longer going to be the case, I also add a header and footer
outlining where the actual checks begin and end so any errors become
obvious.

Release note: None

80111: sqlsmith: skip crdb_internal.reset_multi_region_zone_configs_for_database r=rytaft a=rytaft

Don't use crdb_internal.reset_multi_region_zone_configs_for_database when
generating sqlsmith queries, since it currently causes an internal error.

Informs #80023

Release note: None

Co-authored-by: Yahor Yuzefovich <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
Co-authored-by: Rebecca Taft <[email protected]>
rytaft added a commit to rytaft/cockroach that referenced this issue Apr 19, 2022
@rytaft
sqlsmith: skip crdb_internal.reset_multi_region_zone_configs_for_data…
d7b9016 
…base

Don't use crdb_internal.reset_multi_region_zone_configs_for_database when
generating sqlsmith queries, since it currently causes an internal error.

Informs cockroachdb#80023

Release note: None
@rytaft rytaft mentioned this issue Apr 19, 2022
Closed
rytaft added a commit to rytaft/cockroach that referenced this issue Apr 19, 2022
@rytaft
sqlsmith: skip crdb_internal.reset_multi_region_zone_configs_for_data…
e3801f0 
…base

Don't use crdb_internal.reset_multi_region_zone_configs_for_database when
generating sqlsmith queries, since it currently causes an internal error.

Informs cockroachdb#80023

Release note: None
@rytaft rytaft mentioned this issue Apr 19, 2022
Merged
@jlinder jlinder added sync-me and removed sync-me labels May 20, 2022
@e-mbrown
Copy link
Contributor

Did some investigation and narrowed it down to this commit
@ajwerner any clue why?

@blathers-crl blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Jul 18, 2022
@ajwerner
Copy link
Contributor

I'm pretty sure the problem here is that we shouldn't be calling this from distsql which leads to parallelism. The above commit just adds invariant checking. The fix is to add this builtin to the distsql blocklist.

@ajwerner ajwerner assigned ajwerner and unassigned e-mbrown Jul 19, 2022
@ajwerner ajwerner mentioned this issue Jul 19, 2022
Merged
craig bot pushed a commit that referenced this issue Jul 19, 2022
@ajwerner
Merge #84653
a9beb6c 
84653: sql/sem/builtins: add multi-region functions to distsql blocklist r=e-mbrown a=ajwerner

These functions in some cases look at the gateway region and in others actually
perform schema changes. These operations must not be executed in parallel.

Fixes #80023

Release note: None

Co-authored-by: Andrew Werner <[email protected]>
@craig craig bot closed this as completed in 6acc16c Jul 19, 2022
@yuzefovich yuzefovich mentioned this issue Aug 24, 2022
Merged
yuzefovich pushed a commit to yuzefovich/cockroach that referenced this issue Aug 24, 2022
@ajwerner @yuzefovich
sql/sem/builtins: add multi-region functions to distsql blocklist
c1b4ef5 
These functions in some cases look at the gateway region and in others actually
perform schema changes. These operations must not be executed in parallel.

Fixes cockroachdb#80023

Release note: None
@exalate-issue-sync exalate-issue-sync bot removed T-sql-schema-deprecated Use T-sql-foundations instead T-sql-queries SQL Queries Team labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql/sem/builtins: add multi-region functions to distsql blocklist ajwerner/cockroach
5 participants
@jlinder @ajwerner @rytaft @cockroach-teamcity @e-mbrown