demo movr: fatal: morestack on g0 #57885

robert-s-lee · 2020-12-14T04:39:04Z

Describe the problem

alter of non-existent index and table seem to have caused a fatal error.

To Reproduce

Could not reproduce, but the below commands were run while testing various syntax for a customer.

cockroach demo movr --geo-partitioned-replicas
create table city (city string primary key);
insert into city values ('seattle'), ('san francisco'), ('los angeles');
create table city2 as select * from city;
select * from city where city in (select city from city);
create index on rides (start_address) partition by list (start_address) (partition us_west values in (select city from city));


ALTER INDEX students_by_list@name_idx PARTITION BY LIST (country) (
    PARTITION north_america VALUES IN (select city from city),
    PARTITION australia VALUES IN (select city from city),
    PARTITION DEFAULT VALUES IN (default)
  );

Expected behavior

N/A

Additional data / screenshots

[email protected]:59938/movr> ALTER INDEX students_by_list@name_idx PARTITION BY LIST (country) (
    PARTITION north_america VALUES IN (select city from city),
    PARTITION australia VALUES IN (select city from city),
    PARTITION DEFAULT VALUES IN (default)
  );
invalid syntax: statement ignored: at or near "select": syntax error
SQLSTATE: 42601
DETAIL: source SQL:
ALTER INDEX students_by_list@name_idx PARTITION BY LIST (country) (
    PARTITION north_america VALUES IN (select city from city),
                                       ^
HINT: try \h ALTER INDEX
[email protected]:59938/movr> fatal: morestack on g0
SIGTRAP: trace trap
PC=0x4061b42 m=2 sigcode=1

goroutine 824633722176 [???, locked to thread]:
runtime.abort()
	/usr/local/go/src/runtime/asm_amd64.s:859 +0x2
runtime.morestack()
	/usr/local/go/src/runtime/asm_amd64.s:416 +0x25

goroutine 1 [syscall]:
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix._C2func_go_libedit_gets(0x13303600, 0x13c10c60, 0x0, 0x0, 0xc0093a7dd8, 0xc0093a7ddc, 0xc000000001, 0x0, 0x0, 0x0)
	_cgo_gotypes.go:187 +0x56
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix.EditLine.GetLine.func1(0xc00f8db100, 0xc0093a7dd8, 0xc0093a7ddc, 0xc00f8db148, 0xc00f8db100, 0xc00c2c56e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix/editline_unix.go:281 +0xf5
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix.EditLine.GetLine(0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix/editline_unix.go:281 +0xaa
github.com/cockroachdb/cockroach/pkg/cli.(*cliState).doReadLine(0xc011115c00, 0x6, 0x5)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:893 +0x617
github.com/cockroachdb/cockroach/pkg/cli.runInteractive(0xc00be98b40, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:1316 +0xf3
github.com/cockroachdb/cockroach/pkg/cli.runClient(0xc000725b80, 0xc00be98b40, 0xc00758fb00, 0xb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:1531 +0x78
github.com/cockroachdb/cockroach/pkg/cli.runDemo(0xc000725b80, 0x82891e0, 0xc000643320, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/demo.go:353 +0x8a7
github.com/cockroachdb/cockroach/pkg/cli.init.5.func1(0xc000725b80, 0xc000275be0, 0x0, 0x1, 0xc000891e70, 0xc000b2dbe8)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/demo.go:142 +0x3d
github.com/cockroachdb/cockroach/pkg/cli.MaybeDecorateGRPCError.func1(0xc000725b80, 0xc000275be0, 0x0, 0x1, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/error.go:187 +0x79
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).execute(0xc000725b80, 0xc000275b80, 0x1, 0x1, 0xc000725b80, 0xc000275b80)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:762 +0x460
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xa86dea0, 0xc00007a120, 0xc00023768a, 0x9)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:852 +0x2ea
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:800
github.com/cockroachdb/cockroach/pkg/cli.Run(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:224
github.com/cockroachdb/cockroach/pkg/cli.Main()
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:67 +0x272
main.main()
	/go/src/github.com/cockroachdb/cockroach/pkg/cmd/cockroach/main.go:26 +0x20

goroutine 65535 [??? (scan)]:

Environment:

CockroachDB version [20.2.0]
Server OS: [OSX 11.0.1]
Client app [cockroach demo movr, JDBC]

Additional context

N/A

Jira issue: CRDB-6295

Epic CRDB-22182

The text was updated successfully, but these errors were encountered:

jordanlewis · 2020-12-14T04:47:01Z

It's a bug in the CLI's line editor, not sure what.

knz · 2020-12-14T07:45:03Z

We'd need more testing but I suspect this happens when either:

resizing the terminal
handling ctrl+z / resume
the tab key

And is fundamentally caused by yet another oddity of macos Big Sur.

At this point (and given the other big sur issues cropping up) I am tempted to mark native macos execution as unsupported, disable the interactive readline handling entirely in that case, and recommend folk use the Linux binary inside docker instead.

robert-s-lee · 2020-12-16T13:48:26Z

this time, the iterm2 session crashed by itself. The session was not closed, the laptop was closed, the session was idle overnight, and the next morning it crashed.

[email protected]:60745/movr> fatal: morestack on g0
SIGTRAP: trace trap
PC=0x4061b42 m=2 sigcode=1

goroutine 824633722176 [???, locked to thread]:
runtime.abort()
	/usr/local/go/src/runtime/asm_amd64.s:859 +0x2
runtime.morestack()
	/usr/local/go/src/runtime/asm_amd64.s:416 +0x25

goroutine 1 [syscall]:
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix._C2func_go_libedit_gets(0x15343600, 0x1120dba0, 0x0, 0x0, 0xc00e6d5948, 0xc00e6d594c, 0xc000000001, 0x0, 0x0, 0x0)
	_cgo_gotypes.go:187 +0x56
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix.EditLine.GetLine.func1(0xc01124fe80, 0xc00e6d5948, 0xc00e6d594c, 0xc01124fec8, 0xc01124fe80, 0xc003dcf6e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix/editline_unix.go:281 +0xf5
github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix.EditLine.GetLine(0x0, 0x0, 0x0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/knz/go-libedit/unix/editline_unix.go:281 +0xaa
github.com/cockroachdb/cockroach/pkg/cli.(*cliState).doReadLine(0xc01136d700, 0x6, 0x5)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:893 +0x617
github.com/cockroachdb/cockroach/pkg/cli.runInteractive(0xc0122d32c0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:1316 +0xf3
github.com/cockroachdb/cockroach/pkg/cli.runClient(0xc000715b80, 0xc0122d32c0, 0xc008047560, 0xb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/sql.go:1531 +0x78
github.com/cockroachdb/cockroach/pkg/cli.runDemo(0xc000715b80, 0x82891e0, 0xc00061f560, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/demo.go:353 +0x8a7
github.com/cockroachdb/cockroach/pkg/cli.init.5.func1(0xc000715b80, 0xc0002bd7f0, 0x0, 0x1, 0xc00091bf40, 0xc000b39be8)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/demo.go:142 +0x3d
github.com/cockroachdb/cockroach/pkg/cli.MaybeDecorateGRPCError.func1(0xc000715b80, 0xc0002bd7f0, 0x0, 0x1, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/error.go:187 +0x79
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).execute(0xc000715b80, 0xc0002bd7e0, 0x1, 0x1, 0xc000715b80, 0xc0002bd7e0)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:762 +0x460
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xa86dea0, 0xc00007a120, 0xc0002e366a, 0x9)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:852 +0x2ea
github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/spf13/cobra/command.go:800
github.com/cockroachdb/cockroach/pkg/cli.Run(...)
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:224
github.com/cockroachdb/cockroach/pkg/cli.Main()
	/go/src/github.com/cockroachdb/cockroach/pkg/cli/cli.go:67 +0x272
main.main()
	/go/src/github.com/cockroachdb/cockroach/pkg/cmd/cockroach/main.go:26 +0x20

goroutine 65535 [??? (scan)]:

knz · 2020-12-16T14:22:15Z

Robert when you laptop resumed from sleep, all processes were sent the signal SIGCONT.

The issue is with signal handling so that will also cause it.

stevendanna · 2022-06-07T17:23:09Z

I've now seen this on a non-macOS build as well. This is cockroach sql running on linux (via SSH via roachprod). It occurred during a terminal resize.

root@localhost:26257/defaultdb> SIGTRAP: trace trap
PC=0x4bf1a2 m=0 sigcode=128

goroutine 0 [idle]:
runtime.abort()
	GOROOT/src/runtime/asm_amd64.s:980 +0x2
runtime.morestack()
	GOROOT/src/runtime/asm_amd64.s:429 +0x25

goroutine 1 [syscall]:
runtime.cgocall(0x429c830, 0xc000b63660)
	GOROOT/src/runtime/cgocall.go:156 +0x5c fp=0xc000b63638 sp=0xc000b63600 pc=0x45717c
github.com/knz/go-libedit/unix._C2func_go_libedit_gets(0x7f35c3e3c600, 0x7f35bfe0d000, 0x0, 0x0, 0xc000762d08, 0xc000762d0c, 0x1)
	github.com/knz/go-libedit/unix/_cgo_gotypes.go:191 +0x57 fp=0xc000b63660 sp=0xc000b63638 pc=0x4048e17
github.com/knz/go-libedit/unix.EditLine.GetLine.func1(0xc00108c448, 0xc00108c400, 0xc000b637f8)
	/home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/6946/execroot/com_github_cockroachdb_cockroach/external/com_github_knz_go_libedit/unix/editline_unix.go:281 +0xdb fp=0xc000b636f8 sp=0xc000b63660 pc=0x404c2fb
github.com/knz/go-libedit/unix.EditLine.GetLine(0x0)
	/home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/6946/execroot/com_github_cockroachdb_cockroach/external/com_github_knz_go_libedit/unix/editline_unix.go:281 +0x8a fp=0xc000b63758 sp=0xc000b636f8 pc=0x404c08a
github.com/cockroachdb/cockroach/pkg/cli/clisqlshell.(*cliState).doReadLine(0xc000e12140, 0x6)
	github.com/cockroachdb/cockroach/pkg/cli/clisqlshell/sql.go:1010 +0x1d8 fp=0xc000b63808 sp=0xc000b63758 pc=0x40519b8
github.com/cockroachdb/cockroach/pkg/cli/clisqlshell.(*cliState).doRunShell(0xc000e12140, 0x60, 0x7f35ec7c0108, 0x60, 0xc000094800)
	github.com/cockroachdb/cockroach/pkg/cli/clisqlshell/sql.go:1952 +0xb6 fp=0xc000b63888 sp=0xc000b63808 pc=0x4056896
github.com/cockroachdb/cockroach/pkg/cli/clisqlshell.(*cliState).RunInteractive(0xc000e8aea0, 0x673fad8, 0xc00014c780, 0x4f73978)
	github.com/cockroachdb/cockroach/pkg/cli/clisqlshell/sql.go:1915 +0x6d fp=0xc000b63900 sp=0xc000b63888 pc=0x405674d
github.com/cockroachdb/cockroach/pkg/cli/clisqlcfg.(*Context).Run(0xc000e8aea0, {0x6674bc8, 0xc000074098}, {0x673fad8, 0xc00014c780})
	github.com/cockroachdb/cockroach/pkg/cli/clisqlcfg/context.go:222 +0x2df fp=0xc000b63988 sp=0xc000b63900 pc=0x405b99f
github.com/cockroachdb/cockroach/pkg/cli.runTerm(0x0, {0xc000986729, 0xc000986738, 0x0})
	github.com/cockroachdb/cockroach/pkg/cli/sql_shell_cmd.go:63 +0x1ca fp=0xc000b63a70 sp=0xc000b63988 pc=0x40cf5ea
github.com/cockroachdb/cockroach/pkg/cli/clierrorplus.MaybeDecorateError.func1(0x94deea0, {0xc000892160, 0x0, 0x2})
	github.com/cockroachdb/cockroach/pkg/cli/clierrorplus/decorate_error.go:67 +0x43 fp=0xc000b63cf0 sp=0xc000b63a70 pc=0x3f74f63
github.com/spf13/cobra.(*Command).execute(0x94deea0, {0xc000892140, 0x2, 0x2})
	github.com/spf13/cobra/external/com_github_spf13_cobra/command.go:856 +0x60e fp=0xc000b63db0 sp=0xc000b63cf0 pc=0x3557a0e
github.com/spf13/cobra.(*Command).ExecuteC(0x94d1520)
	github.com/spf13/cobra/external/com_github_spf13_cobra/command.go:974 +0x3bc fp=0xc000b63e68 sp=0xc000b63db0 pc=0x35580fc
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra/external/com_github_spf13_cobra/command.go:902
github.com/cockroachdb/cockroach/pkg/cli.Run(...)
	github.com/cockroachdb/cockroach/pkg/cli/cli.go:297
github.com/cockroachdb/cockroach/pkg/cli.doMain(0x94deea0, {0xc0007623aa, 0x3})
	github.com/cockroachdb/cockroach/pkg/cli/cli.go:139 +0x1e9 fp=0xc000b63ee8 sp=0xc000b63e68 pc=0x4089ca9
github.com/cockroachdb/cockroach/pkg/cli.Main()
	github.com/cockroachdb/cockroach/pkg/cli/cli.go:66 +0x136 fp=0xc000b63f70 sp=0xc000b63ee8 pc=0x4089956
main.main()
	main/pkg/cmd/cockroach/main.go:26 +0x17 fp=0xc000b63f80 sp=0xc000b63f70 pc=0x4184517
runtime.main()
	GOROOT/src/runtime/proc.go:255 +0x227 fp=0xc000b63fe0 sp=0xc000b63f80 pc=0x48d367
runtime.goexit()
	GOROOT/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc000b63fe8 sp=0xc000b63fe0 pc=0x4bf321

erikgrinaker · 2022-08-01T10:31:32Z

Yeah, I'm also seeing this. Rapidly resizing the terminal triggers it consistently. As with Steven, this happened with cockroach sql running on a Linux gceworker via SSH from a macOS terminal.

86457: cli: replace libedit with bubbline r=ZhouXing19 a=knz First commit from #88574. Benefits from (but is not dependent on) #87606 server-side. Fixes #21826 Fixes #71207 Fixes #71209 Fixes #57885 NB: this will benefit from upstream library releases based off the still-unmerged PRs listed in knz/bubbline#2. Release justification: n/a will not merge before stability ends Release note (cli change): The engine used as line editor in the interactive shell (`cockroach sql`, `demo`) has been updated. It includes numerous bug fixes and new features. The previous engine can still be accessed via the env var COCKROACH_SQL_FORCE_LIBEDIT. This support will be removed in a later version. 92335: kvserver,logstore: introduce log StateLoader r=tbg a=pavelkalinnikov Previously the `StateLoader` accessed both log and state machine state. This commit moves most of the log-specific accesstors to the `logstore` package. Part of #91979 Release note: None Co-authored-by: Raphael 'kena' Poss <[email protected]> Co-authored-by: Pavel Kalinnikov <[email protected]>

robert-s-lee added the C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. label Dec 14, 2020

jordanlewis assigned knz Dec 14, 2020

knz added B-os-macos Issues specific to macOS. A-cli labels Dec 14, 2020

knz added A-cli-client CLI commands that pertain to using SQL features and removed A-cli labels Mar 20, 2021

knz mentioned this issue Mar 25, 2021

demo: seg fault at interactive prompt without specific input #62283

Closed

knz mentioned this issue Apr 15, 2021

cli: remove cgosymbolizer #63719

Closed

knz removed their assignment Oct 6, 2021

knz mentioned this issue Sep 26, 2022

cli: replace libedit with bubbline #86457

Merged

craig bot closed this as completed in #86457 Dec 1, 2022

exalate-issue-sync bot assigned knz Dec 21, 2022

exalate-issue-sync bot added the T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) label Dec 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

demo movr: fatal: morestack on g0 #57885

demo movr: fatal: morestack on g0 #57885

robert-s-lee commented Dec 14, 2020 •

edited by exalate-issue-sync bot

Loading

jordanlewis commented Dec 14, 2020

knz commented Dec 14, 2020 •

edited

Loading

robert-s-lee commented Dec 16, 2020

knz commented Dec 16, 2020

stevendanna commented Jun 7, 2022

erikgrinaker commented Aug 1, 2022 •

edited

Loading

demo movr: fatal: morestack on g0 #57885

demo movr: fatal: morestack on g0 #57885

Comments

robert-s-lee commented Dec 14, 2020 • edited by exalate-issue-sync bot Loading

jordanlewis commented Dec 14, 2020

knz commented Dec 14, 2020 • edited Loading

robert-s-lee commented Dec 16, 2020

knz commented Dec 16, 2020

stevendanna commented Jun 7, 2022

erikgrinaker commented Aug 1, 2022 • edited Loading

robert-s-lee commented Dec 14, 2020 •

edited by exalate-issue-sync bot

Loading

knz commented Dec 14, 2020 •

edited

Loading

erikgrinaker commented Aug 1, 2022 •

edited

Loading