Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentry: catch.go:29: runtime error: invalid memory address or nil pointer dereference (1) assertion failure Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/u... #126304

Closed
cockroach-sentry opened this issue Jun 26, 2024 · 1 comment
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Jun 26, 2024

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/5541346194/?referrer=webhooks_plugin

Panic Message:

catch.go:29: runtime error: invalid memory address or nil pointer dereference
(1) assertion failure
Wraps: (2) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/util/errorutil.ShouldCatch
  | 	github.com/cockroachdb/cockroach/pkg/util/errorutil/catch.go:29
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).Build.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:212
  | runtime.gopanic
  | 	GOROOT/src/runtime/panic.go:884
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildCreateFunction.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/create_function.go:77
  | runtime.gopanic
  | 	GOROOT/src/runtime/panic.go:884
  | runtime.panicmem
  | 	GOROOT/src/runtime/panic.go:260
  | runtime.sigpanic
  | 	GOROOT/src/runtime/signal_unix.go:839
  | github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser.(*plpgsqlParserImpl).Parse
  | 	plpgsql-gen.y:1035
  | github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser.(*Parser).parse
  | 	github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser/parse.go:98
  | github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser.(*Parser).parseWithDepth
  | 	github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser/parse.go:82
  | github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser.Parse
  | 	github.com/cockroachdb/cockroach/pkg/sql/plpgsql/parser/parse.go:132
  | github.com/cockroachdb/cockroach/pkg/sql/sem/plpgsqltree/utils.CountPLpgSQLStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/plpgsqltree/utils/plpg_visitor.go:90
  | github.com/cockroachdb/cockroach/pkg/sql/sem/plpgsqltree/utils.ParseAndCollectTelemetryForPLpgSQLFunc
  | 	github.com/cockroachdb/cockroach/pkg/sql/sem/plpgsqltree/utils/plpg_visitor.go:112
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildCreateFunction
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/create_function.go:111
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:359
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).buildStmtAtRoot
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:272
  | github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder.(*Builder).Build
  | 	github.com/cockroachdb/cockroach/pkg/sql/opt/optbuilder/builder.go:246
  | github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).buildReusableMemo
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:433
  | github.com/cockroachdb/cockroach/pkg/sql.(*planner).prepareUsingOptimizer
  | 	github.com/cockroachdb/cockroach/pkg/sql/plan_opt.go:164
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).populatePrepared
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:333
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare.func2
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:271
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).prepare
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:276
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).addPreparedStmt
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:115
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPrepare
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor_prepare.go:76
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2314
  | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:2105
  | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn
  | 	github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:902
  | github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1
  | 	github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:310
  | runtime.goexit
  | 	GOROOT/src/runtime/asm_amd64.s:1594
Wraps: (3) runtime error: invalid memory address or nil pointer dereference
  | -- cause hidden behind barrier
  | runtime error: invalid memory address or nil pointer dereference
  | (1) runtime error: invalid memory address or nil pointer dereference
  | Error types: (1) runtime.errorString
Error types: (1) *assert.withAssertionFailure (2) *withstack.withStack (3) *barriers.barrierErr
-- report composition:
*barriers.barrierErr: masked error: runtime error: invalid memory address or nil pointer dereference
catch.go:29: *withstack.withStack (top exception)
*assert.withAssertionFailure
Stacktrace (expand for inline code snippets):

GOROOT/src/runtime/asm_amd64.s#L1593-L1595

reservedOwned = false // We're about to pass ownership away.
retErr = sqlServer.ServeConn(
ctx,

}(ctx, h)
return h.ex.run(ctx, s.pool, reserved, cancel)
}

var err error
if err = ex.execCmd(); err != nil {
// Both of these errors are normal ways for the connExecutor to exit.

stmtCtx := withStatement(ctx, ex.curStmtAST)
ev, payload = ex.execPrepare(stmtCtx, tcmd)
case DescribeStmt:

stmt := makeStatement(parseCmd.Statement, ex.server.cfg.GenerateID())
_, err := ex.addPreparedStmt(
ctx,

// Prepare the query. This completes the typing of placeholders.
prepared, err := ex.prepare(ctx, stmt, placeholderHints, rawTypeHints, origin)
if err != nil {

// Use the existing transaction.
if err := prepare(ctx, ex.state.mu.txn); err != nil {
if origin != PreparedStatementOriginSessionMigration {

p.semaCtx.Annotations = tree.MakeAnnotations(stmt.NumAnnotations)
flags, err = ex.populatePrepared(ctx, txn, placeholderHints, p, origin)
return err

// and execute paths.
flags, err := p.prepareUsingOptimizer(ctx)
if err != nil {

memo, err := opc.buildReusableMemo(ctx)
if err != nil {

bld.KeepPlaceholders = true
if err := bld.Build(); err != nil {
return nil, err

// and physical properties.
outScope := b.buildStmtAtRoot(b.stmt, nil /* desiredTypes */)

b.ctes = nil
outScope = b.buildStmt(stmt, desiredTypes, inScope)
// Build With operators for any CTEs hoisted to the top level.

case *tree.CreateFunction:
return b.buildCreateFunction(stmt, inScope)

if opt == tree.FunctionLangPLpgSQL {
if err := utils.ParseAndCollectTelemetryForPLpgSQLFunc(cf); err != nil {
// Until plpgsql is fully implemented DealWithPlpgSQlFunc will always

if _, err := CountPLpgSQLStmt(funcBodyStr); err != nil {
return errors.Wrap(err, "plpgsql not supported in user-defined functions")

v := MakePLpgSQLTelemetryVisitor()
stmt, err := parser.Parse(sql)
if err != nil {

var p Parser
return p.parseWithDepth(1, sql, defaultNakedIntType)
}

sql, tokens, done := p.scanFnBlock()
stmt, err := p.parse(depth+1, sql, tokens, nakedIntType)
if err != nil {

defer p.lexer.cleanup()
if p.parserImpl.Parse(&p.lexer) != 0 {
if p.lexer.lastError == nil {

plpgsql-gen.y#L1034-L1036
GOROOT/src/runtime/signal_unix.go#L838-L840
GOROOT/src/runtime/panic.go#L259-L261
GOROOT/src/runtime/panic.go#L883-L885
}
panic(recErr)
default:

GOROOT/src/runtime/panic.go#L883-L885
// manipulate locks.
if ok, e := errorutil.ShouldCatch(r); ok {
err = e

// get reported to Sentry.
err = errors.HandleAsAssertionFailure(err)
}

GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/sql/pgwire/conn.go in pkg/sql/pgwire.(*conn).processCommandsAsync.func1 at line 310
pkg/sql/conn_executor.go in pkg/sql.(*Server).ServeConn at line 902
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2105
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2314
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).execPrepare at line 76
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).addPreparedStmt at line 115
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare at line 276
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).prepare.func2 at line 271
pkg/sql/conn_executor_prepare.go in pkg/sql.(*connExecutor).populatePrepared at line 333
pkg/sql/plan_opt.go in pkg/sql.(*planner).prepareUsingOptimizer at line 164
pkg/sql/plan_opt.go in pkg/sql.(*optPlanningCtx).buildReusableMemo at line 433
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).Build at line 246
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).buildStmtAtRoot at line 272
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).buildStmt at line 359
pkg/sql/opt/optbuilder/create_function.go in pkg/sql/opt/optbuilder.(*Builder).buildCreateFunction at line 111
pkg/sql/sem/plpgsqltree/utils/plpg_visitor.go in pkg/sql/sem/plpgsqltree/utils.ParseAndCollectTelemetryForPLpgSQLFunc at line 112
pkg/sql/sem/plpgsqltree/utils/plpg_visitor.go in pkg/sql/sem/plpgsqltree/utils.CountPLpgSQLStmt at line 90
pkg/sql/plpgsql/parser/parse.go in pkg/sql/plpgsql/parser.Parse at line 132
pkg/sql/plpgsql/parser/parse.go in pkg/sql/plpgsql/parser.(*Parser).parseWithDepth at line 82
pkg/sql/plpgsql/parser/parse.go in pkg/sql/plpgsql/parser.(*Parser).parse at line 98
plpgsql-gen.y in pkg/sql/plpgsql/parser.(*plpgsqlParserImpl).Parse at line 1035
GOROOT/src/runtime/signal_unix.go in runtime.sigpanic at line 839
GOROOT/src/runtime/panic.go in runtime.panicmem at line 260
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884
pkg/sql/opt/optbuilder/create_function.go in pkg/sql/opt/optbuilder.(*Builder).buildCreateFunction.func1 at line 77
GOROOT/src/runtime/panic.go in runtime.gopanic at line 884
pkg/sql/opt/optbuilder/builder.go in pkg/sql/opt/optbuilder.(*Builder).Build.func1 at line 212
pkg/util/errorutil/catch.go in pkg/util/errorutil.ShouldCatch at line 29

Tags

Tag Value
Command start-single-node
Environment v23.1.13
Go Version go1.19.13
Platform linux amd64
Distribution CCL
Cockroach Release v23.1.13
Cockroach SHA 8d065df
# of CPUs 4
# of Goroutines 579

Jira issue: CRDB-39840

@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 labels Jun 26, 2024
@yuzefovich
Copy link
Member

dup of #109342

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-release-23.1 Used to mark GA and release blockers, technical advisories, and bugs for 23.1 C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Development

No branches or pull requests

2 participants