Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... #114118

Closed
cockroach-sentry opened this issue Nov 9, 2023 · 1 comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Nov 9, 2023

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/4615494196/?referrer=webhooks_plugin

Panic Message:

type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking
(1)
Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validation_errors.read.backward_references.type]
Wraps: (3) secondary error attachment
  | type × (224): referencing × 256 was dropped without dependency unlinking
  | (1) keys: [sql.schema.validation_errors.read.backward_references.type]
  | Wraps: (2) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:253
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:177
  |   | [...repeated from below...]
  | Wraps: (3) type × (224)
  | Wraps: (4) assertion failure
  | Wraps: (5) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc/type_desc.go:701
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func3
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:105
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:193
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:100
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).Validate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/validate.go:36
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).finalizeDescriptors
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:566
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.getDescriptorsByID
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:203
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorsByID
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:104
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:416
  |   | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2533
  |   | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2485
  |   | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDesc
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2413
  |   | github.com/cockroachdb/cockroach/pkg/sql.glob..func130
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1437
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:613
  |   | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
  |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  |   | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
  |   | runtime.goexit
  |   | 	GOROOT/src/runtime/asm_amd64.s:1594
  | Wraps: (6) referencing × 256 was dropped without dependency unlinking
  | Error types: (1) *telemetrykeys.withTelemetry (2) *withstack.withStack (3) *errutil.withPrefix (4) *assert.withAssertionFailure (5) *withstack.withStack (6) *errutil.leafError
Wraps: (4) keys: [sql.schema.validation_errors.read.backward_references.type]
Wraps: (5) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:253
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:177
  | [...repeated from below...]
Wraps: (6) type × (223)
Wraps: (7) assertion failure
Wraps: (8) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc/type_desc.go:701
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func3
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:105
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:193
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:100
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).Validate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/validate.go:36
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).finalizeDescriptors
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:566
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.getDescriptorsByID
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:203
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorsByID
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:104
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:416
  | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2533
  | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2485
  | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDesc
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2413
  | github.com/cockroachdb/cockroach/pkg/sql.glob..func130
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1437
  | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
  | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:613
  | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
  | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
  | runtime.goexit
  | 	GOROOT/src/runtime/asm_amd64.s:1594
Wraps: (9) referencing × 256 was dropped without dependency unlinking
Error types: (1) *colexecerror.notInternalError (2) *telemetrykeys.withTelemetry (3) *secondary.withSecondaryError (4) *telemetrykeys.withTelemetry (5) *withstack.withStack (6) *errutil.withPrefix (7) *assert.withAssertionFailure (8) *withstack.withStack (9) *errutil.leafError
-- report composition:
*errutil.leafError: referencing × 256 was dropped without dependency unlinking
type_desc.go:701: *withstack.withStack (top exception)
*assert.withAssertionFailure
*errutil.withPrefix: type × (223)
validate.go:253: *withstack.withStack (1)
*telemetrykeys.withTelemetry: sql.schema.validation_errors.read.backward_references.type
*secondary.withSecondaryError: details for github.com/cockroachdb/errors/telemetrykeys/*telemetrykeys.withTelemetry:::
*telemetrykeys.withTelemetry: sql.schema.validation_errors.read.backward_references.type
*colexecerror.notInternalError
(check the extra data payloads)
Stacktrace (expand for inline code snippets):

GOROOT/src/runtime/asm_amd64.s#L1593-L1595

sp.UpdateGoroutineIDToCurrent()
f(ctx)
}()

}
err := worker(ctx, funcRowPusher(addRow))
// Notify that we are done sending rows.

generator, cleanup, setupError := setupGenerator(ctx, func(ctx context.Context, pusher rowPusher) error {
return def.populate(ctx, p, dbDesc, func(row ...tree.Datum) error {
if err := e.validateRow(row, columns); err != nil {

populate: func(ctx context.Context, p *planner, dbContext catalog.DatabaseDescriptor, addRow func(...tree.Datum) error) error {
return forEachTableDesc(ctx, p, dbContext, virtualMany, addTablesTableRow(addRow))
},

) error {
return forEachTableDescWithTableLookup(ctx, p, dbContext, virtualOpts, func(
db catalog.DatabaseDescriptor,

) error {
return forEachTableDescWithTableLookupInternal(
ctx, p, dbContext, virtualOpts, false /* allowAdding */, fn,

if dbContext != nil && useIndexLookupForDescriptorsInDatabase.Get(&p.EvalContext().Settings.SV) {
all, err = p.Descriptors().GetAllDescriptorsForDatabase(ctx, p.txn, dbContext)
} else {

// getDescriptorsByID must be used to ensure proper validation hydration etc.
descs, err := tc.getDescriptorsByID(ctx, txn, flags, ids.Ordered()...)
if err != nil {

descs := make([]catalog.Descriptor, len(ids))
if err := getDescriptorsByID(
ctx, tc, txn, flags, descs, ids...,

// At this point, all descriptors are in the slice, finalize and hydrate them.
if err := tc.finalizeDescriptors(ctx, txn, flags, descs, vls); err != nil {
return err

if len(toValidate) > 0 {
if err := tc.Validate(ctx, txn, catalog.ValidationReadTelemetry, requiredLevel, toValidate...); err != nil {
return err

version := tc.settings.Version.ActiveVersion(ctx)
return validate.Validate(
ctx,

// Descriptor backward-reference checks.
if !vea.validateDescriptorsAtLevel(
catalog.ValidationLevelBackReferences,

vea.currentDescriptor = desc
validationFn(desc)
}

if !desc.Dropped() {
desc.ValidateBackReferences(&vea, vdg)
}

if depDesc.Dropped() {
vea.Report(errors.AssertionFailedf(
"referencing %s %d was dropped without dependency unlinking", depDesc.DescriptorType(), id))

}
vea.errors = append(vea.errors, vea.decorate(err))
}

case catalog.Type:
err = errors.Wrapf(err, string(catalog.Type)+" %q (%d)", name, id)
case catalog.Function:

GOROOT/src/runtime/asm_amd64.s#L1593-L1595
sp.UpdateGoroutineIDToCurrent()
f(ctx)
}()

}
err := worker(ctx, funcRowPusher(addRow))
// Notify that we are done sending rows.

generator, cleanup, setupError := setupGenerator(ctx, func(ctx context.Context, pusher rowPusher) error {
return def.populate(ctx, p, dbDesc, func(row ...tree.Datum) error {
if err := e.validateRow(row, columns); err != nil {

populate: func(ctx context.Context, p *planner, dbContext catalog.DatabaseDescriptor, addRow func(...tree.Datum) error) error {
return forEachTableDesc(ctx, p, dbContext, virtualMany, addTablesTableRow(addRow))
},

) error {
return forEachTableDescWithTableLookup(ctx, p, dbContext, virtualOpts, func(
db catalog.DatabaseDescriptor,

) error {
return forEachTableDescWithTableLookupInternal(
ctx, p, dbContext, virtualOpts, false /* allowAdding */, fn,

if dbContext != nil && useIndexLookupForDescriptorsInDatabase.Get(&p.EvalContext().Settings.SV) {
all, err = p.Descriptors().GetAllDescriptorsForDatabase(ctx, p.txn, dbContext)
} else {

// getDescriptorsByID must be used to ensure proper validation hydration etc.
descs, err := tc.getDescriptorsByID(ctx, txn, flags, ids.Ordered()...)
if err != nil {

descs := make([]catalog.Descriptor, len(ids))
if err := getDescriptorsByID(
ctx, tc, txn, flags, descs, ids...,

// At this point, all descriptors are in the slice, finalize and hydrate them.
if err := tc.finalizeDescriptors(ctx, txn, flags, descs, vls); err != nil {
return err

if len(toValidate) > 0 {
if err := tc.Validate(ctx, txn, catalog.ValidationReadTelemetry, requiredLevel, toValidate...); err != nil {
return err

version := tc.settings.Version.ActiveVersion(ctx)
return validate.Validate(
ctx,

// Descriptor backward-reference checks.
if !vea.validateDescriptorsAtLevel(
catalog.ValidationLevelBackReferences,

vea.currentDescriptor = desc
validationFn(desc)
}

if !desc.Dropped() {
desc.ValidateBackReferences(&vea, vdg)
}

if depDesc.Dropped() {
vea.Report(errors.AssertionFailedf(
"referencing %s %d was dropped without dependency unlinking", depDesc.DescriptorType(), id))

GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 489
pkg/sql/virtual_table.go in pkg/sql.setupGenerator.func3 at line 127
pkg/sql/virtual_schema.go in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1 at line 613
pkg/sql/information_schema.go in pkg/sql.glob..func130 at line 1437
pkg/sql/information_schema.go in pkg/sql.forEachTableDesc at line 2413
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookup at line 2485
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookupInternal at line 2533
pkg/sql/catalog/descs/collection.go in pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase at line 416
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).getDescriptorsByID at line 104
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.getDescriptorsByID at line 203
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).finalizeDescriptors at line 566
pkg/sql/catalog/descs/validate.go in pkg/sql/catalog/descs.(*Collection).Validate at line 36
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate at line 100
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel at line 193
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate.func3 at line 105
pkg/sql/catalog/typedesc/type_desc.go in pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences at line 701
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report at line 177
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate at line 253
GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 489
pkg/sql/virtual_table.go in pkg/sql.setupGenerator.func3 at line 127
pkg/sql/virtual_schema.go in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1 at line 613
pkg/sql/information_schema.go in pkg/sql.glob..func130 at line 1437
pkg/sql/information_schema.go in pkg/sql.forEachTableDesc at line 2413
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookup at line 2485
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookupInternal at line 2533
pkg/sql/catalog/descs/collection.go in pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase at line 416
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).getDescriptorsByID at line 104
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.getDescriptorsByID at line 203
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).finalizeDescriptors at line 566
pkg/sql/catalog/descs/validate.go in pkg/sql/catalog/descs.(*Collection).Validate at line 36
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate at line 100
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel at line 193
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate.func3 at line 105
pkg/sql/catalog/typedesc/type_desc.go in pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences at line 701

Tags

Tag Value
Command server
Environment v22.2.14
Go Version go1.19.6
Platform linux amd64
Distribution CCL
Cockroach Release v22.2.14
Cockroach SHA af6752b
# of CPUs 8
# of Goroutines 564

Jira issue: CRDB-33351

@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels Nov 9, 2023
@yuzefovich
Copy link
Member

dup of #91645

@exalate-issue-sync exalate-issue-sync bot changed the title Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... Nov 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Development

No branches or pull requests

2 participants