Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... #114106

Closed
cockroach-sentry opened this issue Nov 8, 2023 · 1 comment
Closed

Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... #114106

cockroach-sentry opened this issue Nov 8, 2023 · 1 comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-sentry
Copy link
Collaborator

cockroach-sentry commented Nov 8, 2023 •
edited by cockroach-jira-scripts
Loading

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/4614916092/?referrer=webhooks_plugin

Panic Message:

type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking
(1)
Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validation_errors.read.backward_references.type]
Wraps: (3) secondary error attachment
  | type × (224): referencing × 256 was dropped without dependency unlinking
  | (1) keys: [sql.schema.validation_errors.read.backward_references.type]
  | Wraps: (2) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:253
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:177
  |   | [...repeated from below...]
  | Wraps: (3) type × (224)
  | Wraps: (4) assertion failure
  | Wraps: (5) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc/type_desc.go:701
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func3
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:105
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:193
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:100
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).Validate
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/validate.go:36
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).finalizeDescriptors
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:566
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.getDescriptorsByID
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:203
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorsByID
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:104
  |   | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:416
  |   | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2533
  |   | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2485
  |   | github.com/cockroachdb/cockroach/pkg/sql.glob..func128
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1274
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:613
  |   | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
  |   | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
  |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  |   | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
  |   | runtime.goexit
  |   | 	GOROOT/src/runtime/asm_amd64.s:1594
  | Wraps: (6) referencing × 256 was dropped without dependency unlinking
  | Error types: (1) *telemetrykeys.withTelemetry (2) *withstack.withStack (3) *errutil.withPrefix (4) *assert.withAssertionFailure (5) *withstack.withStack (6) *errutil.leafError
Wraps: (4) keys: [sql.schema.validation_errors.read.backward_references.type]
Wraps: (5) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:253
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:177
  | [...repeated from below...]
Wraps: (6) type × (223)
Wraps: (7) assertion failure
Wraps: (8) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/typedesc/type_desc.go:701
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func3
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:105
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:193
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:100
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).Validate
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/validate.go:36
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).finalizeDescriptors
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:566
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.getDescriptorsByID
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:203
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).getDescriptorsByID
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/descriptor.go:104
  | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase
  | 	github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:416
  | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2533
  | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2485
  | github.com/cockroachdb/cockroach/pkg/sql.glob..func128
  | 	github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1274
  | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
  | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:613
  | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
  | 	github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  | 	github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:489
  | runtime.goexit
  | 	GOROOT/src/runtime/asm_amd64.s:1594
Wraps: (9) referencing × 256 was dropped without dependency unlinking
Error types: (1) *colexecerror.notInternalError (2) *telemetrykeys.withTelemetry (3) *secondary.withSecondaryError (4) *telemetrykeys.withTelemetry (5) *withstack.withStack (6) *errutil.withPrefix (7) *assert.withAssertionFailure (8) *withstack.withStack (9) *errutil.leafError
-- report composition:
*errutil.leafError: referencing × 256 was dropped without dependency unlinking
type_desc.go:701: *withstack.withStack (top exception)
*assert.withAssertionFailure
*errutil.withPrefix: type × (223)
validate.go:253: *withstack.withStack (1)
*telemetrykeys.withTelemetry: sql.schema.validation_errors.read.backward_references.type
*secondary.withSecondaryError: details for github.com/cockroachdb/errors/telemetrykeys/*telemetrykeys.withTelemetry:::
*telemetrykeys.withTelemetry: sql.schema.validation_errors.read.backward_references.type
*colexecerror.notInternalError
(check the extra data payloads)
Stacktrace (expand for inline code snippets):

GOROOT/src/runtime/asm_amd64.s#L1593-L1595

cockroach/pkg/util/stop/stopper.go

Lines 488 to 490 in af6752b

sp.UpdateGoroutineIDToCurrent()
f(ctx)
}()

cockroach/pkg/sql/virtual_table.go

Lines 126 to 128 in af6752b

}
err := worker(ctx, funcRowPusher(addRow))
// Notify that we are done sending rows.

cockroach/pkg/sql/virtual_schema.go

Lines 612 to 614 in af6752b

generator, cleanup, setupError := setupGenerator(ctx, func(ctx context.Context, pusher rowPusher) error {
return def.populate(ctx, p, dbDesc, func(row ...tree.Datum) error {
if err := e.validateRow(row, columns); err != nil {

cockroach/pkg/sql/information_schema.go

Lines 1273 to 1275 in af6752b

h := makeOidHasher()
return forEachTableDescWithTableLookup(ctx, p, dbContext, hideVirtual, /* virtual tables have no constraints */
func(

cockroach/pkg/sql/information_schema.go

Lines 2484 to 2486 in af6752b

) error {
return forEachTableDescWithTableLookupInternal(
ctx, p, dbContext, virtualOpts, false /* allowAdding */, fn,

cockroach/pkg/sql/information_schema.go

Lines 2532 to 2534 in af6752b

if dbContext != nil && useIndexLookupForDescriptorsInDatabase.Get(&p.EvalContext().Settings.SV) {
all, err = p.Descriptors().GetAllDescriptorsForDatabase(ctx, p.txn, dbContext)
} else {

cockroach/pkg/sql/catalog/descs/collection.go

Lines 415 to 417 in af6752b

// getDescriptorsByID must be used to ensure proper validation hydration etc.
descs, err := tc.getDescriptorsByID(ctx, txn, flags, ids.Ordered()...)
if err != nil {

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 103 to 105 in af6752b

descs := make([]catalog.Descriptor, len(ids))
if err := getDescriptorsByID(
ctx, tc, txn, flags, descs, ids...,

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 202 to 204 in af6752b

// At this point, all descriptors are in the slice, finalize and hydrate them.
if err := tc.finalizeDescriptors(ctx, txn, flags, descs, vls); err != nil {
return err

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 565 to 567 in af6752b

if len(toValidate) > 0 {
if err := tc.Validate(ctx, txn, catalog.ValidationReadTelemetry, requiredLevel, toValidate...); err != nil {
return err

cockroach/pkg/sql/catalog/descs/validate.go

Lines 35 to 37 in af6752b

version := tc.settings.Version.ActiveVersion(ctx)
return validate.Validate(
ctx,

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 99 to 101 in af6752b

// Descriptor backward-reference checks.
if !vea.validateDescriptorsAtLevel(
catalog.ValidationLevelBackReferences,

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 192 to 194 in af6752b

vea.currentDescriptor = desc
validationFn(desc)
}

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 104 to 106 in af6752b

if !desc.Dropped() {
desc.ValidateBackReferences(&vea, vdg)
}

cockroach/pkg/sql/catalog/typedesc/type_desc.go

Lines 700 to 702 in af6752b

if depDesc.Dropped() {
vea.Report(errors.AssertionFailedf(
"referencing %s %d was dropped without dependency unlinking", depDesc.DescriptorType(), id))

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 176 to 178 in af6752b

}
vea.errors = append(vea.errors, vea.decorate(err))
}

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 252 to 254 in af6752b

case catalog.Type:
err = errors.Wrapf(err, string(catalog.Type)+" %q (%d)", name, id)
case catalog.Function:

GOROOT/src/runtime/asm_amd64.s#L1593-L1595

cockroach/pkg/util/stop/stopper.go

Lines 488 to 490 in af6752b

sp.UpdateGoroutineIDToCurrent()
f(ctx)
}()

cockroach/pkg/sql/virtual_table.go

Lines 126 to 128 in af6752b

}
err := worker(ctx, funcRowPusher(addRow))
// Notify that we are done sending rows.

cockroach/pkg/sql/virtual_schema.go

Lines 612 to 614 in af6752b

generator, cleanup, setupError := setupGenerator(ctx, func(ctx context.Context, pusher rowPusher) error {
return def.populate(ctx, p, dbDesc, func(row ...tree.Datum) error {
if err := e.validateRow(row, columns); err != nil {

cockroach/pkg/sql/information_schema.go

Lines 1273 to 1275 in af6752b

h := makeOidHasher()
return forEachTableDescWithTableLookup(ctx, p, dbContext, hideVirtual, /* virtual tables have no constraints */
func(

cockroach/pkg/sql/information_schema.go

Lines 2484 to 2486 in af6752b

) error {
return forEachTableDescWithTableLookupInternal(
ctx, p, dbContext, virtualOpts, false /* allowAdding */, fn,

cockroach/pkg/sql/information_schema.go

Lines 2532 to 2534 in af6752b

if dbContext != nil && useIndexLookupForDescriptorsInDatabase.Get(&p.EvalContext().Settings.SV) {
all, err = p.Descriptors().GetAllDescriptorsForDatabase(ctx, p.txn, dbContext)
} else {

cockroach/pkg/sql/catalog/descs/collection.go

Lines 415 to 417 in af6752b

// getDescriptorsByID must be used to ensure proper validation hydration etc.
descs, err := tc.getDescriptorsByID(ctx, txn, flags, ids.Ordered()...)
if err != nil {

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 103 to 105 in af6752b

descs := make([]catalog.Descriptor, len(ids))
if err := getDescriptorsByID(
ctx, tc, txn, flags, descs, ids...,

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 202 to 204 in af6752b

// At this point, all descriptors are in the slice, finalize and hydrate them.
if err := tc.finalizeDescriptors(ctx, txn, flags, descs, vls); err != nil {
return err

cockroach/pkg/sql/catalog/descs/descriptor.go

Lines 565 to 567 in af6752b

if len(toValidate) > 0 {
if err := tc.Validate(ctx, txn, catalog.ValidationReadTelemetry, requiredLevel, toValidate...); err != nil {
return err

cockroach/pkg/sql/catalog/descs/validate.go

Lines 35 to 37 in af6752b

version := tc.settings.Version.ActiveVersion(ctx)
return validate.Validate(
ctx,

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 99 to 101 in af6752b

// Descriptor backward-reference checks.
if !vea.validateDescriptorsAtLevel(
catalog.ValidationLevelBackReferences,

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 192 to 194 in af6752b

vea.currentDescriptor = desc
validationFn(desc)
}

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 104 to 106 in af6752b

if !desc.Dropped() {
desc.ValidateBackReferences(&vea, vdg)
}

cockroach/pkg/sql/catalog/typedesc/type_desc.go

Lines 700 to 702 in af6752b

if depDesc.Dropped() {
vea.Report(errors.AssertionFailedf(
"referencing %s %d was dropped without dependency unlinking", depDesc.DescriptorType(), id)) 

GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 489
pkg/sql/virtual_table.go in pkg/sql.setupGenerator.func3 at line 127
pkg/sql/virtual_schema.go in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1 at line 613
pkg/sql/information_schema.go in pkg/sql.glob..func128 at line 1274
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookup at line 2485
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookupInternal at line 2533
pkg/sql/catalog/descs/collection.go in pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase at line 416
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).getDescriptorsByID at line 104
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.getDescriptorsByID at line 203
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).finalizeDescriptors at line 566
pkg/sql/catalog/descs/validate.go in pkg/sql/catalog/descs.(*Collection).Validate at line 36
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate at line 100
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel at line 193
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate.func3 at line 105
pkg/sql/catalog/typedesc/type_desc.go in pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences at line 701
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report at line 177
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate at line 253
GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1594
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 489
pkg/sql/virtual_table.go in pkg/sql.setupGenerator.func3 at line 127
pkg/sql/virtual_schema.go in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1 at line 613
pkg/sql/information_schema.go in pkg/sql.glob..func128 at line 1274
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookup at line 2485
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookupInternal at line 2533
pkg/sql/catalog/descs/collection.go in pkg/sql/catalog/descs.(*Collection).GetAllDescriptorsForDatabase at line 416
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).getDescriptorsByID at line 104
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.getDescriptorsByID at line 203
pkg/sql/catalog/descs/descriptor.go in pkg/sql/catalog/descs.(*Collection).finalizeDescriptors at line 566
pkg/sql/catalog/descs/validate.go in pkg/sql/catalog/descs.(*Collection).Validate at line 36
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate at line 100
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel at line 193
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate.func3 at line 105
pkg/sql/catalog/typedesc/type_desc.go in pkg/sql/catalog/typedesc.(*immutable).ValidateBackReferences at line 701

Tags

Tag Value
Command server
Environment v22.2.14
Go Version go1.19.6
Platform linux amd64
Distribution CCL
Cockroach Release v22.2.14
Cockroach SHA af6752b
# of CPUs 8
# of Goroutines 570

Jira issue: CRDB-33345
@cockroach-sentry cockroach-sentry added O-sentry Originated from an in-the-wild panic report. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. labels Nov 8, 2023
@yuzefovich
Copy link
Member

dup of #91645

@exalate-issue-sync exalate-issue-sync bot changed the title Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... Sentry: type_desc.go:701: type × (223): referencing × 256 was dropped without dependency unlinking (1) Wraps: (2) keys: [sql.schema.validation_errors.read.backward_references.type sql.schema.validat... Nov 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yuzefovich @cockroach-sentry