Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sentry: validate.go:298: relation × (2514): invalid depended-on-by relation back reference: referenced table ID 2445: referenced descriptor not found (1) Wraps: (2) keys: [sql.schema.validation_errors.read.cr... #96171

Closed
cockroach-teamcity opened this issue Jan 30, 2023 · 1 comment
Closed

sentry: validate.go:298: relation × (2514): invalid depended-on-by relation back reference: referenced table ID 2445: referenced descriptor not found (1) Wraps: (2) keys: [sql.schema.validation_errors.read.cr... #96171

cockroach-teamcity opened this issue Jan 30, 2023 · 1 comment
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jan 30, 2023 •
edited by cockroach-jira-scripts
Loading

This issue was autofiled by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry link: https://sentry.io/organizations/cockroach-labs/issues/3905508974/?referrer=webhooks_plugin

Panic message:

validate.go:298: relation × (2514): invalid depended-on-by relation back reference: referenced table ID 2445: referenced descriptor not found
(1)
Wraps: (2) keys: [sql.schema.validation_errors.read.cross_references.relation]
Wraps: (3) keys: [sql.schema.validation_errors.read.cross_references.relation]
Wraps: (4) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:214
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:149
| [...repeated from below...]
Wraps: (5) relation × (2514)
Wraps: (6) assertion failure
Wraps: (7) attached stack trace
-- stack trace:
| github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc.(*wrapper).validateInboundTableRef
| github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc/validate.go:298
| github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc.(*wrapper).ValidateCrossReferences
| github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc/validate.go:188
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func2
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:77
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:165
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
| github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:72
| github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree.Catalog.Validate
| github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree/catalog.go:144
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*kvDescriptors).getAllDescriptors
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/kv_descriptors.go:279
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptors
| github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:284
| github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
| github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2403
| github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
| github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2357
| github.com/cockroachdb/cockroach/pkg/sql.forEachTableDesc
| github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2285
| github.com/cockroachdb/cockroach/pkg/sql.glob..func166
| github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1420
| github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
| github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:567
| github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
| github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
| github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:494
| runtime.goexit
| GOROOT/src/runtime/asm_amd64.s:1581
Wraps: (8) invalid depended-on-by relation back reference
Wraps: (9) referenced table ID 2445: referenced descriptor not found
| -- cause hidden behind barrier
| referenced table ID 2445: referenced descriptor not found
| (1) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/sql/catalog.WrapTableDescRefErr
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/errors.go:164
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationDescGetterImpl).GetTableDescriptor
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:284
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc.(*wrapper).validateInboundTableRef
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc/validate.go:296
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc.(*wrapper).ValidateCrossReferences
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/tabledesc/validate.go:188
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate.func2
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:77
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:165
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate.Validate
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/internal/validate/validate.go:72
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree.Catalog.Validate
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/nstree/catalog.go:144
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*kvDescriptors).getAllDescriptors
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/kv_descriptors.go:279
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs.(*Collection).GetAllDescriptors
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/descs/collection.go:284
| | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookupInternal
| | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2403
| | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDescWithTableLookup
| | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2357
| | github.com/cockroachdb/cockroach/pkg/sql.forEachTableDesc
| | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:2285
| | github.com/cockroachdb/cockroach/pkg/sql.glob..func166
| | github.com/cockroachdb/cockroach/pkg/sql/information_schema.go:1420
| | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1
| | github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:567
| | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
| | github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
| | [...repeated from below...]
| Wraps: (2) referenced table ID 2445
| Wraps: (3) attached stack trace
| -- stack trace:
| | github.com/cockroachdb/cockroach/pkg/sql/catalog.init
| | github.com/cockroachdb/cockroach/pkg/sql/catalog/errors.go:88
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6498
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.doInit
| | GOROOT/src/runtime/proc.go:6475
| | runtime.main
| | GOROOT/src/runtime/proc.go:238
| | runtime.goexit
| | GOROOT/src/runtime/asm_amd64.s:1581
| Wraps: (4) referenced descriptor not found
| Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.leafError
Error types: (1) *colexecerror.notInternalError (2) *telemetrykeys.withTelemetry (3) *telemetrykeys.withTelemetry (4) *withstack.withStack (5) *errutil.withPrefix (6) *assert.withAssertionFailure (7) *withstack.withStack (8) *errutil.withPrefix (9) *barriers.barrierErr
-- report composition:
*barriers.barrierErr: details for github.com/cockroachdb/errors/withstack/*withstack.withStack:::
*errutil.withPrefix: invalid depended-on-by relation back reference
validate.go:298: *withstack.withStack (top exception)
*assert.withAssertionFailure
*errutil.withPrefix: relation × (2514)
validate.go:214: *withs...

Stacktrace (expand for inline code snippets):

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 213 to 215 in 8dfa17e

case catalog.Table:
err = errors.Wrapf(err, catalog.Table+" %q (%d)", name, id)
case catalog.Database:
in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 148 to 150 in 8dfa17e

}
vea.errors = append(vea.errors, vea.decorate(err))
}
in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report

cockroach/pkg/sql/catalog/tabledesc/validate.go

Lines 187 to 189 in 8dfa17e

for _, by := range desc.DependedOnBy {
vea.Report(desc.validateInboundTableRef(by, vdg))
}
in pkg/sql/catalog/tabledesc.(*wrapper).ValidateCrossReferences

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 76 to 78 in 8dfa17e

if !desc.Dropped() {
desc.ValidateCrossReferences(&vea, vdg)
}
in pkg/sql/catalog/internal/validate.Validate.func2

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 164 to 166 in 8dfa17e

vea.currentDescriptor = desc
validationFn(desc)
}
in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel

cockroach/pkg/sql/catalog/internal/validate/validate.go

Lines 71 to 73 in 8dfa17e

// Descriptor cross-reference checks.
if !vea.validateDescriptorsAtLevel(
catalog.ValidationLevelCrossReferences,
in pkg/sql/catalog/internal/validate.Validate

cockroach/pkg/sql/catalog/nstree/catalog.go

Lines 143 to 145 in 8dfa17e

) (ve catalog.ValidationErrors) {
return validate.Validate(ctx, version, c, telemetry, targetLevel, descriptors...)
}
in pkg/sql/catalog/nstree.Catalog.Validate

cockroach/pkg/sql/catalog/descs/kv_descriptors.go

Lines 278 to 280 in 8dfa17e

descs := c.OrderedDescriptors()
ve := c.Validate(ctx, version, catalog.ValidationReadTelemetry, catalog.ValidationLevelCrossReferences, descs...)
if err := ve.CombinedError(); err != nil {
in pkg/sql/catalog/descs.(*kvDescriptors).getAllDescriptors

cockroach/pkg/sql/catalog/descs/collection.go

Lines 283 to 285 in 8dfa17e

func (tc *Collection) GetAllDescriptors(ctx context.Context, txn *kv.Txn) (nstree.Catalog, error) {
return tc.kv.getAllDescriptors(ctx, txn, tc.version)
}
in pkg/sql/catalog/descs.(*Collection).GetAllDescriptors

cockroach/pkg/sql/information_schema.go

Lines 2402 to 2404 in 8dfa17e

) error {
all, err := p.Descriptors().GetAllDescriptors(ctx, p.txn)
if err != nil {
in pkg/sql.forEachTableDescWithTableLookupInternal

cockroach/pkg/sql/information_schema.go

Lines 2356 to 2358 in 8dfa17e

) error {
return forEachTableDescWithTableLookupInternal(
ctx, p, dbContext, virtualOpts, false /* allowAdding */, fn,
in pkg/sql.forEachTableDescWithTableLookup

cockroach/pkg/sql/information_schema.go

Lines 2284 to 2286 in 8dfa17e

) error {
return forEachTableDescWithTableLookup(ctx, p, dbContext, virtualOpts, func(
db catalog.DatabaseDescriptor,
in pkg/sql.forEachTableDesc

cockroach/pkg/sql/information_schema.go

Lines 1419 to 1421 in 8dfa17e

populate: func(ctx context.Context, p *planner, dbContext catalog.DatabaseDescriptor, addRow func(...tree.Datum) error) error {
return forEachTableDesc(ctx, p, dbContext, virtualMany, addTablesTableRow(addRow))
},
in pkg/sql.glob..func166

cockroach/pkg/sql/virtual_schema.go

Lines 566 to 568 in 8dfa17e

generator, cleanup, setupError := setupGenerator(ctx, func(ctx context.Context, pusher rowPusher) error {
return def.populate(ctx, p, dbDesc, func(row ...tree.Datum) error {
if err := e.validateRow(row, columns); err != nil {
in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1

cockroach/pkg/sql/virtual_table.go

Lines 126 to 128 in 8dfa17e

}
err := worker(ctx, funcRowPusher(addRow))
// If the query was canceled, next() will already return a
in pkg/sql.setupGenerator.func3

cockroach/pkg/util/stop/stopper.go

Lines 493 to 495 in 8dfa17e

f(ctx)
}()
in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
GOROOT/src/runtime/asm_amd64.s#L1580-L1582 in runtime.goexit

pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).decorate at line 214
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).Report at line 149
pkg/sql/catalog/tabledesc/validate.go in pkg/sql/catalog/tabledesc.(*wrapper).ValidateCrossReferences at line 188
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate.func2 at line 77
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.(*validationErrorAccumulator).validateDescriptorsAtLevel at line 165
pkg/sql/catalog/internal/validate/validate.go in pkg/sql/catalog/internal/validate.Validate at line 72
pkg/sql/catalog/nstree/catalog.go in pkg/sql/catalog/nstree.Catalog.Validate at line 144
pkg/sql/catalog/descs/kv_descriptors.go in pkg/sql/catalog/descs.(*kvDescriptors).getAllDescriptors at line 279
pkg/sql/catalog/descs/collection.go in pkg/sql/catalog/descs.(*Collection).GetAllDescriptors at line 284
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookupInternal at line 2403
pkg/sql/information_schema.go in pkg/sql.forEachTableDescWithTableLookup at line 2357
pkg/sql/information_schema.go in pkg/sql.forEachTableDesc at line 2285
pkg/sql/information_schema.go in pkg/sql.glob..func166 at line 1420
pkg/sql/virtual_schema.go in pkg/sql.(*virtualDefEntry).getPlanInfo.func1.1 at line 567
pkg/sql/virtual_table.go in pkg/sql.setupGenerator.func3 at line 127
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 494
GOROOT/src/runtime/asm_amd64.s in runtime.goexit at line 1581
Tag Value
Cockroach Release v22.1.13
Cockroach SHA: 8dfa17e
Platform linux amd64
Distribution CCL
Environment v22.1.13
Command server
Go Version ``
# of CPUs
# of Goroutines

Jira issue: CRDB-23980
@cockroach-teamcity cockroach-teamcity added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report. labels Jan 30, 2023
@yuzefovich
Copy link
Member

dup of #91645

@exalate-issue-sync exalate-issue-sync bot changed the title sentry: validate.go:298: relation × (2514): invalid depended-on-by relation back reference: referenced table ID 2445: referenced descriptor not found (1) Wraps: (2) keys: [sql.schema.validation_errors.read.cr... sentry: validate.go:298: relation × (2514): invalid depended-on-by relation back reference: referenced table ID 2445: referenced descriptor not found (1) Wraps: (2) keys: [sql.schema.validation_errors.read.cr... Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-sentry Originated from an in-the-wild panic report.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@yuzefovich @cockroach-teamcity