Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql/rowcontainer: TestDiskRowContainer failed #96420

Closed
cockroach-teamcity opened this issue Feb 2, 2023 · 1 comment
Closed

sql/rowcontainer: TestDiskRowContainer failed #96420

cockroach-teamcity opened this issue Feb 2, 2023 · 1 comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Feb 2, 2023 •
edited by cockroach-jira-scripts
Loading

sql/rowcontainer.TestDiskRowContainer failed with artifacts on master @ c4257c934858dcdb54cec514c1d5642d4992d5c2:

Fatal error:

panic: concurrent write operations detected on file [recovered]
	panic: concurrent write operations detected on file

Stack:

goroutine 190 [running]:
github.com/cockroachdb/pebble.(*DB).runCompaction.func1()
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:2258 +0x23d
panic({0x279f620, 0x33fc7e0})
	GOROOT/src/runtime/panic.go:890 +0x262
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).timeDiskOp(0xc00077ed20, 0x2, 0xc000f86508)
	github.com/cockroachdb/pebble/vfs/external/com_github_cockroachdb_pebble/vfs/disk_health.go:262 +0x1a5
github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).Sync(0xc00077ed20)
	github.com/cockroachdb/pebble/vfs/external/com_github_cockroachdb_pebble/vfs/disk_health.go:219 +0x69
github.com/cockroachdb/pebble/vfs.(*enospcFile).Sync(0xc000339b18)
	github.com/cockroachdb/pebble/vfs/external/com_github_cockroachdb_pebble/vfs/disk_full.go:391 +0x6d
github.com/cockroachdb/pebble.(*DB).runCompaction(0xc001858000, 0x4b, 0xc000dd2400)
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:2836 +0x312f
github.com/cockroachdb/pebble.(*DB).compact1(0xc001858000, 0xc000dd2400, 0x0)
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:2199 +0x31d
github.com/cockroachdb/pebble.(*DB).compact.func1({0x341b220, 0xc00044de30})
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:2170 +0xd6
runtime/pprof.Do({0x341b1b0, 0xc00013e008}, {{0xc0002fcc20?, 0xc000dc1140?, 0xc0008208a0?}}, 0xc000091f78)
	GOROOT/src/runtime/pprof/runtime.go:40 +0x123
github.com/cockroachdb/pebble.(*DB).compact(0xc001858000, 0xc000dd2400, 0x0)
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:2167 +0xb6
created by github.com/cockroachdb/pebble.(*DB).maybeScheduleCompactionPicker
	github.com/cockroachdb/pebble/external/com_github_cockroachdb_pebble/compaction.go:1888 +0x1105
Log preceding fatal error 

=== RUN   TestDiskRowContainer
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/2464ce3205778481c7831decfd0edcdf/logTestDiskRowContainer1466308123
    test_log_scope.go:79: use -show-logs to present logs inline
=== RUN   TestDiskRowContainer/EncodeDecode

Parameters: TAGS=bazel,gss,race

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-24115
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Feb 2, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Feb 2, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Feb 2, 2023
@yuzefovich
Copy link
Member

should be fixed by cockroachdb/pebble#2298

jbowens added a commit to jbowens/cockroach that referenced this issue Feb 2, 2023
@jbowens
go.mod: bump Pebble to e9d3bb388ad6
f89ff0c 
```
e9d3bb38 vfs: handle concurrent directory Syncs in disk-health checking
917d3f3e db: add Options.WithFSDefaults
9fc4a208 db: flushable ingested sstable implementation
4a453f64 Revert "db: unflake TestArchiveCleaner"
31c33365 db: unflake TestArchiveCleaner
6f3bed0d pebble: minor cleanup around obsolete tables
35c90436 objstorage: add link-or-copy functionality
d443ab31 objstorage: use provider in table cache
0ff0f5d4 pebble: add a test for fatal message when table cache hits "no such file"
2367e8d7 sstable: introduce objstorage interface
59603de1 vfs: move Prefetch to vfs.File
654253a6 sstable: sort user-added range keys by suffix descending
```

Epic: None
Release note: None

Close cockroachdb#96414.
Close cockroachdb#96422.
Informs cockroachdb#96420.
Informs cockroachdb#96421.
@jbowens jbowens mentioned this issue Feb 2, 2023
Merged
craig bot pushed a commit that referenced this issue Feb 3, 2023
@iAaronBuck @jbowens
Merge #95868 #96446
18fda97 
95868: ui: set overview page node list data with stale tag if node is dead r=iAaronBuck a=iAaronBuck

71618: ui: set overview page node list data with stale tag if node is dead

![Screen Shot 2023-01-25 at 4 25 17 PM](https://user-images.githubusercontent.com/73749490/214694475-b946ec5b-0834-4cf0-86a2-cbe28ab529a7.png)
    
Issue:  [#71618](#71618)
Epic: None
    
Release note (ui change): Currently, the stale node metrics displayed to a user in the Cluster Overview Nodes Table may mislead users in to thinking that they are current values when in fact they are stale. This change rectifies that and adds a stale tag to metrics displayed to the user. This allows for users to be informed about the staleness of the data displayed to them regarding dead nodes.

96446: go.mod: bump Pebble to e9d3bb388ad6 r=RaduBerinde a=jbowens

```
e9d3bb38 vfs: handle concurrent directory Syncs in disk-health checking
917d3f3e db: add Options.WithFSDefaults
9fc4a208 db: flushable ingested sstable implementation
4a453f64 Revert "db: unflake TestArchiveCleaner"
31c33365 db: unflake TestArchiveCleaner
6f3bed0d pebble: minor cleanup around obsolete tables
35c90436 objstorage: add link-or-copy functionality
d443ab31 objstorage: use provider in table cache
0ff0f5d4 pebble: add a test for fatal message when table cache hits "no such file"
2367e8d7 sstable: introduce objstorage interface
59603de1 vfs: move Prefetch to vfs.File
654253a6 sstable: sort user-added range keys by suffix descending
```

Epic: None
Release note: None

Close #96414.
Close #96422.
Informs #96420.
Informs #96421.

Co-authored-by: Aaron Buck <[email protected]>
Co-authored-by: Jackson Owens <[email protected]>
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
Milestone
23.1
Development

No branches or pull requests
2 participants
@yuzefovich @cockroach-teamcity