Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sql: flake in TestEvaluatedMemoryIsChecked #100119

Closed
knz opened this issue Mar 30, 2023 · 0 comments · Fixed by #100550
Closed

sql: flake in TestEvaluatedMemoryIsChecked #100119

knz opened this issue Mar 30, 2023 · 0 comments · Fixed by #100550
Assignees
@yuzefovich
Labels
A-sql-execution Relating to SQL execution. A-sql-memmon SQL memory monitoring C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). T-sql-queries SQL Queries Team

Comments

@knz
Copy link
Contributor

knz commented Mar 30, 2023 •
edited by cockroach-jira-scripts
Loading

Found here:
https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests_BazelUnitTests/9340738?showRootCauses=false&expandBuildChangesSection=true&expandBuildProblemsSection=true&expandBuildTestsSection=true

Failed
=== RUN   TestEvaluatedMemoryIsChecked/#00
    test_server_shim.go:331: migration-manager-find-jobs: system-jobs-scan: root: memory budget exceeded: 20480 bytes requested, 4986880 currently allocated, 4999999 bytes in budget
        (1) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func1.1
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:855
          | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:409
          | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:460
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).queryInternalBuffered
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:570
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).QueryBufferedEx
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:533
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).QueryBuffered
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:515
          | github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).getRunningMigrationJob
          |   github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager/manager.go:767
          | github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).getOrCreateMigrationJob.func1
          |   github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager/manager.go:729
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).Txn.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:1422
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).txn.func4
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:1500
          | github.com/cockroachdb/cockroach/pkg/kv.runTxn.func1
          |   github.com/cockroachdb/cockroach/pkg/kv/db.go:965
          | github.com/cockroachdb/cockroach/pkg/kv.(*Txn).exec
          |   github.com/cockroachdb/cockroach/pkg/kv/txn.go:928
          | github.com/cockroachdb/cockroach/pkg/kv.runTxn
          |   github.com/cockroachdb/cockroach/pkg/kv/db.go:964
          | github.com/cockroachdb/cockroach/pkg/kv.(*DB).TxnWithAdmissionControl
          |   github.com/cockroachdb/cockroach/pkg/kv/db.go:927
          | github.com/cockroachdb/cockroach/pkg/kv.(*DB).Txn
          |   github.com/cockroachdb/cockroach/pkg/kv/db.go:902
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).txn
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:1488
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalDB).Txn
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:1423
          | github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).getOrCreateMigrationJob
          |   github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager/manager.go:718
          | github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).runMigration
          |   github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager/manager.go:700
          | github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager.(*Manager).RunPermanentUpgrades
          |   github.com/cockroachdb/cockroach/pkg/upgrade/upgrademanager/manager.go:271
          | github.com/cockroachdb/cockroach/pkg/server.(*SQLServer).preStart
          |   github.com/cockroachdb/cockroach/pkg/server/server_sql.go:1582
          | github.com/cockroachdb/cockroach/pkg/server.(*Server).PreStart
          |   github.com/cockroachdb/cockroach/pkg/server/server.go:1902
          | github.com/cockroachdb/cockroach/pkg/server.(*TestServer).Start
          |   github.com/cockroachdb/cockroach/pkg/server/testserver.go:612
          | github.com/cockroachdb/cockroach/pkg/testutils/serverutils.StartServer
          |   github.com/cockroachdb/cockroach/pkg/testutils/serverutils/test_server_shim.go:330
          | github.com/cockroachdb/cockroach/pkg/sql.TestEvaluatedMemoryIsChecked.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/builtin_mem_usage_test.go:110
          | testing.tRunner
          |   GOROOT/src/testing/testing.go:1446
        Wraps: (2) migration-manager-find-jobs
        Wraps: (3)
        Wraps: (4) secondary error attachment
          | system-jobs-scan: root: memory budget exceeded: 20480 bytes requested, 4986880 currently allocated, 4999999 bytes in budget
          | (1) attached stack trace
          |   -- stack trace:
          |   | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func1.1
          |   |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:855
          |   | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next.func1
          |   |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:409
          |   | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
          |   |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:460
          |   | github.com/cockroachdb/cockroach/pkg/sql.populateSystemJobsTableRows
          |   |   github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:1050
          |   | github.com/cockroachdb/cockroach/pkg/sql.glob..func35
          |   |   github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:1005
          |   | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).makeConstrainedRowsGenerator.func1
          |   |   github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:823
          |   | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
          |   |   github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
          |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
          |   |   github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
          | Wraps: (2) system-jobs-scan
          | Wraps: (3) attached stack trace
          |   -- stack trace:
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:990
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).adjustMemoryUsage
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:532
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).AdjustMemoryUsageAfterAllocation
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:558
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformAppend
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:506
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*AppendOnlyBufferedBatch).AppendTuples
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/utils.go:207
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash.(*HashTable).FullBuild
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash/hashtable.go:499
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).build
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:329
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).Next
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:291
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:202
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:200
          | Wraps: (4) root
          | Wraps: (5) Consider increasing --max-sql-memory startup parameter.
          | Wraps: (6) candidate pg code: 53200
          | Wraps: (7) attached stack trace
          |   -- stack trace:
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.newMemoryBudgetExceededError
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/resource.go:29
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.memoryResourceWithErrorHint.NewBudgetExceededError
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/resource.go:70
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:990
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          |   | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).adjustMemoryUsage
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:532
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).AdjustMemoryUsageAfterAllocation
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:558
          |   | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformAppend
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:506
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*AppendOnlyBufferedBatch).AppendTuples
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/utils.go:207
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash.(*HashTable).FullBuild
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash/hashtable.go:499
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).build
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:329
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).Next
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:291
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:202
          |   | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
          |   |   github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
          | Wraps: (8) memory budget exceeded: 20480 bytes requested, 4986880 currently allocated, 4999999 bytes in budget
          | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *hintdetail.withHint (6) *pgerror.withCandidateCode (7) *withstack.withStack (8) *errutil.leafError
        Wraps: (5) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func1.1
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:855
          | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:409
          | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
          |   github.com/cockroachdb/cockroach/pkg/sql/internal.go:460
          | github.com/cockroachdb/cockroach/pkg/sql.populateSystemJobsTableRows
          |   github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:1050
          | github.com/cockroachdb/cockroach/pkg/sql.glob..func35
          |   github.com/cockroachdb/cockroach/pkg/sql/crdb_internal.go:1005
          | github.com/cockroachdb/cockroach/pkg/sql.(*virtualDefEntry).makeConstrainedRowsGenerator.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/virtual_schema.go:823
          | github.com/cockroachdb/cockroach/pkg/sql.setupGenerator.func3
          |   github.com/cockroachdb/cockroach/pkg/sql/virtual_table.go:127
          | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
          |   github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:470
        Wraps: (6) system-jobs-scan
        Wraps: (7) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:990
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).adjustMemoryUsage
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:532
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).AdjustMemoryUsageAfterAllocation
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:558
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformAppend
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:506
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*AppendOnlyBufferedBatch).AppendTuples
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/utils.go:207
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash.(*HashTable).FullBuild
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash/hashtable.go:499
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).build
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:329
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).Next
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:291
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:202
          | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
          |   github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:200
        Wraps: (8) root
        Wraps: (9) Consider increasing --max-sql-memory startup parameter.
        Wraps: (10) candidate pg code: 53200
        Wraps: (11) attached stack trace
          -- stack trace:
          | github.com/cockroachdb/cockroach/pkg/util/mon.newMemoryBudgetExceededError
          |   github.com/cockroachdb/cockroach/pkg/util/mon/resource.go:29
          | github.com/cockroachdb/cockroach/pkg/util/mon.memoryResourceWithErrorHint.NewBudgetExceededError
          |   github.com/cockroachdb/cockroach/pkg/util/mon/resource.go:70
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:990
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).increaseBudget
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:998
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BytesMonitor).reserveBytes
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:923
          | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Grow
          |   github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:869
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).adjustMemoryUsage
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:532
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).AdjustMemoryUsageAfterAllocation
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:558
          | github.com/cockroachdb/cockroach/pkg/sql/colmem.(*Allocator).PerformAppend
          |   github.com/cockroachdb/cockroach/pkg/sql/colmem/allocator.go:506
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils.(*AppendOnlyBufferedBatch).AppendTuples
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecutils/utils.go:207
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash.(*HashTable).FullBuild
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexechash/hashtable.go:499
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).build
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:329
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin.(*hashJoiner).Next
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecjoin/hashjoiner.go:291
          | github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk.(*diskSpillerBase).Next.func1
          |   github.com/cockroachdb/cockroach/pkg/sql/colexec/colexecdisk/disk_spiller.go:202
          | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
          |   github.com/cockroachdb/cockroach/pkg/sql/colexecerror/error.go:92
        Wraps: (12) memory budget exceeded: 20480 bytes requested, 4986880 currently allocated, 4999999 bytes in budget
        Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *colexecerror.notInternalError (4) *secondary.withSecondaryError (5) *withstack.withStack (6) *errutil.withPrefix (7) *withstack.withStack (8) *errutil.withPrefix (9) *hintdetail.withHint (10) *pgerror.withCandidateCode (11) *withstack.withStack (12) *errutil.leafError
    --- FAIL: TestEvaluatedMemoryIsChecked/#00 (63.53s)

Relates to #79014.

Jira issue: CRDB-26263
@knz knz added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). A-sql-execution Relating to SQL execution. A-sql-memmon SQL memory monitoring labels Mar 30, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Mar 30, 2023
@knz knz mentioned this issue Mar 30, 2023
Merged
This was referenced Apr 3, 2023
Closed
Merged
@yuzefovich yuzefovich self-assigned this Apr 4, 2023
@yuzefovich yuzefovich mentioned this issue Apr 4, 2023
Closed
@craig craig bot closed this as completed in 0d15e0d Apr 4, 2023
@yuzefovich yuzefovich mentioned this issue Apr 4, 2023
Merged
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yuzefovich yuzefovich

Labels
A-sql-execution Relating to SQL execution. A-sql-memmon SQL memory monitoring C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql: adjust a couple of memory monitoring tests yuzefovich/cockroach
2 participants
@knz @yuzefovich