Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkg/sql/opt/exec/execbuilder/tests/local/local_test: TestExecBuild_sql_statistics_persisted failed #99118

Closed
cockroach-teamcity opened this issue Mar 21, 2023 · 2 comments · Fixed by #99155
Closed

pkg/sql/opt/exec/execbuilder/tests/local/local_test: TestExecBuild_sql_statistics_persisted failed #99118

cockroach-teamcity opened this issue Mar 21, 2023 · 2 comments · Fixed by #99155
Assignees
@rytaft
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Milestone
23.1

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Mar 21, 2023 •
edited by cockroach-jira-scripts
Loading

pkg/sql/opt/exec/execbuilder/tests/local/local_test.TestExecBuild_sql_statistics_persisted failed with artifacts on master @ 5f5cf265a8717da8ee327f41c0d06836d11ac8ec:

=== RUN   TestExecBuild_sql_statistics_persisted
    test_log_scope.go:161: test logs captured to: /artifacts/tmp/_tmp/9986f69167dd2633633c749cd542ec4c/logTestExecBuild_sql_statistics_persisted2954968955
    test_log_scope.go:79: use -show-logs to present logs inline
    logic.go:2874: 
         
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/11604/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/opt/exec/execbuilder/tests/local/local_test_/local_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/opt/exec/execbuilder/testdata/sql_statistics_persisted:73: EXPLAIN (VERBOSE)
        SELECT
          aggregated_ts,
          fingerprint_id,
          app_name,
          node_id,
          agg_interval,
          metadata,
          statistics
        FROM
          system.transaction_statistics
        expected:
              distribution: local
              vectorized: true
            ·
              • scan
                columns: (aggregated_ts, fingerprint_id, app_name, node_id, agg_interval, metadata, statistics)
                estimated row count: 1,000 (missing stats)
                table: transaction_statistics@primary
                spans: FULL SCAN
            
        but found (query options: "") :
            distribution: local
            vectorized: true
            ·
            • scan
              columns: (aggregated_ts, fingerprint_id, app_name, node_id, agg_interval, metadata, statistics)
              estimated row count: 1 (100% of the table; stats collected <hidden> ago)
              table: transaction_statistics@primary
              spans: FULL SCAN
[13:34:23] --- done: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/11604/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/opt/exec/execbuilder/tests/local/local_test_/local_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/opt/exec/execbuilder/testdata/sql_statistics_persisted with config local: 3 tests, 1 failures
    logic.go:3972: 
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/11604/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/opt/exec/execbuilder/tests/local/local_test_/local_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/opt/exec/execbuilder/testdata/sql_statistics_persisted:95: error while processing
    logic.go:3972: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/11604/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/sql/opt/exec/execbuilder/tests/local/local_test_/local_test.runfiles/com_github_cockroachdb_cockroach/pkg/sql/opt/exec/execbuilder/testdata/sql_statistics_persisted:95: too many errors encountered, skipping the rest of the input
    panic.go:522: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/9986f69167dd2633633c749cd542ec4c/logTestExecBuild_sql_statistics_persisted2954968955
--- FAIL: TestExecBuild_sql_statistics_persisted (2.43s)
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-25709
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels Mar 21, 2023
@cockroach-teamcity cockroach-teamcity added this to the 23.1 milestone Mar 21, 2023
@blathers-crl blathers-crl bot added the T-sql-queries SQL Queries Team label Mar 21, 2023
@cucaroach
Copy link
Contributor

Looks like this test needs to turn off auto stats creation maybe?

@yuzefovich
Copy link
Member

Stats should already been turned off, this issue is similar to what we fixed in #97775. It appears that somehow system tables don't respect the fact that auto stats are turned off.

@rytaft rytaft self-assigned this Mar 21, 2023
rytaft added a commit to rytaft/cockroach that referenced this issue Mar 21, 2023
@rytaft
sql/logictest: disable stats collection for system tables earlier
a160c61 
This commit updates the logictests to disable stats collection for system
tables before the test cluster is started. This avoids a race condition where
stats might be collected on system tables before collection is disabled with
SET CLUSTER SETTING.

This should prevent flakes for tests that show EXPLAIN output for queries over
system tables.

Fixes cockroachdb#99118

Release note: None
@rytaft rytaft mentioned this issue Mar 21, 2023
Merged
craig bot pushed a commit that referenced this issue Mar 22, 2023
@stevendanna @yuzefovich
@michae2 @rytaft @ZhouXing19
Merge #98671 #98914 #98917 #99155 #99173
704f5ad 
98671: streamingccl: use fingerprinting in more tests r=dt a=stevendanna

This uses fingerprinting in more tests to avoid starting the destination tenant before cutting over.

Epic: none

Release note: None

98914: sqlsmith: add DELETE FROM ... USING and UPDATE ... FROM support r=yuzefovich a=yuzefovich

This commit makes it so that the sqlsmith can now generate statements of the form `DELETE FROM ... USING` and `UPDATE ... FROM`. We toss a coin every time before deciding to add another table (meaning in 50% cases these forms are not used, in 25% we have 1 extra table, etc). It also adjusts the generation of the RETURNING clause for DELETE and UPDATE to be able to pick from any of the table sources.

Fixes: #98910.

Release note: None

98917: sql: add prepared_statements_cache_size setting r=yuzefovich a=michae2

Add a new circular doubly-linked list of prepared statements to `prepStmtNamespace` which tracks the least-recently-used prepared statement. When new setting `prepared_statements_cache_size` is set, use this LRU list to automatically deallocate prepared statements.

Fixes: #97866

Epic: None

Release note (sql change): Add a new `prepared_statements_cache_size` setting which, when set to a non-zero number of bytes, causes the least-recently-used prepared statements to be automatically deallocated when prepared statement memory usage goes above the cache size. This setting can be used to avoid prepared statement leaks from long-lived connections which never `DEALLOCATE` prepared statements.

99155: sql/logictest: disable stats collection for system tables earlier r=rytaft a=rytaft

This commit updates the logictests to disable stats collection for system tables before the test cluster is started. This avoids a race condition where stats might be collected on system tables before collection is disabled with `SET CLUSTER SETTING`.

This should prevent flakes for tests that show `EXPLAIN` output for queries over system tables.

Fixes #99118

Release note: None

99173: sql: enable resumption of a flow for pausable portals r=yuzefovich a=ZhouXing19

This PR is part of the implementation of multiple active portals. (Extracted from #96358)

We now introduce a `Resume()` method for flow, and when a pausable portal is being re-executed, rather than generating a new flow, we resume the persisted flow to continue the previous execution.

---
### sql: add telemetry MultipleActivePortalCounter

This commit added a telemetry counter `MultipleActivePortalCounter` that would
be incremented each time the cluster setting
`sql.pgwire.multiple_active_portals.enabled` is set to true

---
### sql: add Resume method for flowinfra.Flow and execinfra.Processor

For pausable portals, each execution needs to resume the processor with new
output receiver. We don't need to restart the processors, and this `Resume()`
step can be called many times after `Run()` is called.

----
### sql: reuse flow for pausable portal
To execute portals in an interleaving manner, we need to persist the flow and
queryID so that we can _continue_ fetching the result when we come back to the same
portal.

We now introduce `pauseInfo` field in `sql.PreparedPortal` that stores this
metadata. It's set during the first-time execution of an engine, and all
following executions will reuse the flow and the queryID. This also implies that
these resources should not be cleaned up with the end of each execution.
Implementation for the clean-up steps is included in the next commit.

Also, in this commit we hang a `*PreparedPortal` to the planner, and how it is
set can be seen in the next commit as well.

Epic: CRDB-17622

Release note: None


Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
Co-authored-by: Michael Erickson <[email protected]>
Co-authored-by: Rebecca Taft <[email protected]>
Co-authored-by: Jane Xing <[email protected]>
@craig craig bot closed this as completed in 9c6fcd1 Mar 22, 2023
@blathers-crl blathers-crl bot mentioned this issue Mar 22, 2023
Merged
blathers-crl bot pushed a commit that referenced this issue Mar 22, 2023
@rytaft
sql/logictest: disable stats collection for system tables earlier
1153696 
This commit updates the logictests to disable stats collection for system
tables before the test cluster is started. This avoids a race condition where
stats might be collected on system tables before collection is disabled with
SET CLUSTER SETTING.

This should prevent flakes for tests that show EXPLAIN output for queries over
system tables.

Fixes #99118

Release note: None
@mgartner mgartner moved this to Done in SQL Queries Jul 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rytaft rytaft

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. T-sql-queries SQL Queries Team
Projects
SQL Queries
Archived in project
Milestone
23.1
Development

Successfully merging a pull request may close this issue.

sql/logictest: disable stats collection for system tables earlier rytaft/cockroach
4 participants
@rytaft @yuzefovich @cockroach-teamcity @cucaroach