Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Older statement bundles no longer visible in the UI #80104

Closed
ghost opened this issue Apr 18, 2022 · 0 comments · Fixed by #88138
Closed

Older statement bundles no longer visible in the UI #80104

ghost opened this issue Apr 18, 2022 · 0 comments · Fixed by #88138
Assignees
Labels
A-sql-observability Related to observability of the SQL layer C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.

Comments

@ghost
Copy link

ghost commented Apr 18, 2022

Describe the problem

Statement bundles disappear after a short time but are accessible via the CLI.

To Reproduce

What did you do? Describe in your own words.

If possible, provide steps to reproduce the behavior:

  1. Run cockroach demo movr --num-histories 250000 --num-promo-codes 250000 --num-rides 125000 --num-users 12500 --num-vehicles 3750 --nodes 3 --insecure
  2. Run SELECT
    name, count(rides.id) AS sum
    FROM
    users JOIN rides ON users.id = rides.rider_id
    WHERE
    rides.start_time BETWEEN '2018-12-31 00:00:00' AND '2020-01-01 00:00:00'
    GROUP BY
    name
    ORDER BY
    sum DESC
    LIMIT
    10;
  3. Start diagnostics collection for statement in 2.
  4. Run SELECT
    name, count(rides.id) AS sum
    FROM
    users JOIN rides ON users.id = rides.rider_id
    WHERE
    rides.start_time BETWEEN '2018-12-31 00:00:00' AND '2020-01-01 00:00:00'
    GROUP BY
    name
    ORDER BY
    sum DESC
    LIMIT
    10;
  5. Start diagnostics collection for 2.
  6. Run drop index rides_start_time_idx; to drop index.
  7. Run SELECT
    name, count(rides.id) AS sum
    FROM
    users JOIN rides ON users.id = rides.rider_id
    WHERE
    rides.start_time BETWEEN '2018-12-31 00:00:00' AND '2020-01-01 00:00:00'
    GROUP BY
    name
    ORDER BY
    sum DESC
    LIMIT
    10;
  8. View Statement diagnostics history
  9. Over less than 60 minutes, bundles are dropped from history.

Screen Shot 2022-04-18 at 8 05 43 AM

Screen Shot 2022-04-18 at 8 18 05 AM

Screen Shot 2022-04-18 at 8 22 35 AM

Expected behavior
Statement bundles retained forever

Additional data / screenshots
If the problem is SQL-related, include a copy of the SQL query and the schema
of the supporting tables.

If a node in your cluster encountered a fatal error, supply the contents of the
log directories (at minimum of the affected node(s), but preferably all nodes).

Note that log files can contain confidential information. Please continue
creating this issue, but contact [email protected] to submit the log
files in private.

If applicable, add screenshots to help explain your problem.

Environment:

  • CockroachDB version V22.1.0-BETA.1-176-G1EC23D5938 and V22.1.0-BETA.2-151-GA0FEF45C83
  • Server OS: MacOS
  • Client app: cockroach sql

Additional context
What was the impact?

Add any other context about the problem here.

I can see them using cockroach statement-diag list --insecure:
Statement diagnostics bundles:
ID Collection time Statement
754409175114907649 2022-04-18 16:03:29 UTC SELECT name, count(rides.id) AS sum FROM users JOIN rides ON users.id = rides.rider_id WHERE rides.start_time BETWEEN '' AND '' GROUP BY name ORDER BY sum DESC LIMIT _
754408888394907649 2022-04-18 16:02:01 UTC SELECT name, count(rides.id) AS sum FROM users JOIN rides ON users.id = rides.rider_id WHERE rides.start_time BETWEEN '' AND '' GROUP BY name ORDER BY sum DESC LIMIT _

No outstanding activation requests.

But they are not visible in the UI:

Screen Shot 2022-04-18 at 8 42 25 AM

Screen Shot 2022-04-18 at 10 38 10 AM

Jira issue: CRDB-15845

@ghost ghost added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-observability A-sql-cli-observability Issues related to surfacing SQL observability in SHOW, CRDB_INTERNAL, SYSTEM, etc. labels Apr 18, 2022
@ghost ghost added A-sql-observability Related to observability of the SQL layer and removed A-sql-cli-observability Issues related to surfacing SQL observability in SHOW, CRDB_INTERNAL, SYSTEM, etc. labels Apr 18, 2022
@ghost ghost changed the title Statement bundles no longer retained Statement bundles no longer visible in the UI Apr 18, 2022
@ghost ghost changed the title Statement bundles no longer visible in the UI Older statement bundles no longer visible in the UI Apr 18, 2022
@jlinder jlinder added sync-me and removed sync-me labels May 20, 2022
@maryliag maryliag assigned matthewtodd and unassigned jocrl Jun 7, 2022
@matthewtodd matthewtodd removed their assignment Aug 26, 2022
craig bot pushed a commit that referenced this issue Sep 20, 2022
86334: *: add restriction to running DDL with internal executors r=ajwerner,rafiss a=ZhouXing19

The current internal executor has its lifecycle, which makes it erroneous
when being used to execute DDL statements if under an outer txn. 

In this commit, we 
1. Migrated the existing DDLs with internal executor with not-nil txn to either 
`descs.CollectionFactory.TxnWithExecutor()` or `planner.WithInternalExecutor()`.
Only internal executors inited via these 2 interfaces are bounded with txn-related
metadata, and hence are allowed to run DDLs in a transactional manner.
2. Added a restriction for running DDLs with internal executor only if it's bound with
txn-related metadata.

fixes #87281

Release justification: bug fix for the internal executor
Release note: none

87304: roachtest: refactor github issue posting and expose actual vm args for reporting r=renatolabs a=smg260

The change is split into 2 commits. 
1. More strictly addresses [81846](#81846).  Also addresses [this](https://github.com/cockroachdb/cockroach/pull/87304/files#diff-1e3ec1fc5bade2df4b8681c44959be2a390f703839f30a979da98f963769c9fcR1046) existing TODO
2. Refactors functions related to posting an issue to github into a separate source file and adds tests.

These commits can be squashed but are separate at the moment for convenience. 

Resolves: #81846

Release justification: test-only change
Release note: none

87896: ci: fix validation for mixed version schema change corpus r=fqazi a=fqazi

Informs: #86381

Previously, the command for validating the mixed
version corpus was invalid. This patch fixes the validation 
command to address this, so the mixed version corpus is 
automatically uploaded.

Release note: None

88138: server: return all completed stmt diag requests r=xinhaoz a=xinhaoz

Fixes #80104

Previously, we only return statement diagnostics requests that have not yet expired. Since we use the results of this request to populate completed statement diagnostics bundles in addition to outstanding requests, completed statement diag bundles would disappear from the UI after the request expired.

This commit ensures that `StatementDiagnosticsRequests` returns all completed stmt diag requests so that we can display the complete history of completed bundles in the UI.

Release note (bug fix): completed stmt diagnostics bundles now persist in the UI in stmt diag bundle pages

88159: ui/cluster-ui: fix jobs page polling r=xinhaoz a=xinhaoz

Fixes #68109

Previously, the jobs page would not poll for new data until a re-render was triggered. This commit updates the jobs page polling to every 10s regardless of
whether or not the rest of the page has changed.

Release note (bug fix): jobs page refreshes page data at an interval of 10s

88165: sql: version gate idx recommendations in insert-stmt-stats r=ericharmeling a=ericharmeling

This commit version gates index recommendation insert in insert-stmt-stats.

Fixes #88140.

Release note: None

88168: api: increase timeout of sql api r=maryliag a=maryliag

Previously, some request using the sql-over-http
api were hitting a timeout, with the default value of 5s. This commit increases to 300s (5min) on the calls made from SQL Activity and Insights pages.

Fixes #88094

Release note: None

88173: cli/sql: put a limit on history size r=DrewKimball a=knz

Fixes #54679.

Previously, there was no limit. Some users managed to make their history run into megabyte-size, despite de-duplication, which was causing slowness.

This patch fixes it by adding a limit of 1000 entries. Sufficiently large to not be inconvenient, but sufficiently small that it prevents the history file from growing abnormally large.

Release note (cli change): The interactive SQL shell now retains a maximum of 1000 entries. There was no limit previously.

88193: kv: hoist log.ExpensiveLogEnabled outside loop in appendRefreshSpans r=arulajmani a=nvanbenschoten

This commit hoists the call to `log.ExpensiveLogEnabled` outside of the per-span loop in `txnSpanRefresher.appendRefreshSpans`. The function is too expensive to call once per span.

Release justification: low risk change that may close perf gap.

Release note: None.

Co-authored-by: Jane Xing <[email protected]>
Co-authored-by: Miral Gadani <[email protected]>
Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: Eric Harmeling <[email protected]>
Co-authored-by: Marylia Gutierrez <[email protected]>
Co-authored-by: Raphael 'kena' Poss <[email protected]>
Co-authored-by: Nathan VanBenschoten <[email protected]>
@craig craig bot closed this as completed in da6c653 Sep 20, 2022
blathers-crl bot pushed a commit that referenced this issue Sep 20, 2022
Fixes #80104

Previously, we only return statement diagnostics requests
that have not yet expired. Since we use the results of this
request to populate completed statement diagnostics bundles
in addition to outstanding requests, completed statement
diag bundles would disappear from the UI after the request
expired.

This commit ensures that `StatementDiagnosticsRequests`
returns all completed stmt diag requests so that we can
display the complete history of completed bundles in the UI.

Release note (bug fix): completed stmt diagnostics bundles
now persist in the UI in stmt diag bundle pages
xinhaoz added a commit to xinhaoz/cockroach that referenced this issue Sep 21, 2022
Fixes cockroachdb#80104

Previously, we only return statement diagnostics requests
that have not yet expired. Since we use the results of this
request to populate completed statement diagnostics bundles
in addition to outstanding requests, completed statement
diag bundles would disappear from the UI after the request
expired.

This commit ensures that `StatementDiagnosticsRequests`
returns all completed stmt diag requests so that we can
display the complete history of completed bundles in the UI.

Release note (bug fix): completed stmt diagnostics bundles
now persist in the UI in stmt diag bundle pages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sql-observability Related to observability of the SQL layer C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants