Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui: obtain redacted statement diagnostics bundles from the DB console #119040

Closed
michae2 opened this issue Feb 10, 2024 · 3 comments · Fixed by #134723
Closed

ui: obtain redacted statement diagnostics bundles from the DB console #119040

michae2 opened this issue Feb 10, 2024 · 3 comments · Fixed by #134723
Assignees
Labels
A-observability-inf A-sql-debug-bundle Issues related to statement bundle improvements A-sql-explain Issues related to EXPLAIN and EXPLAIN ANALYZE improvements A-sql-ui Why is my query slow? branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-support Would prevent or help troubleshoot a customer escalation - bugs, missing observability/tooling, docs P-3 Issues/test failures with no fix SLA T-observability

Comments

@michae2
Copy link
Collaborator

michae2 commented Feb 10, 2024

In 23.1 we added the ability to collect redacted statement diagnostics bundles using EXPLAIN ANALYZE (DEBUG, REDACT) which was tracked by #68570 and #98817.

In order to make collection of these redacted bundles easier, we should add a redaction option to the DB console's existing UI for bundle collection.

We should probably also add a redaction option to crdb_internal.request_statement_bundle().

Jira issue: CRDB-35994

Epic CRDB-37557

@michae2 michae2 added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-sql-ui Why is my query slow? A-sql-explain Issues related to EXPLAIN and EXPLAIN ANALYZE improvements T-observability-inf T-sql-queries SQL Queries Team labels Feb 10, 2024
@github-project-automation github-project-automation bot moved this to Triage in SQL Queries Feb 10, 2024
@michae2
Copy link
Collaborator Author

michae2 commented Feb 10, 2024

In 23.1 we also added:

  • redacted query plans using EXPLAIN (REDACT)
  • redacted create statements using SHOW CREATE ... WITH REDACT

Maybe these should also be exposed through the DB console somehow?

@michae2 michae2 added A-sql-debug-bundle Issues related to statement bundle improvements O-support Would prevent or help troubleshoot a customer escalation - bugs, missing observability/tooling, docs P-3 Issues/test failures with no fix SLA labels Feb 10, 2024
@mgartner mgartner moved this from Triage to New Backlog in SQL Queries Feb 20, 2024
@mgartner mgartner moved this from New Backlog to 24.2 Release in SQL Queries Feb 20, 2024
@exalate-issue-sync exalate-issue-sync bot added T-observability and removed T-sql-queries SQL Queries Team labels Apr 4, 2024
@yuzefovich yuzefovich added the T-sql-queries SQL Queries Team label May 2, 2024
@yuzefovich
Copy link
Member

#123859 made all the necessary changes to the backend (added new overloads to crdb_internal.request_statement_bundle builtin as well as extended the server API). The remaining work is left on the frontend side of things will be tackled by the Observability team (currently planned for second half of the year), so I'm removing this from the Queries board.

@yuzefovich yuzefovich removed the T-sql-queries SQL Queries Team label May 15, 2024
@yuzefovich yuzefovich removed this from SQL Queries May 15, 2024
@yuzefovich yuzefovich changed the title obtain redacted statement diagnostics bundles from the DB console ui: obtain redacted statement diagnostics bundles from the DB console May 15, 2024
dhartunian added a commit to dhartunian/cockroach that referenced this issue Nov 8, 2024
When users with the `VIEWACTIVITY` role activate statement diagnostics
via the modal in SQL Activity, they now have the option to redact the
output via a checkbox.

Resolves: cockroachdb#119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in
the DB Console, users now have the option to produce a redacted bundle
as output. This bundle will omit sensitive data.
craig bot pushed a commit that referenced this issue Nov 11, 2024
134229: changefeedccl: fix race in test r=andyyang890 a=asg0451

Fix a data race in duration overrides in tests.

Fixes: #133817

Release note: None


134723: ui: add Redact option to stmt diag activation modal r=kyle-a-wong a=dhartunian

When users with the `VIEWACTIVITY` role activate statement diagnostics via the modal in SQL Activity, they now have the option to redact the output via a checkbox.

Resolves: #119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data.

134746: scjob: support handling permanent job errors r=fqazi a=fqazi

Previously, when a descriptor was detect as a dropped the error was marked as permanent for schema change jobs. However, the declarative schema change job framework did not properly handle these errors. As a result, even if the erorr was tagged we would keep attempting rollbacks. To address this, this patch makes the OnFailOrCancel logic detect permanent errors and allows them to cause the schema change to hit the terminal state.

Fixes: #131405
Release note: None

Co-authored-by: Miles Frankel <[email protected]>
Co-authored-by: David Hartunian <[email protected]>
Co-authored-by: Faizan Qazi <[email protected]>
craig bot pushed a commit that referenced this issue Nov 11, 2024
134723: ui: add Redact option to stmt diag activation modal r=kyle-a-wong a=dhartunian

When users with the `VIEWACTIVITY` role activate statement diagnostics via the modal in SQL Activity, they now have the option to redact the output via a checkbox.

Resolves: #119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data.

Co-authored-by: David Hartunian <[email protected]>
dhartunian added a commit to dhartunian/cockroach that referenced this issue Nov 11, 2024
When users with the `VIEWACTIVITY` role activate statement diagnostics
via the modal in SQL Activity, they now have the option to redact the
output via a checkbox.

Resolves: cockroachdb#119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in
the DB Console, users now have the option to produce a redacted bundle
as output. This bundle will omit sensitive data.
craig bot pushed a commit that referenced this issue Nov 12, 2024
134525: kvserver: avoid key heap allocation in multiSSTWriter.rolloverSST r=tbg a=sumeerbhola

rolloverSST is called for every key-value pair in the incoming snapshot, and the key parameter was inadvertently escaping to the heap. This was 20% of the number of allocations on a node in the 150 node cluster test.

Epic: none

Release note: None

134723: ui: add Redact option to stmt diag activation modal r=dhartunian a=dhartunian

When users with the `VIEWACTIVITY` role activate statement diagnostics via the modal in SQL Activity, they now have the option to redact the output via a checkbox.

Resolves: #119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in the DB Console, users now have the option to produce a redacted bundle as output. This bundle will omit sensitive data.

134919: sql: fix FETCH ABSOLUTE 0 cursor handling r=yuzefovich a=yuzefovich

`FETCH ABSOLUTE 0` positions the cursor before the first row, so nothing should be fetched. Previously, this would result in an internal error because we'd be working on unset datums.

Fixes: #131115.

Release note (bug fix): Previously, CockroachDB would encounter an internal error when evaluating `FETCH ABSOLUTE 0` statements, and this is now fixed. The bug has been present since 22.1 version.

Co-authored-by: sumeerbhola <[email protected]>
Co-authored-by: David Hartunian <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
@craig craig bot closed this as completed in 5d21117 Nov 12, 2024
Copy link

blathers-crl bot commented Nov 12, 2024

Based on the specified backports for linked PR #134723, I applied the following new label(s) to this issue: branch-release-24.3. Please adjust the labels as needed to match the branches actually affected by this issue, including adding any known older branches.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@blathers-crl blathers-crl bot added the branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 label Nov 12, 2024
blathers-crl bot pushed a commit that referenced this issue Nov 12, 2024
When users with the `VIEWACTIVITY` role activate statement diagnostics
via the modal in SQL Activity, they now have the option to redact the
output via a checkbox.

Resolves: #119040
Epic: CRDB-37557

Release note (ui change): when activating statement diagnostics in
the DB Console, users now have the option to produce a redacted bundle
as output. This bundle will omit sensitive data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-observability-inf A-sql-debug-bundle Issues related to statement bundle improvements A-sql-explain Issues related to EXPLAIN and EXPLAIN ANALYZE improvements A-sql-ui Why is my query slow? branch-release-24.3 Used to mark GA and release blockers, technical advisories, and bugs for 24.3 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) O-support Would prevent or help troubleshoot a customer escalation - bugs, missing observability/tooling, docs P-3 Issues/test failures with no fix SLA T-observability
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants