Skip to content

Commit

Permalink
Add 1.1 page on managing long-running queries
Browse files Browse the repository at this point in the history
Also update query performance troubleshooting page to link to new page,
and remove details about query logging (for now).
  • Loading branch information
Jesse Seldess committed Aug 23, 2017
1 parent e942000 commit fd31a6c
Show file tree
Hide file tree
Showing 3 changed files with 73 additions and 56 deletions.
52 changes: 0 additions & 52 deletions v1.1/debug-and-error-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ As a command generates messages, CockroachDB uses the [command](#commands)'s [lo

Each node's logs detail only the internal activity of that node without visibility into the behavior of other nodes in the cluster. When troubleshooting, this means that you must identify the node where the problem occurred or [collect the logs from all active nodes in your cluster](debug-zip.html).

{{site.data.alerts.callout_info}}You can also <a href="#log-queries">log queries</a> your cluster receives.{{site.data.alerts.end}}

### Commands

All [`cockroach` commands](cockroach-commands.html) support logging. However, it's important to note:
Expand Down Expand Up @@ -93,56 +91,6 @@ By default, commands besides `cockroach start` discard messages with the `INFO`

The `--log-backtrace-at`, `--verbosity`, and `--v` flags are intended for internal debugging by CockroachDB contributors.

## Log Queries

To help troubleshoot [query performance issues](query-behavior-troubleshooting.html#performance-issues), you can use [cluster-wide settings](cluster-settings.html) to enable logging for long-running SQL transactions or all queries, regardless of time.

{{site.data.alerts.callout_danger}}These settings makes <em>all</em> queries slower and causes nodes to consume more memory. You should <a href="#disable-query-logging">disable query logging</a> as soon as you're done troubleshooting the query's issues.{{site.data.alerts.end}}

### Enable Query Logging

- **Long-running transactions**:

~~~ sql
> SET CLUSTER SETTING sql.trace.txn.enable_threshold = '[time]';
~~~

The `[time]` parameter accepts common time specifiers, such as `100ms` or `2s`.

- **All queries**:

~~~ sql
> SET CLUSTER SETTING sql.trace.log_statement_execute = true;
~~~

### Details

After you enable query logging, whenever nodes process SQL statements, they generate messages with an `INFO` [severity level](#severity-levels).

By default, these messages will get [written to files](#write-to-file), but are ultimate handled by whatever logging behavior you set for [`cockroach start`](start-a-node.html#logging).

### Improve Query Performance

After finding which queries are slow, use [`EXPLAIN`](explain.html) to examine them. It's possible that the query is slow because it performs a full-table scan. In these cases, you can likely improve the query's performance by [adding an index](create-index.html).

*(More guidance around query performance optimization forthcoming.)*

### Disable Query Logging

Once you're done troubleshooting, you should disable query logging to prevent it from unnecessarily consuming resources.
- **Long-running transactions**:
~~~ sql
> SET CLUSTER SETTING sql.trace.txn.enable_threshold = '0s';
~~~
- **All queries**:
~~~ sql
> SET CLUSTER SETTING sql.trace.log_statement_execute = false;
~~~
## See Also

- [Troubleshooting Overview](troubleshooting-overview.html)
Expand Down
69 changes: 69 additions & 0 deletions v1.1/manage-long-running-queries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
---
title: Manage Long-Running Queries
summary: Learn how to identify and cancel long-running queries.
toc: false
---

This page shows you how to identify and, if necessary, cancel SQL queries that are taking longer than expected to process.

{{site.data.alerts.callout_info}}Schema changes (statements beginning with <code>ALTER</code>) cannot currently be cancelled. However, to monitor the progress of schema changes, you can use <a href="show-jobs.html"><code>SHOW JOBS</code></a>.{{site.data.alerts.end}}

<div id="toc"></div>

## Identify Long-Running Queries

Use the [`SHOW QUERIES`](show-queries.html) statement to list details about currently active SQL queries, including each query's `start` timestamp:

~~~ sql
> SHOW QUERIES;
~~~

~~~
+----------------------------------+---------+----------+----------------------------------+-------------------------------------------+---------------------+------------------+-------------+-----------+
| query_id | node_id | username | start | query | client_address | application_name | distributed | phase |
+----------------------------------+---------+----------+----------------------------------+-------------------------------------------+---------------------+------------------+-------------+-----------+
| 14db657443230c3e0000000000000001 | 1 | root | 2017-08-16 18:00:50.675151+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.56:54119 | test_app | false | executing |
| 14db657443b68c7d0000000000000001 | 1 | root | 2017-08-16 18:00:50.684818+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.56:54123 | test_app | false | executing |
| 14db65744382c2340000000000000001 | 1 | root | 2017-08-16 18:00:50.681431+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.56:54103 | test_app | false | executing |
| 14db657443c9dc660000000000000001 | 1 | root | 2017-08-16 18:00:50.686083+00:00 | SHOW CLUSTER QUERIES | 192.168.12.56:54108 | | NULL | preparing |
| 14db657443e30a850000000000000003 | 3 | root | 2017-08-16 18:00:50.68774+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.58:54118 | test_app | false | executing |
| 14db6574439f477d0000000000000003 | 3 | root | 2017-08-16 18:00:50.6833+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.58:54122 | test_app | false | executing |
| 14db6574435817d20000000000000002 | 2 | root | 2017-08-16 18:00:50.678629+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.57:54121 | test_app | false | executing |
| 14db6574433c621f0000000000000002 | 2 | root | 2017-08-16 18:00:50.676813+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.57:54124 | test_app | false | executing |
| 14db6574436f71d50000000000000002 | 2 | root | 2017-08-16 18:00:50.680165+00:00 | UPSERT INTO test.kv(k, v) VALUES ($1, $2) | 192.168.12.57:54117 | test_app | false | executing |
+----------------------------------+---------+----------+----------------------------------+-------------------------------------------+---------------------+------------------+-------------+-----------+
(9 rows)
~~~

You can also filter for queries that have been running for a certain amount of time. For example, to find queries that have been running for more than 3 hours, you would run the following:

~~~ sql
> SELECT * FROM [SHOW CLUSTER QUERIES]
WHERE start < (now() - INTERVAL '3 hours');
~~~

## Cancel Long-Running Queries

Once you've identified a long-running query via [`SHOW QUERIES`](show-queries.html), note the `query_id` and use it with the [`CANCEL QUERY`](cancel-query.html) statement:

~~~ sql
> CANCEL QUERY '14dacc1f9a781e3d0000000000000001';
~~~

When a query is successfully cancelled, CockoachDB sends a `query execution canceled` error to the client that issued the query.

- If the canceled query was a single, standalone statement, no further action is required by the client.
- If the canceled query was part of a larger, multi-statement [transaction](transactions.html), the client should then issue a [`ROLLBACK`](rollback-transaction.html) statement.

## Improve Query Performance

After cancelling a long-running query, use the [`EXPLAIN`](explain.html) statement to examine it. It's possible that the query was slow because it performs a full-table scan. In these cases, you can likely improve the query's performance by [adding an index](create-index.html).

*(More guidance around query performance optimization forthcoming.)*

## See Also

- [`SHOW QUERIES`](show-queries.html)
- [`CANCEL QUERY`](cancel-query.html)
- [`EXPLAIN`](explain.html)
- [Query Behavior Troubleshooting](query-behavior-troubleshooting.html)
8 changes: 4 additions & 4 deletions v1.1/query-behavior-troubleshooting.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Query Behavior Troubleshooting
summary: Learn how to troubleshoot issues with specific queries with CockroachDB
summary: Learn how to troubleshoot issues with specific queries with CockroachDB
toc: false
---

Expand All @@ -14,15 +14,15 @@ If your queries return unexpected results, there are several possibilities:

- You’ve encountered a [known limitation](https://github.com/cockroachdb/cockroach/issues?q=is%3Aopen+is%3Aissue+label%3Aknown-limitation), [UX surprise](https://github.com/cockroachdb/cockroach/issues?utf8=%E2%9C%93&q=is%3Aopen%20is%3Aissue%20label%3Aux-surprise) or other problem with [SQL semantics](https://github.com/cockroachdb/cockroach/issues?utf8=%E2%9C%93&q=is%3Aopen%20is%3Aissue%20label%3Asql-semantics). Feel free to leave a comment on the existing issue indicating that you’ve encountered a problem as well.
- Your application has a bug. It's always worthwhile to check and double-check your application’s logic before filing an issue. That said, you can always [reach out for support](support-resources.html).
- CockroachDB has a bug. Please [file an issue](file-an-issue.html).
- CockroachDB has a bug. Please [file an issue](file-an-issue.html).

## Performance Issues

If your queries take much longer than expected, there are a few things you can check into:
If queries are taking longer than expected to process, there are a few things you can check into:

- Review your deployment's monitoring. General network latency or partitioning events can affect query response times.

- [Log and debug long-running queries/transactions](debug-and-error-logs.html#log-queries).
- [Identify and cancel long-running queries](manage-long-running-queries.html).

If you're still unable to determine why the query executes slowly, please [file an issue](file-an-issue.html).

Expand Down

0 comments on commit fd31a6c

Please sign in to comment.