Skip to content

Commit

Permalink
Adding known limitation for inverted index scans. (#10967)
Browse files Browse the repository at this point in the history
  • Loading branch information
Ian Evans authored Jul 27, 2021
1 parent d5f6315 commit 32394ed
Show file tree
Hide file tree
Showing 2 changed files with 37 additions and 1 deletion.
20 changes: 19 additions & 1 deletion v20.2/known-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ Once restarted, monitor the Replica Quiescence graph on the [**Replication Dashb
Resolved as of [v20.2.4](../releases/v20.2.4.html). See [#57789](https://github.com/cockroachdb/cockroach/pull/57789).
{{site.data.alerts.end}}

When a node is offline, the [Raft logs](architecture/replication-layer.html#raft-logs) for the ranges on the node get truncated. When the node comes back online, it therefore often needs [Raft snapshots](architecture/replication-layer.html#snapshots) to get many of its ranges back up-to-date. While in this state, requests to a range will hang until its snapshot has been applied, which can take a long time.
When a node is offline, the [Raft logs](architecture/replication-layer.html#raft-logs) for the ranges on the node get truncated. When the node comes back online, it therefore often needs [Raft snapshots](architecture/replication-layer.html#snapshots) to get many of its ranges back up-to-date. While in this state, requests to a range will hang until its snapshot has been applied, which can take a long time.

To work around this limitation, you can adjust the `kv.snapshot_recovery.max_rate` [cluster setting](cluster-settings.html) to temporarily relax the throughput rate limiting applied to snapshots. For example, changing the rate limiting from the default 8 MB/s, at which 1 GB of snapshots takes at least 2 minutes, to 64 MB/s can result in an 8x speedup in snapshot transfers and, therefore, a much shorter interruption of requests to an impacted node:

Expand Down Expand Up @@ -537,3 +537,21 @@ If the execution of a [join](joins.html) query exceeds the limit set for memory-
CockroachDB does not support partitioning inverted indexes, including [spatial indexes](spatial-indexes.html).

[Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/43643)

### Inverted index scans can't be generated for some statement filters

CockroachDB cannot generate [inverted index](inverted-indexes.html) scans for statements with filters that have both JSON fetch values and containment operators. For example the following statement won't be index-accelerated:

~~~ sql
SELECT * FROM mytable WHERE j->'a' @> '{"b": "c"}';
~~~

CockroachDB v20.1 and earlier would generate index scans for these filters, though it is not recommended as the normalization rules used to convert the filters into JSON containment expressions would sometimes produce inequivalent expressions.

The workaround is to rewrite the statement filters to avoid using both JSON fetch values and containment operators. The following statement is index-accelerated and equivalent to the non-accelerated statement above:

~~~ sql
SELECT * FROM mytable WHERE j @> '{"a": {"b": "c"}}'
~~~

[Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/55318)
18 changes: 18 additions & 0 deletions v21.1/known-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -613,3 +613,21 @@ If the execution of a [join](joins.html) query exceeds the limit set for memory-
### Using interleaved tables in backups

{% include {{ page.version.version }}/known-limitations/backup-interleaved.md %}

### Inverted index scans can't be generated for some statement filters

CockroachDB cannot generate [inverted index](inverted-indexes.html) scans for statements with filters that have both JSON fetch values and containment operators. For example the following statement won't be index-accelerated:

~~~ sql
SELECT * FROM mytable WHERE j->'a' @> '{"b": "c"}';
~~~

CockroachDB v20.1 and earlier would generate index scans for these filters, though it is not recommended as the normalization rules used to convert the filters into JSON containment expressions would sometimes produce inequivalent expressions.

The workaround is to rewrite the statement filters to avoid using both JSON fetch values and containment operators. The following statement is index-accelerated and equivalent to the non-accelerated statement above:

~~~ sql
SELECT * FROM mytable WHERE j @> '{"a": {"b": "c"}}'
~~~

[Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/55318)

0 comments on commit 32394ed

Please sign in to comment.