Skip to content

Commit

Permalink
sql: add trie tree based workload index recommendations
Browse files Browse the repository at this point in the history
This commit adds the trie and the logic for getting the workload index
recommendations. In addition, it fills the gap between built-in functions and
backend implementation for workload index recommendations.  The whole process
consists of collecting candidates and finding representative indexes.

All the index recommendations in the table `system.statement_statistics`
(satisfying some time requirement) will be collected as the candidates and then
inserted to the trie.

The trie is designed for all the indexes of one table. The indexed columns will
be regarded as the key to insert into the tree in their original orders. The
storing part will be attached to the node after the insertion of indexed
columns.

The general idea of finding representative indexes is to use all the indexes
represented by the leaf nodes. One optimization is to remove the storings that
are covered by some leaf nodes. Next, we will push down all the storings
attached to the internal nodes to the shallowest leaf nodes (You can find the
reasons in RFC). Finally, all the indexes represented by the leaf nodes will be
returned.

As for the `DROP INDEX`, since we collect all the indexes represented by the
leaf nodes (a superset of dropped indexes), so we can directly drop all of
them.

Release note (sql change): new builtin functions `workload_index_recs()` and
`workload_index_recs(timestamptz)`, return workload level index recommendations
(columns of string, each string represent an index recommendation) from
statement level index recommendations (as candidates) in
`system.statement_statistics`. If the timestamptz is given, it will only
consider those candidates who is generated after that timestampsz.
  • Loading branch information
qiyanghe1998 committed Aug 10, 2023
1 parent eff0185 commit 0f2b7e6
Show file tree
Hide file tree
Showing 10 changed files with 1,280 additions and 58 deletions.
4 changes: 0 additions & 4 deletions docs/generated/sql/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -1386,11 +1386,7 @@ the locality flag on node startup. Returns an error if no region is set.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="workload_index_recs"></a><code>workload_index_recs() &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="workload_index_recs"></a><code>workload_index_recs(budget: <a href="string.html">string</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="workload_index_recs"></a><code>workload_index_recs(timestamptz: <a href="timestamp.html">timestamptz</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="workload_index_recs"></a><code>workload_index_recs(timestamptz: <a href="timestamp.html">timestamptz</a>, budget: <a href="string.html">string</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns set of index recommendations</p>
</span></td><td>Immutable</td></tr></tbody>
</table>

Expand Down
3 changes: 3 additions & 0 deletions pkg/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,7 @@ ALL_TESTS = [
"//pkg/sql/opt/testutils/opttester:opttester_test",
"//pkg/sql/opt/testutils/testcat:testcat_test",
"//pkg/sql/opt/testutils:testutils_test",
"//pkg/sql/opt/workloadindexrec:workloadindexrec_test",
"//pkg/sql/opt/xform:xform_test",
"//pkg/sql/opt:opt_test",
"//pkg/sql/parser:parser_disallowed_imports_test",
Expand Down Expand Up @@ -1892,6 +1893,8 @@ GO_TARGETS = [
"//pkg/sql/opt/testutils/testexpr:testexpr",
"//pkg/sql/opt/testutils:testutils",
"//pkg/sql/opt/testutils:testutils_test",
"//pkg/sql/opt/workloadindexrec:workloadindexrec",
"//pkg/sql/opt/workloadindexrec:workloadindexrec_test",
"//pkg/sql/opt/xform:xform",
"//pkg/sql/opt/xform:xform_test",
"//pkg/sql/opt:opt",
Expand Down
Loading

0 comments on commit 0f2b7e6

Please sign in to comment.