explain: Clarify meaning of distribution full vs local

[sheaffej]

It would be helpful to clarify the meaning distribution in explain plans. The possible values appear to be full and local.

On the page: https://www.cockroachlabs.com/docs/v20.2/explain#default-query-plans

It states:

distribution:full
The query plan will be distributed across all nodes on a distributed cluster.

However, it does not explain what local means. And it is not clear what distribution is referring to.

Reference internal Slack:
https://cockroachlabs.slack.com/archives/CHKQGKYEM/p1616110866018700
and follow-on
https://cockroachlabs.slack.com/archives/CHKQGKYEM/p1616110946019100

Summarizing the slack conversation.

distribution: full
Means that the planner chose a distributed execution plan, where execution of the query is performed by multiple nodes in parallel. Such as a distributed join or aggregation, where each node performs a portion of the work in parallel, and then final or downstream results are returned by the gateway node.

distribution: local
Means the planner chose a local execution plan, where execution of the query is performed only on the local node (aka gateway node). It is important to note that rows may be fetched by one or more remote nodes, but the processing of those rows is performed only by the local node.

Therefore "distribution" refers more to the style of processing of the records during query execution (e.g. distributed processing, or local processing), vs the fetching of the rows that will be processed by the query.

Examples:

distribution: local

[email protected]:59856/movr> explain 
select u.name, u.id
from users u
where u.city = 'Amsterdam' and u.id = 'b3333333-3333-4000-8000-000000000023';
  tree |        field        |                                                 description
-------+---------------------+--------------------------------------------------------------------------------------------------------------
       | distribution        | local
       | vectorized          | false
  scan |                     |
       | estimated row count | 1
       | table               | users@primary
       | spans               | [/'Amsterdam'/'b3333333-3333-4000-8000-000000000023' - /'Amsterdam'/'b3333333-3333-4000-8000-000000000023']
(6 rows)

Notice that all query processing occurs on node 1.

distribution: full

[email protected]:59856/movr> explain 
select u.city, count(*) as num
from users u
where u.city = 'Amsterdam' group by u.city;
    tree    |        field        |          description
------------+---------------------+--------------------------------
            | distribution        | full
            | vectorized          | false
  group     |                     |
   └── scan |                     |
            | estimated row count | 0
            | table               | users@primary
            | spans               | [/'Amsterdam' - /'Amsterdam']
(7 rows)

Notice that query processing involves node 1 and node5.

Therefore it is important to note the full does not mean that processing happens on ALL nodes, but that it happens on multiple nodes simultaneously.