Merge #59566 #60753 #60772 #60796 #60818

59566:  kvserver: introduce a Raft-based transport for closedts r=andreimatei a=andreimatei

This patch introduces a replacement for the existing closed timestamp
mechanism / transport. The new mechanism is gated by a cluster version.

Raft commands now carry increasing closed timestamps generated by the
propBuf by using the recent request Tracker for synchronizing with
in-flight requests (i.e. not closing timestamps below them).
Raft commands get a closed ts field, and the range state gets the field
as well.

The propBuf pays attention to the range's closed timestamp policy for
deciding whether to close lagging or leading timestamps.

Release note: None

60753: kv: add TestStoreRangeSplitAndMergeWithGlobalReads r=nvanbenschoten a=nvanbenschoten

Made possible by #60567.

This commit adds a new test called TestStoreRangeSplitAndMergeWithGlobalReads
that tests that a range configured to serve global reads can be split and
merged. In essence, this tests whether the split and merge transactions can
handle having their timestamp bumped by the closed timestamp on the ranges
they're operating on.

The test revealed that range merges did have issues in these cases, because
SubsumeRequests assumed that the intent on the RHS's local descriptor was below
the node's HLC clock. This is no longer always true, so we now perform the
inconsistent scan at hlc.MaxTimestamp, just like QueryIntent requests do.

60772: sql: add parse_timestamp builtin r=RaduBerinde a=RaduBerinde

Add a builtin that can be used to parse timestamp strings. This is
like a cast, but it does not accept relative timestamps so it can be
immutable.

Only immutable expressions are allowed in computed column expressions
or partial index predicates; unlike casts, the new function can be
used in such expressions.

Fixes #60578.

Release notes (sql change): A new parse_timestamp function can be used
to parse absolute timestamp strings in computed column expressions or
partial index predicates.

60796: geo: replace GEOS with new WKT parser for EWKT parsing r=otan a=andyyang890

Previously, the `parseEWKT` function used GEOS to parse WKT strings.
This was inadequate because GEOS has issues with parsing Z and M
dimensions. To address this, a new WKT parser was implemented with
goyacc and this patch integrates it into CockroachDB.

Refs: #53091

Release note: None

60818: opt: add partition info to the opt catalog r=rytaft a=rytaft

Prior to this commit, the opt catalog did not include zone information
or prefixes specific to each partition of an index. This commit adds this
information since it will be necessary to support locality optimized
search in a future commit.

Informs #55185

Release note: None

Co-authored-by: Andrei Matei <[email protected]>
Co-authored-by: Nathan VanBenschoten <[email protected]>
Co-authored-by: Radu Berinde <[email protected]>
Co-authored-by: Andy Yang <[email protected]>
Co-authored-by: Rebecca Taft <[email protected]>

docs/generated/sql/functions.md

@@ -2371,6 +2371,8 @@ to recreate a database.’</p>
 </span></td></tr>
 <tr><td><a name="overlay"></a><code>overlay(input: <a href="string.html">string</a>, overlay_val: <a href="string.html">string</a>, start_pos: <a href="int.html">int</a>, end_pos: <a href="int.html">int</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Deletes the characters in <code>input</code> between <code>start_pos</code> and <code>end_pos</code> (count starts at 1), and then insert <code>overlay_val</code> at <code>start_pos</code>.</p>
 </span></td></tr>
+<tr><td><a name="parse_timestamp"></a><code>parse_timestamp(string: <a href="string.html">string</a>) &rarr; <a href="timestamp.html">timestamp</a></code></td><td><span class="funcdesc"><p>Convert a string containing an absolute timestamp to the corresponding timestamp.</p>
+</span></td></tr>
 <tr><td><a name="pg_collation_for"></a><code>pg_collation_for(str: anyelement) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Returns the collation of the argument</p>
 </span></td></tr>
 <tr><td><a name="quote_ident"></a><code>quote_ident(val: <a href="string.html">string</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Return <code>val</code> suitably quoted to serve as identifier in a SQL statement.</p>

pkg/ccl/logictestccl/testdata/logic_test/partitioning

@@ -428,9 +428,13 @@ TABLE ok1
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           ├── (1)
-           └── (2)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           └── p2
+                └── partition by list prefixes
+                     └── (2)
 scan ok1
 
 statement ok
@@ -472,9 +476,13 @@ TABLE ok2
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           ├── (1)
-           └── (2)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           └── p2
+                └── partition by list prefixes
+                     └── (2)
 scan ok2
 
 statement ok
@@ -516,8 +524,13 @@ TABLE ok3
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           └── (1)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           └── p2
+                └── partition by list prefixes
+                     └── ()
 scan ok3
 
 statement ok
@@ -563,10 +576,19 @@ TABLE ok4
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           ├── (1, 1)
-           ├── (1)
-           └── (2, 3)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1, 1)
+           ├── p2
+           │    └── partition by list prefixes
+           │         └── (1)
+           ├── p3
+           │    └── partition by list prefixes
+           │         └── (2, 3)
+           └── p4
+                └── partition by list prefixes
+                     └── ()
 scan ok4
 
 statement ok
@@ -603,9 +625,16 @@ TABLE ok5
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           ├── (1)
-           └── (2)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           ├── p2
+           │    └── partition by list prefixes
+           │         └── (2)
+           └── p3
+                └── partition by list prefixes
+                     └── ()
 scan ok5
 
 query TT
@@ -897,9 +926,13 @@ TABLE ok11
       ├── a int not null
       ├── b int not null
       ├── c int not null
-      └── partition by list prefixes
-           ├── (1)
-           └── (6)
+      └── partitions
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           └── p2
+                └── partition by list prefixes
+                     └── (6)
 scan ok11
 
 statement ok
@@ -943,10 +976,16 @@ TABLE ok12
  └── PRIMARY INDEX primary
       ├── a int not null
       ├── b int not null
-      └── partition by list prefixes
-           ├── (NULL)
-           ├── (1)
-           └── (2)
+      └── partitions
+           ├── pu
+           │    └── partition by list prefixes
+           │         └── (NULL)
+           ├── p1
+           │    └── partition by list prefixes
+           │         └── (1)
+           └── p2
+                └── partition by list prefixes
+                     └── (2)
 scan ok12
 
 # Verify that creating a partition that includes NULL does not change the

pkg/ccl/logictestccl/testdata/logic_test/partitioning_implicit

@@ -149,6 +149,73 @@ CREATE TABLE public.t (
 )
 -- Warning: Partitioned table with no zone configurations.
 
+query T
+EXPLAIN (OPT, CATALOG) SELECT * FROM t
+----
+TABLE t
+ ├── pk int not null
+ ├── a int not null
+ ├── b int
+ ├── c int
+ ├── d int
+ ├── j jsonb
+ ├── crdb_internal_mvcc_timestamp decimal [hidden] [system]
+ ├── tableoid oid [hidden] [system]
+ ├── j_inverted_key bytes not null [virtual-inverted]
+ ├── FAMILY fam_0_pk_a_b_c_d_j (pk, a, b, c, d, j)
+ ├── PRIMARY INDEX primary
+ │    ├── a int not null (implicit)
+ │    ├── pk int not null
+ │    └── partitions
+ │         └── pk_implicit
+ │              └── partition by list prefixes
+ │                   └── (1)
+ ├── INDEX t_b_idx
+ │    ├── a int not null (implicit)
+ │    ├── b int
+ │    ├── pk int not null
+ │    └── partitions
+ │         └── b_implicit
+ │              └── partition by list prefixes
+ │                   └── (2)
+ ├── UNIQUE INDEX t_c_key
+ │    ├── a int not null (implicit)
+ │    ├── c int
+ │    ├── pk int not null (storing)
+ │    └── partitions
+ │         └── c_implicit
+ │              └── partition by list prefixes
+ │                   └── (3)
+ ├── INDEX t_a_b_c_idx
+ │    ├── d int (implicit)
+ │    ├── a int not null
+ │    ├── b int
+ │    ├── c int
+ │    ├── pk int not null
+ │    └── partitions
+ │         └── a_b_c_implicit
+ │              └── partition by list prefixes
+ │                   └── (4)
+ ├── INVERTED INDEX t_j_idx
+ │    ├── a int not null (implicit)
+ │    ├── j_inverted_key bytes not null [virtual-inverted]
+ │    ├── pk int not null
+ │    └── partitions
+ │         └── j_implicit
+ │              └── partition by list prefixes
+ │                   └── (5)
+ ├── INDEX new_idx
+ │    ├── a int not null (implicit)
+ │    ├── d int
+ │    ├── pk int not null
+ │    └── partitions
+ │         └── d_implicit
+ │              └── partition by list prefixes
+ │                   └── (1)
+ ├── UNIQUE WITHOUT INDEX (pk)
+ └── UNIQUE WITHOUT INDEX (c)
+scan t
+
 statement ok
 INSERT INTO t VALUES (1, 2, 3, 4, 5)