Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
57183: importccl,sql: support importing non-public schemas from pgdump r=adityamaru a=adityamaru

Previously, a PGDUMP import did not support any non-public schema
statements. Now that CRDB has user defined schemas, bundle format
IMPORTs need to be taught how to parse, create and cleanup schema
related PGDUMP operations.

Note this PR only adds support for `CREATE SCHEMA` and usage of the
schema in `CREATE TABLE/SEQUENCE` PGDUMP statements. `ALTER SCHEMA`
statements are ignored and support might be added in a follow up.

Release justification: low risk, high reward. Import PGDUMP does not support user
defined schemas.

Release note (sql change): IMPORT PGDUMP can now import dump files with
non-public schemas.

60922: sql,cli: add payloads_for_trace builtin r=knz a=angelapwen

Possibly the final step to #55733. Resolves #58608 😄 

Previously it was quite cumbersome to view all payloads for a given
trace: we needed to join on the `node_inflight_trace_spans` vtable
to filter for span IDs that match a trace ID, then apply the
`payloads_for_span()` builtin to each span ID. This patch adds
syntactic sugar to the above query.

Instead of

```
WITH spans AS (
  SELECT span_id
  FROM crdb_internal.node_inflight_trace_spans
  WHERE trace_id = $TRACE_ID)
) SELECT *
  FROM spans, LATERAL crdb_internal.payloads_for_span(spans.span_id);
```

we can now simply use:
```
crdb_internal.payloads_for_trace($TRACE_ID);
```

and achieve the same result. The patch also adds all payloads for all
long-running spans to the `crdb_internal.node_inflight_trace_spans`
table of the debug.zip file.

Release note (sql change): Add `payloads_for_trace()` builtin so that
all payloads attached to all spans for a given trace ID will be
displayed, utilizing the `crdb_internal.payloads_for_span()`
builtin under the hood. All payloads for long-running spans are also
added to debug.zip in the `crdb_internal.node_inflight_trace_spans`
table dump.

Co-authored-by: Tobias Grieger <[email protected]>

Release justification: This patch is safe for release because it
adds syntactic sugar to an internal observability feature.

61094: kvserver: re-add spuriously removed nil check in `relocateOne` r=aayushshah15 a=aayushshah15

bce8317 introduced a bug by spuriously
removing a nil check over the result of a call to
`allocateTargetFromList`. This commit re-adds the check.

The bug could cause a panic when `AdminRelocateRange` was called by the
`StoreRebalancer` or the `mergeQueue` if one (or more) of the stores
that are supposed to receive a replica for a range become unfit for
receiving the replica (due to balancing reasons / or shifting
constraints) _between when rebalancing decision is made and when it's
executed_.

Resolves #60812

Release justification: fixes bug that causes a panic
Release note: None

61097: opt: use computed columns to improve FDs and remove uniqueness checks r=rytaft a=rytaft

**opt: use computed columns to build functional dependencies**

This commit updates `MakeTableFuncDep` so that it adds equivalencies
or synthesized columns to the table FDs for each of the computed
columns available in the metadata. This will be necessary to support
removing uniqueness checks in some cases in a future commit.

Release justification: This commit is a low risk, high benefit change
to existing functionality.

Release note (performance improvement): The optimizer now infers
additional functional dependencies based on computed columns in tables.
This may enable additional optimizations and lead to better query plans.

**opt: remove uniqueness checks when uniqueness inferred through FDs**

This commit removes uniqueness checks for columns that can be
inferred to be unique through functional dependencies. This is
relevant in particular for `REGIONAL BY ROW` tables with a computed
region column that depends on the primary key. In this case,
uniqueness checks are never needed on the primary key, since
uniqueness is already guaranteed by the primary index.

Fixes #57720

Release justification: This commit is a low-risk, high benefit
update to new functionality.

Release note (performance improvement): Removed uniqueness checks
on the primary key for REGIONAL BY ROW tables with a computed
region column that is a function of the primary key columns.
Uniqueness checks are not necessary in this case since uniqueness
can be suitably guaranteed by the primary index. Removing these
checks improves performance of INSERT, UPDATE, and UPSERT
statements.

61100: diagnostics: lock while populating hardware information r=andy-kimball a=andy-kimball

The shirou/gopsutil/host library that we use to gather hardware information
during diagnostics reporting is not multi-thread safe. As one example, it
lazily initializes a global map the first time the Virtualization function
is called, but takes no lock while doing so. Work around this limitation by
taking our own lock.

This code never triggered race conditions before, but is doing so after recent
changes I made to the diagnostics reporting code. Previously, we were using a
single background goroutine to do both diagnostics reporting and checking for
updates. Now we are doing each of those on different goroutines, which triggers
race detection.

Fixes #61091

Release justification: fixes for high-priority or high-severity bugs in existing
functionality
Release note: None

61105: builtins: implement ST_MakePoint and ST_MakePointM r=otan,rafiss a=andyyang890

This patch implements the geometry builtins `ST_MakePoint`
and `ST_MakePointM`.

Resolves #60857.
Resolves #60858.
Resolves #60859.

Release justification: low-risk update to new functionality
Release note (sql change): The geometry builtins `ST_MakePoint`
and `ST_MakePointM` have been implemented and provide a mechanism
for easily creating new points.

Co-authored-by: Aditya Maru <[email protected]>
Co-authored-by: angelapwen <[email protected]>
Co-authored-by: Aayush Shah <[email protected]>
Co-authored-by: Rebecca Taft <[email protected]>
Co-authored-by: Andrew Kimball <[email protected]>
Co-authored-by: Andy Yang <[email protected]>
  • Loading branch information
7 people committed Feb 25, 2021
7 parents ab60e31 + f36a95a + dddec05 + 72be631 + 1e795d1 + 1aee317 + 392effc commit 4d24f8a
Show file tree
Hide file tree
Showing 57 changed files with 3,303 additions and 1,028 deletions.
8 changes: 8 additions & 0 deletions docs/generated/sql/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -1922,6 +1922,12 @@ calculated, the result is transformed back into a Geography with SRID 4326.</p>
</span></td></tr>
<tr><td><a name="st_makepoint"></a><code>st_makepoint(x: <a href="float.html">float</a>, y: <a href="float.html">float</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Point with the given X and Y coordinates.</p>
</span></td></tr>
<tr><td><a name="st_makepoint"></a><code>st_makepoint(x: <a href="float.html">float</a>, y: <a href="float.html">float</a>, z: <a href="float.html">float</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Point with the given X, Y, and Z coordinates.</p>
</span></td></tr>
<tr><td><a name="st_makepoint"></a><code>st_makepoint(x: <a href="float.html">float</a>, y: <a href="float.html">float</a>, z: <a href="float.html">float</a>, m: <a href="float.html">float</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Point with the given X, Y, Z, and M coordinates.</p>
</span></td></tr>
<tr><td><a name="st_makepointm"></a><code>st_makepointm(x: <a href="float.html">float</a>, y: <a href="float.html">float</a>, m: <a href="float.html">float</a>) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Point with the given X, Y, and M coordinates.</p>
</span></td></tr>
<tr><td><a name="st_makepolygon"></a><code>st_makepolygon(geometry: geometry) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Polygon with the given outer LineString.</p>
</span></td></tr>
<tr><td><a name="st_makepolygon"></a><code>st_makepolygon(outer: geometry, interior: anyelement[]) &rarr; geometry</code></td><td><span class="funcdesc"><p>Returns a new Polygon with the given outer LineString and interior (hole) LineString(s).</p>
Expand Down Expand Up @@ -2647,6 +2653,8 @@ SELECT * FROM crdb_internal.check_consistency(true, ‘\x02’, ‘\x04’)</p>
</span></td></tr>
<tr><td><a name="crdb_internal.payloads_for_span"></a><code>crdb_internal.payloads_for_span(span_id: <a href="int.html">int</a>) &rarr; tuple{string AS payload_type, jsonb AS payload_jsonb}</code></td><td><span class="funcdesc"><p>Returns the payload(s) of the requested span.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.payloads_for_trace"></a><code>crdb_internal.payloads_for_trace(trace_id: <a href="int.html">int</a>) &rarr; tuple{int AS span_id, string AS payload_type, jsonb AS payload_jsonb}</code></td><td><span class="funcdesc"><p>Returns the payload(s) of the requested trace.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.pretty_key"></a><code>crdb_internal.pretty_key(raw_key: <a href="bytes.html">bytes</a>, skip_fields: <a href="int.html">int</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>This function is used only by CockroachDB’s developers for testing purposes.</p>
</span></td></tr>
<tr><td><a name="crdb_internal.range_stats"></a><code>crdb_internal.range_stats(key: <a href="bytes.html">bytes</a>) &rarr; jsonb</code></td><td><span class="funcdesc"><p>This function is used to retrieve range statistics information as a JSON object.</p>
Expand Down
3 changes: 3 additions & 0 deletions pkg/ccl/importccl/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,14 @@ go_library(
"//pkg/settings/cluster",
"//pkg/sql",
"//pkg/sql/catalog",
"//pkg/sql/catalog/catalogkeys",
"//pkg/sql/catalog/catalogkv",
"//pkg/sql/catalog/colinfo",
"//pkg/sql/catalog/dbdesc",
"//pkg/sql/catalog/descpb",
"//pkg/sql/catalog/descs",
"//pkg/sql/catalog/resolver",
"//pkg/sql/catalog/schemadesc",
"//pkg/sql/catalog/schemaexpr",
"//pkg/sql/catalog/tabledesc",
"//pkg/sql/catalog/typedesc",
Expand Down
6 changes: 5 additions & 1 deletion pkg/ccl/importccl/import_processor_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -937,8 +937,12 @@ func newTestSpec(
}
assert.True(t, numCols > 0)

fullTableName := "simple"
if format.Format == roachpb.IOFileFormat_PgDump {
fullTableName = "public.simple"
}
spec.tables = map[string]*execinfrapb.ReadImportDataSpec_ImportTable{
"simple": {Desc: descr.TableDesc(), TargetCols: targetCols[0:numCols]},
fullTableName: {Desc: descr.TableDesc(), TargetCols: targetCols[0:numCols]},
}

for id, path := range inputs {
Expand Down
Loading

0 comments on commit 4d24f8a

Please sign in to comment.