Merge cockroachdb#99856 cockroachdb#100200 cockroachdb#100742 cockroa…

…chdb#100911 cockroachdb#101095 cockroachdb#101105 cockroachdb#101169

99856: roachtest: update 22.2 predecessor to 22.1.18 r=renatolabs a=cameronnunez

Part of: https://cockroachlabs.atlassian.net/browse/REL-309

Release note: None

100200: pgwire: add server.cockroach_cloud.max_client_connections_per_gateway cluster setting r=alyshanjahani-crl a=alyshanjahani-crl

This setting specifies a maximum number of external connections that a server can have open at any given time. External is defined as connections that are not root connections. In the future, we can have the definition of external be a configurable list of users.
<0 - Connections are unlimited (existing behavior)
=0 - External connections are disabled
>0 - External connections are limited
If a new connection would exceed this limit, an error message is returned as postgres: "cluster connections are limited" with the 53300 error code that corresponds to "too many connections".

This functionality is required for serverless.
Part of: https://cockroachlabs.atlassian.net/browse/CC-9288

Release note: None

100742: ownership of Jobs to DR team r=shermanCRL a=shermanCRL

Epic: CRDB-26380

100911: cli: add system.job_info to debug zip r=adityamaru a=adityamaru

We do not dump the value column when
collecting a redacted zip as it might contain
sensitive data.

Release note: None
Informs: cockroachdb#98829

101095: add DR, CDC, Jobs labels to test failures r=shermanCRL a=shermanCRL

Ensure that test failures via the `cockroach-teamcity` bot labels DR, CDC and Jobs teams

Epic: none
Jira: none


101105: sql/schemachanger: relax rules for setting constraint names r=fqazi a=fqazi

Previously, the rules for adding and removing constraint names were strict, which could lead to mixed version compatibility issues. Specifically, plans on the 23.1 branch could have names set in later stages violating a rule that said a constraint should be public/non-public with the name. To address this, this patch will relax these rules to have Precedence instead of SameStagePrecedence

Fixes: cockroachdb#100908

Release note: None

101169: roachtest: prevent shared mutable state across restore roachtest runs r=benbardin a=msbutler

Previously, all `restore/*` roachtests run with `--count` would provide incomprehensible results because multiple roachtest runs of the same test would override each other's state. Specifically, the latest call of `test_spec.Run()`, would override the `test.Test` harness, and `syncedCluster.Cluster` used by all other tests with the same registration.

This patch fixes this problem by moving all fields in `restoreSpecs` that are set during test execution (i.e. a `test_spec.Run` call), to a new `restoreDriver` struct. Now, `restoreSpecs` gets defined during test registration and is shared across test runs, while `restoreDriver` gets set within a test run.

Epic: none

Release note: None

Co-authored-by: Cameron Nunez <[email protected]>
Co-authored-by: Alyshan Jahani <[email protected]>
Co-authored-by: Matt Sherman <[email protected]>
Co-authored-by: adityamaru <[email protected]>
Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Michael Butler <[email protected]>

.github/CODEOWNERS

@@ -58,7 +58,7 @@
 /pkg/sql/row*                @cockroachdb/sql-queries
 /pkg/sql/control_job*        @cockroachdb/sql-queries @cockroachdb/jobs-prs
 /pkg/sql/job_exec_context*   @cockroachdb/sql-queries @cockroachdb/jobs-prs
-/pkg/sql/delegate/*job*.go   @cockroachdb/jobs-prs
+/pkg/sql/delegate/*job*.go   @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 
 /pkg/sql/importer/           @cockroachdb/sql-queries
 /pkg/ccl/importerccl/        @cockroachdb/sql-queries
@@ -114,7 +114,7 @@
 /pkg/cli/demo*.go            @cockroachdb/sql-sessions @cockroachdb/server-prs @cockroachdb/cli-prs
 /pkg/cli/democluster         @cockroachdb/sql-sessions @cockroachdb/server-prs @cockroachdb/cli-prs
 /pkg/cli/debug*.go           @cockroachdb/kv-prs         @cockroachdb/cli-prs
-/pkg/cli/debug_job_trace*.go @cockroachdb/jobs-prs
+/pkg/cli/debug_job_trace*.go @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 /pkg/cli/doctor*.go          @cockroachdb/sql-schema     @cockroachdb/cli-prs
 /pkg/cli/import_test.go      @cockroachdb/sql-sessions @cockroachdb/cli-prs
 /pkg/cli/sql*.go             @cockroachdb/sql-sessions @cockroachdb/cli-prs
@@ -176,7 +176,7 @@
 /pkg/server/user*.go                     @cockroachdb/obs-inf-prs @cockroachdb/server-prs @cockroachdb/prodsec
 
 
-/pkg/ccl/jobsccl/            @cockroachdb/jobs-prs
+/pkg/ccl/jobsccl/            @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 /pkg/ccl/changefeedccl/      @cockroachdb/cdc-prs
 
 /pkg/ccl/streamingccl/       @cockroachdb/disaster-recovery
@@ -401,7 +401,7 @@
 /pkg/internal/rsg/           @cockroachdb/sql-queries
 /pkg/internal/sqlsmith/      @cockroachdb/sql-queries
 /pkg/internal/team/          @cockroachdb/test-eng
-/pkg/jobs/                   @cockroachdb/jobs-prs
+/pkg/jobs/                   @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 /pkg/keys/                   @cockroachdb/kv-prs
 /pkg/keysbase/               @cockroachdb/kv-prs
 # Don't ping KV on updates to reserved descriptor IDs and such.
@@ -426,7 +426,7 @@
 /pkg/roachprod/              @cockroachdb/test-eng
 /pkg/rpc/                    @cockroachdb/kv-prs
 /pkg/rpc/auth.go             @cockroachdb/server-prs @cockroachdb/kv-prs @cockroachdb/prodsec
-/pkg/scheduledjobs/          @cockroachdb/jobs-prs
+/pkg/scheduledjobs/          @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 /pkg/security/               @cockroachdb/server-prs @cockroachdb/prodsec
 /pkg/security/clientsecopts/ @cockroachdb/server-prs @cockroachdb/sql-sessions @cockroachdb/prodsec
 #!/pkg/settings/             @cockroachdb/unowned
@@ -435,7 +435,7 @@
 #!/pkg/testutils/              @cockroachdb/test-eng-noreview
 /pkg/testutils/reduce/       @cockroachdb/sql-queries
 /pkg/testutils/sqlutils/     @cockroachdb/sql-queries
-/pkg/testutils/jobutils/     @cockroachdb/jobs-prs
+/pkg/testutils/jobutils/     @cockroachdb/jobs-prs @cockroachdb/disaster-recovery
 /pkg/ts/                     @cockroachdb/kv-prs
 /pkg/ts/catalog/             @cockroachdb/obs-inf-prs
 #!/pkg/util/                 @cockroachdb/unowned

TEAMS.yaml

@@ -63,10 +63,13 @@ cockroachdb/prodsec:
   triage_column_id: 0 # TODO as well
 cockroachdb/disaster-recovery:
   triage_column_id: 3097123
+  label: T-disaster-recovery
 cockroachdb/cdc:
   aliases:
     cockroachdb/cdc-prs: other
-  triage_column_id: 3570120
+  # CDC team uses GH projects v2, which doesn't have a REST API, so no triage column ID
+  # see .github/workflows/add-issues-to-project.yml
+  label: T-cdc
 cockroachdb/server:
   aliases:
     cockroachdb/cli-prs: other
@@ -87,6 +90,7 @@ cockroachdb/jobs:
   aliases:
     cockroachdb/jobs-prs: other
   triage_column_id: 16360666
+  label: T-jobs
 cockroachdb/cloud-identity:
   triage_column_id: 18588697
 cockroachdb/unowned:

pkg/cli/testdata/declarative-rules/deprules

@@ -295,7 +295,7 @@ deprules
     - joinTargetNode($next, $next-Target, $next-Node)
 - name: Constraint should be hidden before name
   from: constraint-name-Node
-  kind: SameStagePrecedence
+  kind: Precedence
   to: constraint-Node
   query:
     - $constraint-name[Type] = '*scpb.ConstraintWithoutIndexName'
@@ -308,7 +308,7 @@ deprules
     - joinTargetNode($constraint, $constraint-Target, $constraint-Node)
 - name: Constraint should be hidden before name
   from: constraint-name-Node
-  kind: SameStagePrecedence
+  kind: Precedence
   to: constraint-Node
   query:
     - $constraint-name[Type] = '*scpb.ConstraintWithoutIndexName'
@@ -321,7 +321,7 @@ deprules
     - joinTargetNode($constraint, $constraint-Target, $constraint-Node)
 - name: Constraint should be hidden before name
   from: constraint-name-Node
-  kind: SameStagePrecedence
+  kind: Precedence
   to: constraint-Node
   query:
     - $constraint-name[Type] = '*scpb.ConstraintWithoutIndexName'
@@ -335,7 +335,7 @@ deprules
     - joinTargetNode($constraint, $constraint-Target, $constraint-Node)
 - name: Constraint should be hidden before name
   from: constraint-name-Node
-  kind: SameStagePrecedence
+  kind: Precedence
   to: constraint-Node
   query:
     - $constraint-name[Type] = '*scpb.ConstraintWithoutIndexName'
@@ -3402,7 +3402,7 @@ deprules
     - joinTargetNode($dependent, $dependent-Target, $dependent-Node)
 - name: simple constraint visible before name
   from: simple-constraint-Node
-  kind: SameStagePrecedence
+  kind: Precedence
   to: constraint-name-Node
   query:
     - $simple-constraint[Type] IN ['*scpb.UniqueWithoutIndexConstraint', '*scpb.UniqueWithoutIndexConstraintUnvalidated', '*scpb.CheckConstraint', '*scpb.CheckConstraintUnvalidated', '*scpb.ForeignKeyConstraint', '*scpb.ForeignKeyConstraintUnvalidated', '*scpb.ColumnNotNull']

pkg/cli/testdata/zip/partial1

@@ -42,6 +42,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0s /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done

pkg/cli/testdata/zip/partial1_excluded

@@ -42,6 +42,7 @@ debug zip /dev/null --concurrency=1 --exclude-nodes=2 --cpu-profile-duration=0
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done

pkg/cli/testdata/zip/partial2

@@ -42,6 +42,7 @@ debug zip --concurrency=1 --cpu-profile-duration=0 /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done

pkg/cli/testdata/zip/testzip

@@ -42,6 +42,7 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done

pkg/cli/testdata/zip/testzip_concurrent

@@ -146,6 +146,9 @@ zip
 [cluster] retrieving SQL data for system.external_connections...
 [cluster] retrieving SQL data for system.external_connections: done
 [cluster] retrieving SQL data for system.external_connections: writing output: debug/system.external_connections.txt...
+[cluster] retrieving SQL data for system.job_info...
+[cluster] retrieving SQL data for system.job_info: done
+[cluster] retrieving SQL data for system.job_info: writing output: debug/system.job_info.txt...
 [cluster] retrieving SQL data for system.jobs...
 [cluster] retrieving SQL data for system.jobs: done
 [cluster] retrieving SQL data for system.jobs: writing output: debug/system.jobs.txt...

pkg/cli/testdata/zip/testzip_shared_process_tenant

@@ -42,6 +42,7 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done
@@ -231,6 +232,7 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/tenants/test-tenant/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/tenants/test-tenant/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/tenants/test-tenant/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/tenants/test-tenant/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/tenants/test-tenant/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/tenants/test-tenant/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/tenants/test-tenant/system.locations.txt... done

pkg/cli/testdata/zip/testzip_tenant_separate_process

@@ -50,6 +50,7 @@ debug zip --concurrency=1 --cpu-profile-duration=1s /dev/null
 [cluster] retrieving SQL data for system.descriptor... writing output: debug/system.descriptor.txt... done
 [cluster] retrieving SQL data for system.eventlog... writing output: debug/system.eventlog.txt... done
 [cluster] retrieving SQL data for system.external_connections... writing output: debug/system.external_connections.txt... done
+[cluster] retrieving SQL data for system.job_info... writing output: debug/system.job_info.txt... done
 [cluster] retrieving SQL data for system.jobs... writing output: debug/system.jobs.txt... done
 [cluster] retrieving SQL data for system.lease... writing output: debug/system.lease.txt... done
 [cluster] retrieving SQL data for system.locations... writing output: debug/system.locations.txt... done

pkg/cli/zip_table_registry.go

@@ -994,6 +994,18 @@ var zipSystemTables = DebugZipTableRegistry{
 			to_hex(progress) AS hex_progress
 			FROM system.jobs`,
 	},
+	"system.job_info": {
+		// `value` column may contain customer info, such as URI params
+		// containing access keys, encryption salts, etc.
+		customQueryUnredacted: `SELECT *,
+			to_hex(value) AS hex_value
+			FROM system.job_info`,
+		customQueryRedacted: `SELECT job_id,
+			info_key,
+			written,
+			'redacted' AS value
+			FROM system.job_info`,
+	},
 	"system.lease": {
 		nonSensitiveCols: NonSensitiveColumns{
 			`"descID"`,