Skip to content

Commit

Permalink
Merge #65003 #65025
Browse files Browse the repository at this point in the history 
65003: serverutils: Add TestTenantID r=[yuzefovich,nihalpednekar a=stevendanna

The certificates used for testing are only valid for a specified set
of tenant IDs. If you happen to copy and paste some Tenant creation
code but then for some reason change the constant, your test will
start failing.  While not too hard debug, it would be nice if test
code didn't have to know about this in most cases.

The new serverutils.TestTenantID() returns the first tenant ID that
the certs will work for. Many callsites that don't care about the
particular ID can use this directly.

Release note: None

65025: docs, server: fix link in logformats.md r=knz a=taroface

Fixed an incorrect link in the generated logformats.md that was causing a build error on cockroachdb/docs#10273

@knz I wasn't able to correctly generate logformats.md with the changes I made to the code. The command `docgen logformats docs/generated/logformats.md` was causing logformats.md to revert to an even earlier version of that doc, and I don't know why. I manually updated logformats.md with the correct URL and have included that file in this PR; not sure if this will work. I apologize if this creates more housekeeping work. 

Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: taroface <[email protected]>
  • Loading branch information
@stevendanna @taroface
3 people committed May 12, 2021
3 parents 83eef6f + 0786274 + 2b0f6af commit e2f1d1a
Show file tree
Hide file tree
Showing 16 changed files with 88 additions and 87 deletions.
78 changes: 39 additions & 39 deletions docs/generated/logformats.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,18 +77,18 @@ Each line of output starts with the following prefix:

Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker

| Field | Description |
|-----------------|---------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (omitted if zero for use by tests). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. |
| line | The line number where the entry originated. |
| marker | Redactability marker ` + redactableIndicator + ` (see below for details). |
| Field | Description |
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels-severities) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (omitted if zero for use by tests). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. |
| line | The line number where the entry originated. |
| marker | Redactability marker ` + redactableIndicator + ` (see below for details). |

The redactability marker can be empty; in this case, its position in the common prefix is
a double ASCII space character which can be used to reliably identify this situation.
Expand Down Expand Up @@ -142,18 +142,18 @@ Each line of output starts with the following prefix:

Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker tags counter

| Field | Description |
|-----------------|---------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (omitted if zero for use by tests). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. |
| line | The line number where the entry originated. |
| marker | Redactability marker ` + redactableIndicator + ` (see below for details). |
| Field | Description |
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels-severities) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (omitted if zero for use by tests). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. |
| line | The line number where the entry originated. |
| marker | Redactability marker ` + redactableIndicator + ` (see below for details). |
| tags | The logging tags, enclosed between `[` and `]`. May be absent. |
| counter | The entry counter. Always present. |

Expand Down Expand Up @@ -195,21 +195,21 @@ Each line of output starts with the following prefix:

Lyymmdd hh:mm:ss.uuuuuu goid [chan@]file:line marker [tags...] counter cont

| Field | Description |
|-----------------|---------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (zero when cannot be determined). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. Also see below. |
| line | The line number where the entry originated. |
| marker | Redactability marker "⋮" (see below for details). |
| tags | The logging tags, enclosed between `[` and `]`. See below. |
| counter | The optional entry counter (see below for details). |
| cont | Continuation mark for structured and multi-line entries. See below. |
| Field | Description |
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
| L | A single character, representing the [log level](logging.html#logging-levels-severities) (e.g., `I` for `INFO`). |
| yy | The year (zero padded; i.e., 2016 is `16`). |
| mm | The month (zero padded; i.e., May is `05`). |
| dd | The day (zero padded). |
| hh:mm:ss.uuuuuu | Time in hours, minutes and fractional seconds. Timezone is UTC. |
| goid | The goroutine id (zero when cannot be determined). |
| chan | The channel number (omitted if zero for backward compatibility). |
| file | The file name where the entry originated. Also see below. |
| line | The line number where the entry originated. |
| marker | Redactability marker "⋮" (see below for details). |
| tags | The logging tags, enclosed between `[` and `]`. See below. |
| counter | The optional entry counter (see below for details). |
| cont | Continuation mark for structured and multi-line entries. See below. |

The `chan@` prefix before the file name indicates the logging channel,
and is omitted if the channel is `DEV`.
Expand Down
2 changes: 1 addition & 1 deletion docs/generated/logsinks.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ For example:
The default output format for Fluent sinks is
`json-fluent-compact`. The `fluent` variants of the JSON formats
include a `tag` field as required by the Fluentd protocol, which
the non-`fluent` JSON [format variants](logformats.html) do not include.
the non-`fluent` JSON [format variants](log-formats.html) do not include.

{{site.data.alerts.callout_info}}
Run `cockroach debug check-log-config` to verify the effect of defaults inheritance.
Expand Down
8 changes: 4 additions & 4 deletions pkg/ccl/serverccl/server_sql_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ func TestSQLServer(t *testing.T) {
_, db := serverutils.StartTenant(
t,
tc.Server(0),
base.TestTenantArgs{TenantID: roachpb.MakeTenantID(security.EmbeddedTenantIDs()[0])},
base.TestTenantArgs{TenantID: serverutils.TestTenantID()},
)
defer db.Close()
r := sqlutils.MakeSQLRunner(db)
Expand All @@ -74,7 +74,7 @@ func TestTenantCannotSetClusterSetting(t *testing.T) {
defer tc.Stopper().Stop(ctx)

// StartTenant with the default permissions to
_, db := serverutils.StartTenant(t, tc.Server(0), base.TestTenantArgs{TenantID: roachpb.MakeTenantID(10), AllowSettingClusterSettings: false})
_, db := serverutils.StartTenant(t, tc.Server(0), base.TestTenantArgs{TenantID: serverutils.TestTenantID(), AllowSettingClusterSettings: false})
defer db.Close()
_, err := db.Exec(`SET CLUSTER SETTING sql.defaults.vectorize=off`)
require.NoError(t, err)
Expand Down Expand Up @@ -118,7 +118,7 @@ func TestTenantHTTP(t *testing.T) {

tenant, err := tc.Server(0).StartTenant(ctx,
base.TestTenantArgs{
TenantID: roachpb.MakeTenantID(security.EmbeddedTenantIDs()[0]),
TenantID: serverutils.TestTenantID(),
})
require.NoError(t, err)
t.Run("prometheus", func(t *testing.T) {
Expand Down Expand Up @@ -154,7 +154,7 @@ func TestIdleExit(t *testing.T) {
countdownDuration := 4000 * time.Millisecond
tenant, err := tc.Server(0).StartTenant(ctx,
base.TestTenantArgs{
TenantID: roachpb.MakeTenantID(10),
TenantID: serverutils.TestTenantID(),
IdleExitAfter: warmupDuration,
TestingKnobs: base.TestingKnobs{
TenantTestingKnobs: &sql.TenantTestingKnobs{
Expand Down
3 changes: 1 addition & 2 deletions pkg/ccl/streamingccl/streamingest/stream_ingestion_job_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (
_ "github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamproducer"
"github.com/cockroachdb/cockroach/pkg/jobs"
"github.com/cockroachdb/cockroach/pkg/jobs/jobspb"
"github.com/cockroachdb/cockroach/pkg/roachpb"
"github.com/cockroachdb/cockroach/pkg/security"
"github.com/cockroachdb/cockroach/pkg/testutils"
"github.com/cockroachdb/cockroach/pkg/testutils/jobutils"
Expand Down Expand Up @@ -50,7 +49,7 @@ func TestTenantStreaming(t *testing.T) {
defer source.Stopper().Stop(ctx)

// Start tenant server in the srouce cluster.
tenantID := roachpb.MakeTenantID(10)
tenantID := serverutils.TestTenantID()
_, tenantConn := serverutils.StartTenant(t, source, base.TestTenantArgs{TenantID: tenantID})
defer tenantConn.Close()
// sourceSQL refers to the tenant generating the data.
Expand Down
2 changes: 1 addition & 1 deletion pkg/ccl/streamingccl/streamingtest/replication_helpers.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ SET CLUSTER SETTING sql.defaults.experimental_stream_replication.enabled = 'on';
require.NoError(t, err)

// Start tenant server
tenantID := roachpb.MakeTenantID(10)
tenantID := serverutils.TestTenantID()
_, tenantConn := serverutils.StartTenant(t, s, base.TestTenantArgs{TenantID: tenantID})

// Sink to read data from.
Expand Down
1 change: 0 additions & 1 deletion pkg/ccl/testccl/sqlccl/BUILD.bazel
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ go_test(
deps = [
"//pkg/base",
"//pkg/ccl/kvccl/kvtenantccl",
"//pkg/roachpb",
"//pkg/security",
"//pkg/security/securitytest",
"//pkg/server",
Expand Down
3 changes: 1 addition & 2 deletions pkg/ccl/testccl/sqlccl/run_control_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import (

"github.com/cockroachdb/cockroach/pkg/base"
"github.com/cockroachdb/cockroach/pkg/ccl/kvccl/kvtenantccl"
"github.com/cockroachdb/cockroach/pkg/roachpb"
"github.com/cockroachdb/cockroach/pkg/sql/sqltestutils"
"github.com/cockroachdb/cockroach/pkg/testutils/serverutils"
"github.com/cockroachdb/cockroach/pkg/testutils/sqlutils"
Expand Down Expand Up @@ -48,7 +47,7 @@ func makeRunControlTestCases(t *testing.T) ([]runControlTestCase, func()) {
testCases[0].conn1 = tc.ServerConn(0).Conn
testCases[0].conn2 = tc.ServerConn(1).Conn

_, tenantDB := serverutils.StartTenant(t, tc.Server(0), base.TestTenantArgs{TenantID: roachpb.MakeTenantID(10)})
_, tenantDB := serverutils.StartTenant(t, tc.Server(0), base.TestTenantArgs{TenantID: serverutils.TestTenantID()})
testCases[1].name = "Tenant"
testCases[1].conn1 = tenantDB.Conn
testCases[1].conn2 = tenantDB.Conn
Expand Down
6 changes: 3 additions & 3 deletions pkg/kv/kvserver/client_tenant_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ func TestTenantsStorageMetricsOnSplit(t *testing.T) {
require.NoError(t, err)
defer store.Stopper().Stop(ctx)

tenantID := roachpb.MakeTenantID(10)
tenantID := serverutils.TestTenantID()
codec := keys.MakeSQLCodec(tenantID)

tenantPrefix := codec.TenantPrefix()
Expand Down Expand Up @@ -162,7 +162,7 @@ func TestTenantRateLimiter(t *testing.T) {
ctx := context.Background()
defer s.Stopper().Stop(ctx)

tenantID := roachpb.MakeTenantID(10)
tenantID := serverutils.TestTenantID()
codec := keys.MakeSQLCodec(tenantID)

tenantPrefix := codec.TenantPrefix()
Expand Down Expand Up @@ -235,7 +235,7 @@ func TestTenantRateLimiter(t *testing.T) {
return string(read)
}
makeMetricStr := func(expCount int64) string {
const tenantMetricStr = `kv_tenant_rate_limit_write_requests_admitted{store="1",tenant_id="10"}`
tenantMetricStr := fmt.Sprintf(`kv_tenant_rate_limit_write_requests_admitted{store="1",tenant_id="%d"}`, tenantID.ToUint64())
return fmt.Sprintf("%s %d", tenantMetricStr, expCount)
}

Expand Down
3 changes: 1 addition & 2 deletions pkg/server/diagnostics/reporter_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ import (
"github.com/cockroachdb/cockroach/pkg/config/zonepb"
"github.com/cockroachdb/cockroach/pkg/keys"
"github.com/cockroachdb/cockroach/pkg/roachpb"
"github.com/cockroachdb/cockroach/pkg/security"
"github.com/cockroachdb/cockroach/pkg/server"
"github.com/cockroachdb/cockroach/pkg/server/diagnostics"
"github.com/cockroachdb/cockroach/pkg/server/diagnostics/diagnosticspb"
Expand Down Expand Up @@ -55,7 +54,7 @@ func TestTenantReport(t *testing.T) {
defer rt.Close()

tenantArgs := base.TestTenantArgs{
TenantID: roachpb.MakeTenantID(security.EmbeddedTenantIDs()[0]),
TenantID: serverutils.TestTenantID(),
AllowSettingClusterSettings: true,
TestingKnobs: rt.testingKnobs,
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/sql/logictest/logic.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1449,7 +1449,7 @@ func (t *logicTest) newCluster(serverArgs TestServerArgs) {
if cfg.useTenant {
var err error
tenantArgs := base.TestTenantArgs{
TenantID: roachpb.MakeTenantID(10),
TenantID: serverutils.TestTenantID(),
AllowSettingClusterSettings: true,
TestingKnobs: base.TestingKnobs{
SQLExecutor: &sql.ExecutorTestingKnobs{
Expand Down
1 change: 0 additions & 1 deletion pkg/sql/sqltestutils/BUILD.bazel
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@ go_library(
"//pkg/keys",
"//pkg/kv",
"//pkg/roachpb",
"//pkg/security",
"//pkg/server",
"//pkg/server/diagnostics",
"//pkg/sql",
Expand Down
3 changes: 1 addition & 2 deletions pkg/sql/sqltestutils/telemetry.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (

"github.com/cockroachdb/cockroach/pkg/base"
"github.com/cockroachdb/cockroach/pkg/roachpb"
"github.com/cockroachdb/cockroach/pkg/security"
"github.com/cockroachdb/cockroach/pkg/server"
"github.com/cockroachdb/cockroach/pkg/server/diagnostics"
"github.com/cockroachdb/cockroach/pkg/sql"
Expand Down Expand Up @@ -161,7 +160,7 @@ func (tt *telemetryTest) Start(t *testing.T, serverArgs []base.TestServerArgs) {
log.TestingClearServerIdentifiers()

tt.tenant, tt.tenantDB = serverutils.StartTenant(tt.t, tt.server, base.TestTenantArgs{
TenantID: roachpb.MakeTenantID(security.EmbeddedTenantIDs()[0]),
TenantID: serverutils.TestTenantID(),
AllowSettingClusterSettings: true,
TestingKnobs: mapServerArgs[0].Knobs,
})
Expand Down
7 changes: 7 additions & 0 deletions pkg/testutils/serverutils/test_server_shim.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -355,6 +355,13 @@ func StartTenant(
return tenant, goDB
}

// TestTenantID returns a roachpb.TenantID that can be used when
// starting a test Tenant. The returned tenant IDs match those built
// into the test certificates.
func TestTenantID() roachpb.TenantID {
return roachpb.MakeTenantID(security.EmbeddedTenantIDs()[0])
}

// GetJSONProto uses the supplied client to GET the URL specified by the parameters
// and unmarshals the result into response.
func GetJSONProto(ts TestServerInterface, path string, response protoutil.Message) error {
Expand Down
Loading

0 comments on commit e2f1d1a

Please sign in to comment.