Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
109521: cluster-ui: handle partial response errors on the database table page r=THardy98 a=THardy98

Part of: #102386

**Demos** (Note: these demos show this same logic applied to both the databases and database details pages as well):
DB-Console
- https://www.loom.com/share/5108dd655ad342f28323e72eaf68219c
- https://www.loom.com/share/1973383dacd7494a84e10bf39e5b85a3

This change applies the same error handling ideas from #109245 to the
database table page, enabling non-admin users to use the database table
page and providing better transparency to data fetching issues.

`unavailable` fields provide a tooltip that displays the error impacting
that field.

Release note (ui change): Non-admin users are able to use the database
table page.

109788: docs-issue-generation: Create docs issues in Jira r=nickvigilante a=nickvigilante

Fixes DOC-7066

Additional fixes:

- Update the authorization scheme for GitHub to `Bearer`
- Divide `docs_issue_generation.go` into logical package files

Release note: None

110288: roachtest: codify longer ttl external storage buckets r=renatolabs a=msbutler

Previously, the only codified external buckets for roachtests to back up to was
the `cockroachdb-backup-testing` buckets in s3 and gcs which each had a ttl of
1 day. This low ttl is not suitable for roachtests that produce backups that
the test failure investigator may want to inspect. This patch codifies the new
`cockroachdb-backup-testing-long-ttl` buckets in s3 and gcs, which currently
have a ttl of 20 days, the same ttl that team city artifacts have.

This patch also points the c2c, backup-restore/mixed-version, and
disagg-rebalance roachtests to use these new buckets.

Note this PR only points roachtests that run in public TC environments to the
new buckets. A future PR will set the BACKUP_TESTING_BUCKET_LONG_TTL env var
for private roacttests to a new bucket with a longer ttl.

Epic: none

Release note: none


110610: parser/backup: fix normalization of detached = false r=dt a=dt

Release note: none.
Epic: none.

110638: roachtest: add c2c/multiregion/SameRegions/kv0 roachtest r=stevendanna a=msbutler

This patch adds a new c2c roachtest that spins up multiregion source and destination clusters, constrains the kv database to the us-east1-b region, and asserts that the replicated span configuration enforces the regional constraint on the destination cluster during replication.

Informs #109059

Release note: None

Co-authored-by: Thomas Hardy <[email protected]>
Co-authored-by: Nick Vigilante <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: David Taylor <[email protected]>
  • Loading branch information
5 people committed Sep 14, 2023
6 parents 5a6aa76 + 27d6552 + 711e7c9 + 973ddb1 + a71bdcc + c00eac5 commit 58b28b5
Show file tree
Hide file tree
Showing 24 changed files with 4,444 additions and 1,203 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,13 @@ export TESTS="${TESTS:-costfuzz/workload-replay}"
export ROACHTEST_BUCKET="${ROACHTEST_BUCKET:-cockroach-nightly-private}"
export GCE_PROJECT="e2e-infra-381422"
export BACKUP_TESTING_BUCKET="cockroach-backup-testing-private"

# TODO(msbutler): use a different bucket once it is created. Sadly, I don't have the permissions
# currently to create a new bucket in this gce project.
export BACKUP_TESTING_BUCKET_LONG_TTL="cockroach-backup-testing-private"
export COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING=1
export COCKROACH_NO_EXAMPLE_DATABASE=1
export COCKROACH_AUTO_BALLAST=false

BAZEL_SUPPORT_EXTRA_DOCKER_ARGS="-e LITERAL_ARTIFACTS_DIR=$root/artifacts -e BUILD_VCS_NUMBER -e CLOUD=gce -e TESTS -e COUNT -e GITHUB_API_TOKEN -e GITHUB_ORG -e GITHUB_REPO -e GOOGLE_EPHEMERAL_CREDENTIALS -e ROACHTEST_PRIVATE -e ROACHTEST_BUCKET -e SLACK_TOKEN -e TC_BUILDTYPE_ID -e TC_BUILD_BRANCH -e TC_BUILD_ID -e TC_SERVER_URL -e COCKROACH_DEV_LICENSE -e BACKUP_TESTING_BUCKET -e SFUSER -e SFPASSWORD -e COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING -e COCKROACH_NO_EXAMPLE_DATABASE -e COCKROACH_AUTO_BALLAST -e GCE_PROJECT" \
BAZEL_SUPPORT_EXTRA_DOCKER_ARGS="-e LITERAL_ARTIFACTS_DIR=$root/artifacts -e BUILD_VCS_NUMBER -e CLOUD=gce -e TESTS -e COUNT -e GITHUB_API_TOKEN -e GITHUB_ORG -e GITHUB_REPO -e GOOGLE_EPHEMERAL_CREDENTIALS -e ROACHTEST_PRIVATE -e ROACHTEST_BUCKET -e SLACK_TOKEN -e TC_BUILDTYPE_ID -e TC_BUILD_BRANCH -e TC_BUILD_ID -e TC_SERVER_URL -e COCKROACH_DEV_LICENSE -e BACKUP_TESTING_BUCKET -e BACKUP_TESTING_BUCKET_LONG_TTL -e SFUSER -e SFPASSWORD -e COCKROACH_SKIP_ENABLING_DIAGNOSTIC_REPORTING -e COCKROACH_NO_EXAMPLE_DATABASE -e COCKROACH_AUTO_BALLAST -e GCE_PROJECT" \
run_bazel build/teamcity/internal/cockroach/nightlies/private_roachtest_impl.sh
27 changes: 27 additions & 0 deletions pkg/ccl/streamingccl/streamingest/replication_stream_e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1218,4 +1218,31 @@ func TestStreamingRegionalConstraint(t *testing.T) {

testutils.SucceedsSoon(t,
checkLocalities(tableDesc.PrimaryIndexSpan(destCodec), rangedesc.NewScanner(c.DestSysServer.DB())))

tableName := "test"
tabledIDQuery := fmt.Sprintf(`SELECT id FROM system.namespace WHERE name ='%s'`, tableName)

var tableID uint32
c.SrcTenantSQL.QueryRow(t, tabledIDQuery).Scan(&tableID)
fmt.Printf("%d", tableID)

checkLocalityRanges(t, c.SrcSysSQL, srcCodec, uint32(tableDesc.GetID()), "mars")

}

func checkLocalityRanges(
t *testing.T, sysSQL *sqlutils.SQLRunner, codec keys.SQLCodec, tableID uint32, region string,
) {
targetPrefix := codec.TablePrefix(tableID)
distinctQuery := fmt.Sprintf(`
SELECT
DISTINCT replica_localities
FROM
[SHOW CLUSTER RANGES]
WHERE
start_key ~ '%s'
`, targetPrefix)
var locality string
sysSQL.QueryRow(t, distinctQuery).Scan(&locality)
require.Contains(t, locality, region)
}
7 changes: 7 additions & 0 deletions pkg/cmd/docs-issue-generation/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,19 @@ go_library(
name = "docs-issue-generation_lib",
srcs = [
"docs_issue_generation.go",
"extract.go",
"format.go",
"github.go",
"jira.go",
"main.go",
"requests.go",
"structs.go",
],
importpath = "github.com/cockroachdb/cockroach/pkg/cmd/docs-issue-generation",
visibility = ["//visibility:private"],
deps = [
"//pkg/util/timeutil",
"@com_github_cockroachdb_errors//:errors",
"@com_github_spf13_cobra//:cobra",
],
)
Expand Down
Loading

0 comments on commit 58b28b5

Please sign in to comment.