roachtest: switch to regional buckets in GCE; MR is no longer free wrt egress #111371
Assignees
Labels
A-testing
Testing tools and infrastructure
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-testeng
TestEng Team
Comments
srosenberg
added
C-enhancement
|
cc @cockroachdb/test-eng |
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Oct 31, 2023
This weekly test is very costly (leads to cross region egress) and is already run on aws. We can reconsider unskipping it after cockroachdb#111371 is addressed. Epic: none Release note: none
craig bot
pushed a commit
that referenced
this issue
Oct 31, 2023
112106: server: add tests for combined stmts api source tables r=maryliag a=xinhaoz This patch creates the sqlstatstestutils pkg in sqlstats/persistedsqlstats. It moves the function `GetRandomizedCollectedStatementStatisticsForTest` which was previously in sqlstatsutil. This pkg is introduced in preparation for adding more testing utilities to mock sql stats tables without creating cyclic dependencies with sqlstatsutil. Epic: none Release note: None ### 2: [server: add tests for combined stmts api](1bf95c1) Add testing to verify that the combined stmts api uses the correct source tables given different table states. Epic: none 112990: kvserver: setup ranges in large unsplittable replicate test r=andrewbaptist a=kvoli `TestLargeUnsplittableRangeReplicate` would fail when the SQL query to show the table's replicas returned no results for the unsplittable range being tested. No results would be returned when the table split was delayed. Update the test to insert a split at the first row, as well as the existing split at the second row. This avoids the first range starting with 5 replicas and the test's reliance on timely span config splits. The previous deflake attempt is reverted. Resolves: #112774 Release note: None 113500: roachtest: skip restore/tpce/32TB/inc-count=400/gce/nodes=15/cpus=16 r=msbutler a=msbutler This weekly test is very costly (leads to cross region egress) and is already run on aws. We can reconsider unskipping it after #111371 is addressed. Epic: none Release note: none Co-authored-by: Xin Hao Zhang <[email protected]> Co-authored-by: Austen McClernon <[email protected]> Co-authored-by: Michael Butler <[email protected]>
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Dec 8, 2023
…es-us-east1 Informs cockroachdb#111371 Release note: none
craig bot
pushed a commit
that referenced
this issue
Dec 11, 2023
114853: workload/schemachange: update and disable `ALTER TYPE DROP VALUE` r=Xiang-Gu a=chrisseto #### 31a2ac9 workload/schemachange: update and disable `ALTER TYPE DROP VALUE` Previously, `ALTER TYPE DROP VALUE` was using the procedural generation that is most common in this workload. The desire to add support UDFs required revisiting the implement as referenced enum values cannot be dropped. Unfortunately, there is not a straightforward way to determine if a specific enum is being referenced by other descriptors (namely default expressions for columns and UDFs). After the initial update and a few testing rounds with UDF support, a bug between enums and UDFs was discovered. (See #114844 for details). This commit upgrades the implementation of `ALTER TYPE DROP VALUE` to utilize the `Generate` helper and dodge cases of referenced enum values. Due to the aforementioned bug and another outstanding flake, this commit also disabled `ALTER TYPE DROP VALUE` to prevent further flakes an noise. Epic: CRDB-19168 Informs: #113859, #114844 Release note: None 115677: roachtest: replace gs://cockroach-fixtures with gs://cockroach-fixtures-us-east1 r=srosenberg a=msbutler Informs #111371 Release note: none 116072: pgwirecancel: deflake TestCancelCopyTo r=rafiss a=rafiss This test had a race condition since it used the same err variable in different goroutines. fixes #116032 Release note: None Co-authored-by: Chris Seto <[email protected]> Co-authored-by: Michael Butler <[email protected]> Co-authored-by: Rafi Shamim <[email protected]>
DarrylWong
pushed a commit
to DarrylWong/fork
that referenced
this issue
Dec 11, 2023
…es-us-east1 Informs cockroachdb#111371 Release note: none
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Dec 16, 2023
…es-us-east1 Informs cockroachdb#111371 Release note: none
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Dec 21, 2023
…ckroach-fixtures-us-east1 Informs cockroachdb#111371 Release note: none
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Dec 21, 2023
…ckroach-fixtures-us-east1 Informs cockroachdb#111371 Release note: none
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Dec 28, 2023
…es-us-east1 Informs cockroachdb#111371 Release note: none
msbutler
added a commit
to msbutler/cockroach
that referenced
this issue
Jan 4, 2024
…ckroach-fixtures-us-east1 Informs cockroachdb#111371 Release note: none
|
fixture migration has complete and old bucket has been deleted. closing. |
blathers-crl bot
pushed a commit
that referenced
this issue
Feb 2, 2024
…ckroach-fixtures-us-east1 Informs #111371 Release note: none
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As of last year, GCE decided to charge for egress, when reading from a multi-regional (MR) bucket [1]. AWS' s3 doesn't even support multi-regional buckets. (Technically, it supports cross-regional replication, but the bucket is always pinned to a specific region.) In light of the above, we are going to stick with regional buckets for roachtest, across all providers. The reduction in egress (cost) also facilitates better resource load-balancing across regions; e.g., our local SSD quota has been steadily running low in
us-east1, while the request to increase it, continues to linger; moving a small percentage of clusters tous-central1alleviates the quota issue.[1]https://cloud.google.com/storage/pricing-announce#network
Jira issue: CRDB-31875
The text was updated successfully, but these errors were encountered: