Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachprod: fetch secrets from cloud store #124099

Merged

Conversation

nameisbhaskar
Copy link
Contributor

Due to the complexity of fetching the secrets from the secrets manager, the secrets are now maintained in cloud storage.

Fixes: #117125
Epic: none

@nameisbhaskar nameisbhaskar requested a review from a team as a code owner May 14, 2024 05:55
@nameisbhaskar nameisbhaskar requested review from srosenberg and vidit-bhat and removed request for a team May 14, 2024 05:55
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@nameisbhaskar nameisbhaskar force-pushed the user/bhaskar/use-store-secrets branch 12 times, most recently from 988c747 to 9587ca8 Compare May 15, 2024 15:33
Copy link
Member

@srosenberg srosenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Tested on my gceworker by running,

roachtest run --cloud gce '^ycsb/C/nodes=3/cpu=32$'

It works after adding the default cockroach-workers service account to the bucket.

@nameisbhaskar
Copy link
Contributor Author

Thanks Stan for reviewing this!

Due to the complexity of fetching the secrets from the secrets
manager, the secrets are now maintained in cloud storage.

Fixes: cockroachdb#117125
Epic: none
@nameisbhaskar nameisbhaskar force-pushed the user/bhaskar/use-store-secrets branch from 9587ca8 to 473f257 Compare May 16, 2024 05:08
@nameisbhaskar
Copy link
Contributor Author

bors r=@srosenberg

@craig craig bot merged commit 71da6da into cockroachdb:master May 16, 2024
21 of 22 checks passed
@nameisbhaskar nameisbhaskar deleted the user/bhaskar/use-store-secrets branch May 20, 2024 06:42
DarrylWong added a commit to DarrylWong/fork that referenced this pull request Jun 13, 2024
As of cockroachdb#124099 we now store the service account creds in cloud
storage. We already use this to access prometheus when generating
dynamic configs. This change does the same for Grafana annotations
by extracting the common logic into a helper.

This will allow users to have access to Grafana annotations out
of the box locally, and limit the amount of benign but potentially
confusing warnings about invalid credentials.

Epic: none
Fixes: none
Release note: none
DarrylWong added a commit to DarrylWong/fork that referenced this pull request Jun 24, 2024
As of cockroachdb#124099 we now store the service account creds in cloud
storage. We already use this to access prometheus when generating
dynamic configs. This change does the same for Grafana annotations
by extracting the common logic into a helper.

This will allow users to have access to Grafana annotations out
of the box locally, and limit the amount of benign but potentially
confusing warnings about invalid credentials.

Epic: none
Fixes: none
Release note: none
craig bot pushed a commit that referenced this pull request Jun 24, 2024
125571: roachtest: grafana annotations read creds from cloud storage r=herkolategan,renatolabs a=DarrylWong

As of #124099 we now store the service account creds in cloud storage. We already use this to access prometheus when generating dynamic configs. This change does the same for Grafana annotations by extracting the common logic into a helper.

This will allow users to have access to Grafana annotations out of the box locally, and limit the amount of benign but potentially confusing warnings about invalid credentials.

Release note: none
Fixes: none
Epic: none

126084: jobs: limit number of retained dsp-diag-url info rows r=dt a=dt

Fixes #126083.

126109: kvserver: deflake `WALBytesWritten` metric r=raduberinde a=aadityasondhi

There is a race condition in Pebble metrics where sometimes the WAL is rotated prior to updated the BytesIn metric to account for the previous WAL. The metrics collection call happens async so it can sometimes cause this metric to decrease for a scrape window.

Fixes: #125736.

Release note: None

126114: sql: avoid slow lock verification in TestSchemaChangeAfterCreateInTxn r=rafiss a=rafiss

The addition of test-only verification pushed this test over the timeout sometimes, such that running it under the deadlock detector would cause spurious failures. We avoid this by making the test smaller under deadlock, like we do for race builds.

fixes #126075
Release justification: test only change
Release note: None

Co-authored-by: DarrylWong <[email protected]>
Co-authored-by: David Taylor <[email protected]>
Co-authored-by: Aaditya Sondhi <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
asg0451 pushed a commit to asg0451/cockroach that referenced this pull request Jun 25, 2024
As of cockroachdb#124099 we now store the service account creds in cloud
storage. We already use this to access prometheus when generating
dynamic configs. This change does the same for Grafana annotations
by extracting the common logic into a helper.

This will allow users to have access to Grafana annotations out
of the box locally, and limit the amount of benign but potentially
confusing warnings about invalid credentials.

Epic: none
Fixes: none
Release note: none
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: using custom AdminUIPort breaks automated prometheus scraping
3 participants