From 5fb08798574cd75dd3c3307bd2d89fdfa055fe57 Mon Sep 17 00:00:00 2001 From: Chris Smith Date: Fri, 7 Apr 2023 13:02:33 -0600 Subject: [PATCH 1/3] chore(internal): update postprocessor README instructions --- internal/README.md | 23 ++++----- internal/postprocessor/README.md | 84 ++++++++++++++++++++++---------- 2 files changed, 70 insertions(+), 37 deletions(-) diff --git a/internal/README.md b/internal/README.md index c1dc6bdff464..b38a4c1a2da2 100644 --- a/internal/README.md +++ b/internal/README.md @@ -19,25 +19,26 @@ metadata required. For now, `.repo-metadata-full.json` includes everything. ## cloudbuild.yaml -To kick off a build locally run from the repo root: +The `cloudbuild.yaml` Cloud Build configuration currently supports: + +* Building a docker container from the `internal/postprocessor/Dockerfile`. + +The build can be run locally in the `google-cloud-go` root directory: ```bash gcloud builds submit --project=cloud-devrel-kokoro-resources --config=internal/cloudbuild.yaml ``` -### Updating OwlBot SHA +See the [postprocessor/README](postprocessor/README.md) for instructions +regarding updating the post-processor docker container. -You may want to manually update the which version of the post processor will be -used -- to do this you need to update the SHA in the OwlBot lock file. Start by -running the following commands: +### Updating OwlBot SHA -```bash -docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest -docker inspect --format='{{index .RepoDigests 0}}' gcr.io/cloud-devrel-public-resources/owlbot-go:latest -``` +You may want to manually update the which version of the post-processor will be +used -- to do this you need to update the SHA in the OwlBot lock file. -This will give you a SHA. You can use this value to update the value in -`.github/.OwlBot.lock.yaml`. +See the [postprocessor/README](postprocessor/README.md) for detailed +instructions. *Note*: OwlBot will eventually open a pull request to update this value if it discovers a new version of the container. diff --git a/internal/postprocessor/README.md b/internal/postprocessor/README.md index eacc8f8a02b6..bd4924dc8d29 100644 --- a/internal/postprocessor/README.md +++ b/internal/postprocessor/README.md @@ -13,41 +13,73 @@ fork instead of the repo. After following these steps the generated code will have replaced corresponding files in the `google-cloud-go` repo. -## Docker container +## Running the post-processor docker container locally -The Docker container needs to be built with the context of the entire -`google-cloud-go/internal` directory. When building the container, do so from -the `google-cloud-go/internal` directory +You can verify the name of the docker container name can be found in the +`.github/OwlBot.yaml` and `github/OwlBot.lock.yaml` files. -## Running the post-processor locally - -The Docker container name needed will be found in the `.github/OwlBot.yaml` and -`github/OwlBot.lock.yaml` files. -To run post-processor run: +In the `google-cloud-go` root directory: ```bash -docker pull -docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo +docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest +docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo gcr.io/cloud-devrel-public-resources/owlbot-go:latest ``` -## Testing the post-processor locally +## Making changes, rebuilding the docker container and updating the OwlBot SHA -You can run the post-processor locally on selected directories or on all of the -clients in the root directory. +After making changes to the post-processor, you need to publish a new version +of the post-processor docker container and manually update the which version of +the post-processor is used by OwlBot. To do this you need to update the SHA in +the OwlBot lock file. -### Run post-processor on all clients +### Docker container -From the `google-cloud-go/internal/postprocessor` directory run: -```bash -go run main.go -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -``` +The docker container needs to be built with the context of the entire +`google-cloud-go/internal` directory. When building the container, as instructed +below, be sure to do so from the `google-cloud-go/internal` directory. -### Run post-processor on select clients +1. In your `google-cloud-go` repo, create a branch. +2. Make changes to the post-processor. +3. Test your changes. You can run the post-processor locally on selected + clients or on all of the clients in the root directory. If the `branch` + flag is not set to a non-empty value, the post-processor will exit early + without changes. In the `google-cloud-go/internal/postprocessor` directory: -From the `google-cloud-go/internal/postprocessor` directory run the same command, but with an added `dirs` flag containing a comma-separated list of the names of the clients on which to run the post-processor. The example below shows the command for running the post-processor on the `accessapproval` and `asset` libraries: -```bash -go run main.go -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -dirs="accessapproval,asset" -``` + ```bash + go run . -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -branch="my-branch" + ``` + + To test only selected clients: + + ```bash + go run . -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -branch="my-branch" -dirs="accessapproval,asset" + ``` +4. Commit your changes. +5. Open your PR and respond to feedback. +6. Once your PR is complete (but before merging it!), rebuild the post-processor + docker image from your local branch using the + `google-cloud-go/internal/cloudbuild.yaml` Cloud Build configuration. In the + `google-cloud-go` root directory: + + ```bash + gcloud builds submit --project=cloud-devrel-kokoro-resources --config=internal/cloudbuild.yaml + ``` +7. Read the SHA of the latest post-processor docker image. In any location: + + ```bash + docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest + docker inspect --format='{{index .RepoDigests 0}}' gcr.io/cloud-devrel-public-resources/owlbot-go:latest + ``` +8. In your branch, update the SHA of the post-processor docker image in + `google-cloud-go/.github/.OwlBot.lock.yaml`. Commit and push the change to + your PR. + + *Note*: OwlBot will eventually open a pull request to update the SHA if it + discovers a new version of the container. +9. After your PR is approved and CI is green, merge your changes. -### Initializing new modules -To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md` files in a new module, add the module to the slice in `modconfig.go`. The entry should correspond to the location where the `go.mod` file should be initialized minus the prefix "google-cloud-go/" +## Initializing new modules +To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md` +files in a new module, add the module to the slice in `modconfig.go`. The entry +should correspond to the location where the `go.mod` file should be initialized +minus the prefix `google-cloud-go/`. From 03cd7944f8314fa6d743859c35f2126c37d50e89 Mon Sep 17 00:00:00 2001 From: Chris Smith Date: Fri, 7 Apr 2023 13:12:30 -0600 Subject: [PATCH 2/3] add cleanup step --- internal/postprocessor/README.md | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/internal/postprocessor/README.md b/internal/postprocessor/README.md index bd4924dc8d29..8ab77558cdfb 100644 --- a/internal/postprocessor/README.md +++ b/internal/postprocessor/README.md @@ -34,9 +34,10 @@ the OwlBot lock file. ### Docker container -The docker container needs to be built with the context of the entire -`google-cloud-go/internal` directory. When building the container, as instructed -below, be sure to do so from the `google-cloud-go/internal` directory. +*Note*: The docker container needs to be built with the context of the entire +`google-cloud-go/internal` directory. If building the container directly with +`docker`, rather than with Cloud Build as instructed below, be sure to do so +from the `google-cloud-go/internal` directory. 1. In your `google-cloud-go` repo, create a branch. 2. Make changes to the post-processor. @@ -54,9 +55,10 @@ below, be sure to do so from the `google-cloud-go/internal` directory. ```bash go run . -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -branch="my-branch" -dirs="accessapproval,asset" ``` -4. Commit your changes. -5. Open your PR and respond to feedback. -6. Once your PR is complete (but before merging it!), rebuild the post-processor +4. Clean up any changes made by post-processor test runs in the previous step. +5. Commit your changes. +6. Open your PR and respond to feedback. +7. Once your PR is complete (but before merging it!), rebuild the post-processor docker image from your local branch using the `google-cloud-go/internal/cloudbuild.yaml` Cloud Build configuration. In the `google-cloud-go` root directory: @@ -64,19 +66,19 @@ below, be sure to do so from the `google-cloud-go/internal` directory. ```bash gcloud builds submit --project=cloud-devrel-kokoro-resources --config=internal/cloudbuild.yaml ``` -7. Read the SHA of the latest post-processor docker image. In any location: +8. Read the SHA of the latest post-processor docker image. In any location: ```bash docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest docker inspect --format='{{index .RepoDigests 0}}' gcr.io/cloud-devrel-public-resources/owlbot-go:latest ``` -8. In your branch, update the SHA of the post-processor docker image in +9. In your branch, update the SHA of the post-processor docker image in `google-cloud-go/.github/.OwlBot.lock.yaml`. Commit and push the change to your PR. *Note*: OwlBot will eventually open a pull request to update the SHA if it discovers a new version of the container. -9. After your PR is approved and CI is green, merge your changes. +10. After your PR is approved and CI is green, merge your changes. ## Initializing new modules To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md` From 21299fe7cec277e94455566293db91ce61960e70 Mon Sep 17 00:00:00 2001 From: Chris Smith Date: Fri, 7 Apr 2023 15:43:34 -0600 Subject: [PATCH 3/3] delete container note and update new module section --- internal/postprocessor/README.md | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/internal/postprocessor/README.md b/internal/postprocessor/README.md index 8ab77558cdfb..848e5bd36a7f 100644 --- a/internal/postprocessor/README.md +++ b/internal/postprocessor/README.md @@ -32,13 +32,6 @@ of the post-processor docker container and manually update the which version of the post-processor is used by OwlBot. To do this you need to update the SHA in the OwlBot lock file. -### Docker container - -*Note*: The docker container needs to be built with the context of the entire -`google-cloud-go/internal` directory. If building the container directly with -`docker`, rather than with Cloud Build as instructed below, be sure to do so -from the `google-cloud-go/internal` directory. - 1. In your `google-cloud-go` repo, create a branch. 2. Make changes to the post-processor. 3. Test your changes. You can run the post-processor locally on selected @@ -81,7 +74,10 @@ from the `google-cloud-go/internal` directory. 10. After your PR is approved and CI is green, merge your changes. ## Initializing new modules -To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md` -files in a new module, add the module to the slice in `modconfig.go`. The entry -should correspond to the location where the `go.mod` file should be initialized -minus the prefix `google-cloud-go/`. + +The post-processor initializes new modules by generating the required files +`internal/version.go`, `go.mod`, `README.md` and `CHANGES.md`. + +To add a new module, add the directory name of the module to `modules` in +`google-cloud-go/internal/postprocessor/config.yaml`. Please maintain +alphabetical ordering of the module names.