Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(internal): update postprocessor README instructions #7708

Merged
merged 5 commits into from
Apr 10, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions internal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,26 @@ metadata required. For now, `.repo-metadata-full.json` includes everything.

## cloudbuild.yaml

To kick off a build locally run from the repo root:
The `cloudbuild.yaml` Cloud Build configuration currently supports:

* Building a docker container from the `internal/postprocessor/Dockerfile`.

The build can be run locally in the `google-cloud-go` root directory:

```bash
gcloud builds submit --project=cloud-devrel-kokoro-resources --config=internal/cloudbuild.yaml
```

### Updating OwlBot SHA
See the [postprocessor/README](postprocessor/README.md) for instructions
regarding updating the post-processor docker container.

You may want to manually update the which version of the post processor will be
used -- to do this you need to update the SHA in the OwlBot lock file. Start by
running the following commands:
### Updating OwlBot SHA

```bash
docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest
docker inspect --format='{{index .RepoDigests 0}}' gcr.io/cloud-devrel-public-resources/owlbot-go:latest
```
You may want to manually update the which version of the post-processor will be
used -- to do this you need to update the SHA in the OwlBot lock file.

This will give you a SHA. You can use this value to update the value in
`.github/.OwlBot.lock.yaml`.
See the [postprocessor/README](postprocessor/README.md) for detailed
instructions.

*Note*: OwlBot will eventually open a pull request to update this value if it
discovers a new version of the container.
86 changes: 60 additions & 26 deletions internal/postprocessor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,41 +13,75 @@ fork instead of the repo.
After following these steps the generated code will have replaced corresponding
files in the `google-cloud-go` repo.

## Docker container
## Running the post-processor docker container locally

The Docker container needs to be built with the context of the entire
`google-cloud-go/internal` directory. When building the container, do so from
the `google-cloud-go/internal` directory
You can verify the name of the docker container name can be found in the
`.github/OwlBot.yaml` and `github/OwlBot.lock.yaml` files.

## Running the post-processor locally

The Docker container name needed will be found in the `.github/OwlBot.yaml` and
`github/OwlBot.lock.yaml` files.
To run post-processor run:
In the `google-cloud-go` root directory:

```bash
docker pull <container-name>
docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo <container-name>
docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest
docker run --user $(id -u):$(id -g) --rm -v $(pwd):/repo -w /repo gcr.io/cloud-devrel-public-resources/owlbot-go:latest
```

## Testing the post-processor locally
## Making changes, rebuilding the docker container and updating the OwlBot SHA

You can run the post-processor locally on selected directories or on all of the
clients in the root directory.
After making changes to the post-processor, you need to publish a new version
of the post-processor docker container and manually update the which version of
the post-processor is used by OwlBot. To do this you need to update the SHA in
the OwlBot lock file.

### Run post-processor on all clients
### Docker container

From the `google-cloud-go/internal/postprocessor` directory run:
```bash
go run main.go -client-root="../.." -googleapis-dir="/path/to/local/googleapis"
```
*Note*: The docker container needs to be built with the context of the entire
`google-cloud-go/internal` directory. If building the container directly with
`docker`, rather than with Cloud Build as instructed below, be sure to do so
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine for now, but would you file an issue about making this easier. I don't think we should have to have the context of the internal directory anymore since I decoupled the postprocessor from gapicgen earlier this week.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just delete this note? I think building the container with Cloud Build is easy and simple.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted the note. Let me know if you want something added here, or if an issue needs to be filed.

from the `google-cloud-go/internal` directory.

### Run post-processor on select clients
1. In your `google-cloud-go` repo, create a branch.
2. Make changes to the post-processor.
3. Test your changes. You can run the post-processor locally on selected
clients or on all of the clients in the root directory. If the `branch`
flag is not set to a non-empty value, the post-processor will exit early
without changes. In the `google-cloud-go/internal/postprocessor` directory:

From the `google-cloud-go/internal/postprocessor` directory run the same command, but with an added `dirs` flag containing a comma-separated list of the names of the clients on which to run the post-processor. The example below shows the command for running the post-processor on the `accessapproval` and `asset` libraries:
```bash
go run main.go -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -dirs="accessapproval,asset"
```
```bash
go run . -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -branch="my-branch"
```

To test only selected clients:

```bash
go run . -client-root="../.." -googleapis-dir="/path/to/local/googleapis" -branch="my-branch" -dirs="accessapproval,asset"
```
4. Clean up any changes made by post-processor test runs in the previous step.
5. Commit your changes.
6. Open your PR and respond to feedback.
7. Once your PR is complete (but before merging it!), rebuild the post-processor
docker image from your local branch using the
`google-cloud-go/internal/cloudbuild.yaml` Cloud Build configuration. In the
`google-cloud-go` root directory:

```bash
gcloud builds submit --project=cloud-devrel-kokoro-resources --config=internal/cloudbuild.yaml
```
8. Read the SHA of the latest post-processor docker image. In any location:

```bash
docker pull gcr.io/cloud-devrel-public-resources/owlbot-go:latest
docker inspect --format='{{index .RepoDigests 0}}' gcr.io/cloud-devrel-public-resources/owlbot-go:latest
```
9. In your branch, update the SHA of the post-processor docker image in
`google-cloud-go/.github/.OwlBot.lock.yaml`. Commit and push the change to
your PR.

*Note*: OwlBot will eventually open a pull request to update the SHA if it
discovers a new version of the container.
10. After your PR is approved and CI is green, merge your changes.

### Initializing new modules
To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md` files in a new module, add the module to the slice in `modconfig.go`. The entry should correspond to the location where the `go.mod` file should be initialized minus the prefix "google-cloud-go/"
## Initializing new modules
To initialize the `internal/version.go`, `go.mod`, `README.md`, and `CHANGES.md`
files in a new module, add the module to the slice in `modconfig.go`. The entry
codyoss marked this conversation as resolved.
Show resolved Hide resolved
should correspond to the location where the `go.mod` file should be initialized
minus the prefix `google-cloud-go/`.