chore: regenerate common templates (#256)

* build(java): switch to release-publish app for notifying GitHub of release status Source-Author: Jeff Ching <[email protected]> Source-Date: Wed Aug 26 21:48:06 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 019c7168faa0e56619f792693a8acdb30d6de19b Source-Link: googleapis/synthtool@019c716 * build(ci): enable auto-release for dependency-update-only releases Automatically perform a Java client library release when: 1. Only dependency updates are going out in the release since any releases containing bug fixes, build changes or new features should be supervised; 2. There are no outstanding/open dependency update pull requests in the repo. This is to avoid multiple/redundant releases; 3. It is a SNAPSHOT release which is automatically generated post regular release -- this requires no human supervision. Testing done in 5 java-bigquery* client library repos. Example: [chore: release 0.3.4 ](googleapis/java-bigqueryconnection#130) [chore: release 0.3.5-SNAPSHOT](googleapis/java-bigqueryconnection#131) Source-Author: Stephanie Wang <[email protected]> Source-Date: Thu Sep 17 15:30:02 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: 538a68019eb4a36a0cdfa4021f324dd01b784395 Source-Link: googleapis/synthtool@538a680 * chore(java): set yoshi-java as default CODEOWNER Source-Author: Jeff Ching <[email protected]> Source-Date: Mon Sep 21 09:00:06 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 80003a3de2d8a75f5b47cb2e77e018f7f0f776cc Source-Link: googleapis/synthtool@80003a3 * build(java): use yoshi-approver token for auto-approve Source-Author: Jeff Ching <[email protected]> Source-Date: Wed Sep 23 12:46:45 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 916c10e8581804df2b48a0f0457d848f3faa582e Source-Link: googleapis/synthtool@916c10e * chore(ci): skip autorelease workflow on non-release PRs Source-Author: Stephanie Wang <[email protected]> Source-Date: Thu Sep 24 16:57:32 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: 95dbe1bee3c7f7e52ddb24a54c37080620e0d1a2 Source-Link: googleapis/synthtool@95dbe1b * chore(ci): verify autorelease release PR content has changes Source-Author: Stephanie Wang <[email protected]> Source-Date: Thu Sep 24 18:06:14 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: da29da32b3a988457b49ae290112b74f14b713cc Source-Link: googleapis/synthtool@da29da3 * chore(java): use separate autosynth job for README Split java README generation into a separate, per-repo continuous job running from Kokoro. We now generate a new job that runs on commits to the primary branch that strictly manages the README.md. This should prevent us from seeing the contextless autosynth PRs which are caused by non-proto/template upstream changes. Source-Author: Jeff Ching <[email protected]> Source-Date: Tue Sep 29 15:48:03 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: e6168630be3e31eede633ba2c6f1cd64248dec1c Source-Link: googleapis/synthtool@e616863 * build: rename samples lint workflow to checkstyle to disambiguate branch protection with unit lint Source-Author: BenWhitehead <[email protected]> Source-Date: Wed Sep 30 15:14:05 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: 8a7a3021fe97aa0a3641db642fe2b767f1c8110f Source-Link: googleapis/synthtool@8a7a302 * build(java): readme.sh should be executable Source-Author: Jeff Ching <[email protected]> Source-Date: Wed Sep 30 13:20:04 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 0762e8ee2ec21cdfc4d82020b985a104feb0453b Source-Link: googleapis/synthtool@0762e8e * ci(java): suggest formatting fixes GitHub action config to run the java formatter and suggest inline fixes on the pull request. This was tested in java-asset: [example PR](googleapis/java-asset#392). Source-Author: Jeff Ching <[email protected]> Source-Date: Tue Oct 13 09:04:08 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: b65ef07d99946d23e900ef2cc490274a16edd336 Source-Link: googleapis/synthtool@b65ef07 * ci(java): restrict presubmit samples ITs to only snapshot This is to reduce resource consumption since we often times hit resource quota limit in samples testing. e.g. googleapis/java-bigquerydatatransfer#410 Source-Author: Stephanie Wang <[email protected]> Source-Date: Thu Oct 15 17:16:14 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: 27e0e916cbfdb3d5ff6639b686cc04f78a0b0386 Source-Link: googleapis/synthtool@27e0e91 * build(java): auto-approve README regeneration Source-Author: Jeff Ching <[email protected]> Source-Date: Thu Oct 15 16:04:06 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 7c5370937dd9ba9dcf9cd7d2af880a58b389b4f1 Source-Link: googleapis/synthtool@7c53709 * build(java): enable snippet-bot Source-Author: Jeff Ching <[email protected]> Source-Date: Mon Oct 19 16:13:57 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 5a506ec8765cc04f7e29f888b8e9b257d9a7ae11 Source-Link: googleapis/synthtool@5a506ec * Update publish_javadoc.sh We don't want quite as much and need to be in `target/devsite/reference`. Source-Author: Les Vogel <[email protected]> Source-Date: Thu Oct 22 14:10:05 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 9593c3b5b714cc9b17c445aee8834ac2b4b9348b Source-Link: googleapis/synthtool@9593c3b * chore(docs): update code of conduct of synthtool and templates Source-Author: Christopher Wilcox <[email protected]> Source-Date: Thu Oct 22 14:22:01 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: 5f6ef0ec5501d33c4667885b37a7685a30d41a76 Source-Link: googleapis/synthtool@5f6ef0e * chore(ci): fix typo in restrict presubmit samples ITs to only snapshot Fixes below error in Java repos when googleapis/synthtool#804 was merged: ``` github/java-bigquerystorage/.kokoro/build.sh: line 81: pushd: {SAMPLES_DIR}: No such file or directory ``` Thanks @kolea2 for spotting this in Fusion build logs. cc @chingor13 Source-Author: Stephanie Wang <[email protected]> Source-Date: Mon Oct 26 13:44:04 2020 -0400 Source-Repo: googleapis/synthtool Source-Sha: f68649c5f26bcff6817c6d21e90dac0fc71fef8e Source-Link: googleapis/synthtool@f68649c * chore(java): enable generated-files-bot This bot will comment on pull requests that modify templated files. For java, we configure to read from our 2 synth.metadata files: (primary templates and README) Source-Author: Jeff Ching <[email protected]> Source-Date: Fri Oct 30 14:22:02 2020 -0700 Source-Repo: googleapis/synthtool Source-Sha: b19b401571e77192f8dd38eab5fb2300a0de9324 Source-Link: googleapis/synthtool@b19b401 * chore(java): use production staging bucket Source-Author: Les Vogel <[email protected]> Source-Date: Wed Nov 4 10:18:01 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: c7824ea48ff6d4d42dfae0849aec8a85acd90bd9 Source-Link: googleapis/synthtool@c7824ea * chore(java): ignore return code 28 in README autosynth job Exit code 28 is returned if no changes are required and should not be considered a failure. Source-Author: Jeff Ching <[email protected]> Source-Date: Mon Nov 9 14:51:28 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 7db8a6c5ffb12a6e4c2f799c18f00f7f3d60e279 Source-Link: googleapis/synthtool@7db8a6c * build(java): use code-suggester v1 tag rather than full semver Source-Author: Jeff Ching <[email protected]> Source-Date: Thu Nov 12 10:07:08 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 7d652819519dfa24da9e14548232e4aaba71a11c Source-Link: googleapis/synthtool@7d65281 * build(java): use actions/github-script v3 tag rather than full semver So that renovate-bot stops proposing PRs like [this](https://github.com/googleapis/java-bigquerydatatransfer/pull/515/files)! Source-Author: Stephanie Wang <[email protected]> Source-Date: Tue Nov 17 13:02:02 2020 -0500 Source-Repo: googleapis/synthtool Source-Sha: 7fcc405a579d5d53a726ff3da1b7c8c08f0f2d58 Source-Link: googleapis/synthtool@7fcc405 * chore(java): retry staging portion of the release with backoff * chore(java): retry staging portion of the release with backoff * build: skip logging the retried command - it may contain secrets Source-Author: Jeff Ching <[email protected]> Source-Date: Thu Nov 19 12:24:02 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 68742c6016bf0d16948a572633d17955a8737414 Source-Link: googleapis/synthtool@68742c6 * build(java): enable blunderbuss for samples Source-Author: Jeff Ching <[email protected]> Source-Date: Mon Nov 23 10:23:01 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 943bb78606d035001123030067dffcc34f4645f2 Source-Link: googleapis/synthtool@943bb78 * chore: Update publish_javadoc.sh * (chore) Update publish_javadoc.sh * add `export` to the envVar's to make sure maven knows about them. * report the Java version * chore: update script * remove `java -version` Co-authored-by: Jeffrey Rennie <[email protected]> Source-Author: Les Vogel <[email protected]> Source-Date: Mon Dec 7 09:37:58 2020 -0800 Source-Repo: googleapis/synthtool Source-Sha: 5d11bd2888c38ce1fb6fa6bf25494a4219a73928 Source-Link: googleapis/synthtool@5d11bd2
googleapis · Dec 8, 2020 · b9ca69a · b9ca69a
1 parent 06de2c5
commit b9ca69a
Show file tree

Hide file tree

Showing 19 changed files with 349 additions and 73 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -4,5 +4,7 @@
 # For syntax help see:
 # https://help.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners#codeowners-syntax
 
+*                       @googleapis/yoshi-java
+
 # The java-samples-reviewers team is the default owner for samples changes
 samples/**/*.java       @googleapis/java-samples-reviewers
diff --git a/.github/blunderbuss.yml b/.github/blunderbuss.yml
@@ -0,0 +1,7 @@
+# Configuration for the Blunderbuss GitHub app. For more info see
+# https://github.com/googleapis/repo-automation-bots/tree/master/packages/blunderbuss
+assign_prs_by:
+- labels:
+  - samples
+  to:
+  - googleapis/java-samples-reviewers
diff --git a/.github/generated-files-bot.yml b/.github/generated-files-bot.yml
@@ -0,0 +1,7 @@
+externalManifests:
+- type: json
+  file: 'synth.metadata'
+  jsonpath: '$.generatedFiles[*]'
+- type: json
+  file: '.github/readme/synth.metadata/synth.metadata'
+  jsonpath: '$.generatedFiles[*]'
diff --git a/.github/readme/synth.py b/.github/readme/synth.py
@@ -0,0 +1,19 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""This script is used to synthesize generated the README for this library."""
+
+from synthtool.languages import java
+
+java.custom_templates(["java_library/README.md"])
diff --git a/.github/snippet-bot.yml b/.github/snippet-bot.yml
diff --git a/.github/workflows/approve-readme.yaml b/.github/workflows/approve-readme.yaml
@@ -0,0 +1,54 @@
+on:
+  pull_request:
+name: auto-merge-readme
+jobs:
+  approve:
+    runs-on: ubuntu-latest
+    if: github.repository_owner == 'googleapis' && github.head_ref == 'autosynth-readme'
+    steps:
+      - uses: actions/github-script@v3
+        with:
+          github-token: ${{secrets.YOSHI_APPROVER_TOKEN}}
+          script: |
+            // only approve PRs from yoshi-automation
+            if (context.payload.pull_request.user.login !== "yoshi-automation") {
+              return;
+            }
+
+            // only approve PRs like "chore: release <release version>"
+            if (!context.payload.pull_request.title === "chore: regenerate README") {
+              return;
+            }
+
+            // only approve PRs with README.md and synth.metadata changes
+            const files = new Set(
+              (
+                await github.paginate(
+                  github.pulls.listFiles.endpoint({
+                    owner: context.repo.owner,
+                    repo: context.repo.repo,
+                    pull_number: context.payload.pull_request.number,
+                  })
+                )
+              ).map(file => file.filename)
+            );
+            if (files.size != 2 || !files.has("README.md") || !files.has(".github/readme/synth.metadata/synth.metadata")) {
+              return;
+            }
+
+            // approve README regeneration PR
+            await github.pulls.createReview({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              body: 'Rubber stamped PR!',
+              pull_number: context.payload.pull_request.number,
+              event: 'APPROVE'
+            });
+
+            // attach automerge label
+            await github.issues.addLabels({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.payload.pull_request.number,
+              labels: ['automerge']
+            });
diff --git a/.github/workflows/auto-release.yaml b/.github/workflows/auto-release.yaml
@@ -0,0 +1,88 @@
+on:
+  pull_request:
+name: auto-release
+jobs:
+  approve:
+    runs-on: ubuntu-latest
+    if: contains(github.head_ref, 'release-v')
+    steps:
+    - uses: actions/github-script@v3
+      with:
+        github-token: ${{secrets.YOSHI_APPROVER_TOKEN}}
+        debug: true
+        script: |
+          // only approve PRs from release-please[bot]
+          if (context.payload.pull_request.user.login !== "release-please[bot]") {
+            return;
+          }
+
+          // only approve PRs like "chore: release <release version>"
+          if ( !context.payload.pull_request.title.startsWith("chore: release") ) {
+            return;
+          }
+
+          // only approve PRs with pom.xml and versions.txt changes
+          const filesPromise = github.pulls.listFiles.endpoint({
+            owner: context.repo.owner,
+            repo: context.repo.repo,
+            pull_number: context.payload.pull_request.number,
+          });
+          const changed_files = await github.paginate(filesPromise)
+
+          if ( changed_files.length < 1 ) {
+            console.log( "Not proceeding since PR is empty!" )
+            return;
+          }
+
+          if ( !changed_files.some(v => v.filename.includes("pom")) || !changed_files.some(v => v.filename.includes("versions.txt")) ) {
+            console.log( "PR file changes do not have pom.xml or versions.txt -- something is wrong. PTAL!" )
+            return;
+          }
+
+          // trigger auto-release when
+          // 1) it is a SNAPSHOT release (auto-generated post regular release)
+          // 2) there are dependency updates only
+          // 3) there are no open dependency update PRs in this repo (to avoid multiple releases)
+          if (
+            context.payload.pull_request.body.includes("Fix") ||
+            context.payload.pull_request.body.includes("Build") ||
+            context.payload.pull_request.body.includes("Documentation") ||
+            context.payload.pull_request.body.includes("BREAKING CHANGES") ||
+            context.payload.pull_request.body.includes("Features")
+          ) {
+            console.log( "Not auto-releasing since it is not a dependency-update-only release." );
+            return;
+          }
+
+          const promise = github.pulls.list.endpoint({
+            owner: context.repo.owner,
+            repo: context.repo.repo,
+            state: 'open'
+          });
+          const open_pulls = await github.paginate(promise)
+
+          if ( open_pulls.length > 1 && !context.payload.pull_request.title.includes("SNAPSHOT") ) {
+            for ( const pull of open_pulls ) {
+              if ( pull.title.startsWith("deps: update dependency") ) {
+                console.log( "Not auto-releasing yet since there are dependency update PRs open in this repo." );
+                return;
+              }
+            }
+          }
+
+          // approve release PR
+          await github.pulls.createReview({
+            owner: context.repo.owner,
+            repo: context.repo.repo,
+            body: 'Rubber stamped release!',
+            pull_number: context.payload.pull_request.number,
+            event: 'APPROVE'
+          });
+
+          // attach kokoro:force-run and automerge labels
+          await github.issues.addLabels({
+            owner: context.repo.owner,
+            repo: context.repo.repo,
+            issue_number: context.payload.pull_request.number,
+            labels: ['kokoro:force-run', 'automerge']
+          });
diff --git a/.github/workflows/formatting.yaml b/.github/workflows/formatting.yaml
@@ -0,0 +1,25 @@
+on:
+  pull_request_target:
+    types: [opened, synchronize]
+    branches:
+      - master
+name: format
+jobs:
+  format-code:
+    runs-on: ubuntu-latest
+    env:
+      ACCESS_TOKEN: ${{ secrets.YOSHI_CODE_BOT_TOKEN }}
+    steps:
+      - uses: actions/checkout@v2
+        with:
+          ref: ${{github.event.pull_request.head.ref}}
+          repository: ${{github.event.pull_request.head.repo.full_name}}
+      - uses: actions/setup-java@v1
+        with:
+          java-version: 11
+      - run: "mvn com.coveo:fmt-maven-plugin:format"
+      - uses: googleapis/code-suggester@v1
+        with:
+          command: review
+          pull_number: ${{ github.event.pull_request.number }}
+          git_dir: '.'
diff --git a/.github/workflows/samples.yaml b/.github/workflows/samples.yaml
@@ -2,7 +2,7 @@ on:
   pull_request:
 name: samples
 jobs:
-  lint:
+  checkstyle:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v2

diff --git a/.kokoro/build.sh b/.kokoro/build.sh
@@ -69,9 +69,16 @@ integration)
     RETURN_CODE=$?
     ;;
 samples)
-    if [[ -f samples/pom.xml ]]
+    SAMPLES_DIR=samples
+    # only run ITs in snapshot/ on presubmit PRs. run ITs in all 3 samples/ subdirectories otherwise.
+    if [[ ! -z ${KOKORO_GITHUB_PULL_REQUEST_NUMBER} ]]
     then
-        pushd samples
+      SAMPLES_DIR=samples/snapshot
+    fi
+
+    if [[ -f ${SAMPLES_DIR}/pom.xml ]]
+    then
+        pushd ${SAMPLES_DIR}
         mvn -B \
           -Penable-samples \
           -DtrimStackTrace=false \

diff --git a/.kokoro/common.sh b/.kokoro/common.sh
@@ -25,7 +25,6 @@ function retry_with_backoff {
 
     # allow a failures to continue
     set +e
-    echo "${command}"
     ${command}
     exit_code=$?
 

diff --git a/.kokoro/continuous/readme.cfg b/.kokoro/continuous/readme.cfg
@@ -0,0 +1,55 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Format: //devtools/kokoro/config/proto/build.proto
+
+env_vars: {
+  key: "TRAMPOLINE_IMAGE"
+  value: "gcr.io/cloud-devrel-kokoro-resources/python-multi"
+}
+
+env_vars: {
+  key: "TRAMPOLINE_BUILD_FILE"
+  value: "github/java-compute/.kokoro/readme.sh"
+}
+
+# Build logs will be here
+action {
+  define_artifacts {
+    regex: "**/*sponge_log.xml"
+    regex: "**/*sponge_log.log"
+  }
+}
+
+# The github token is stored here.
+before_action {
+  fetch_keystore {
+    keystore_resource {
+      keystore_config_id: 73713
+      keyname: "yoshi-automation-github-key"
+      # TODO(theacodes): remove this after secrets have globally propagated
+      backend_type: FASTCONFIGPUSH
+    }
+  }
+}
+
+# Common env vars for all repositories and builds.
+env_vars: {
+    key: "GITHUB_USER"
+    value: "yoshi-automation"
+}
+env_vars: {
+    key: "GITHUB_EMAIL"
+    value: "[email protected]"
+}
diff --git a/.kokoro/readme.sh b/.kokoro/readme.sh
@@ -0,0 +1,45 @@
+#!/bin/bash
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -eo pipefail
+
+cd ${KOKORO_ARTIFACTS_DIR}/github/java-compute
+
+# Disable buffering, so that the logs stream through.
+export PYTHONUNBUFFERED=1
+
+# Kokoro exposes this as a file, but the scripts expect just a plain variable.
+export GITHUB_TOKEN=$(cat ${KOKORO_KEYSTORE_DIR}/73713_yoshi-automation-github-key)
+
+# Setup git credentials
+echo "https://${GITHUB_TOKEN}:@github.com" >> ~/.git-credentials
+git config --global credential.helper 'store --file ~/.git-credentials'
+
+python3.6 -m pip install git+https://github.com/googleapis/synthtool.git#egg=gcp-synthtool
+
+set +e
+python3.6 -m autosynth.synth \
+    --repository=googleapis/java-compute \
+    --synth-file-name=.github/readme/synth.py \
+    --metadata-path=.github/readme/synth.metadata \
+    --pr-title="chore: regenerate README" \
+    --branch-suffix="readme"
+
+# autosynth returns 28 to signal there are no changes
+RETURN_CODE=$?
+if [[ ${RETURN_CODE} -ne 0 && ${RETURN_CODE} -ne 28 ]]
+then
+    exit ${RETURN_CODE}
+fi
diff --git a/.kokoro/release/publish_javadoc.cfg b/.kokoro/release/publish_javadoc.cfg
@@ -9,7 +9,7 @@ env_vars: {
 
 env_vars: {
   key: "STAGING_BUCKET_V2"
-  value: "docs-staging-v2-staging"
+  value: "docs-staging-v2"
   # Production will be at: docs-staging-v2
 }
 

diff --git a/.kokoro/release/publish_javadoc.sh b/.kokoro/release/publish_javadoc.sh
@@ -38,8 +38,8 @@ python3 -m pip install gcp-docuploader
 # compile all packages
 mvn clean install -B -q -DskipTests=true
 
-NAME=google-cloud-compute
-VERSION=$(grep ${NAME}: versions.txt | cut -d: -f3)
+export NAME=google-cloud-compute
+export VERSION=$(grep ${NAME}: versions.txt | cut -d: -f3)
 
 # build the docs
 mvn site -B -q
@@ -59,10 +59,11 @@ python3 -m docuploader upload . \
 
 popd
 
-# V2
+# V2 due to problems w/ the released javadoc plugin doclava, Java 8 is required.  Beware of accidental updates.
+
 mvn clean site -B -q -Ddevsite.template="${KOKORO_GFILE_DIR}/java/"
 
-pushd target/devsite
+pushd target/devsite/reference
 
 # create metadata
 python3 -m docuploader create-metadata \