Skip to content

Commit

Permalink
Try enabling build cache (#5141)
Browse files Browse the repository at this point in the history
Motivation:

This PR attempts to solve two problems:
- Our CI runs don't necessarily reflect a typical developer's workflow
because it is run with specific flags (i.e. `flakyTests`, `noLint`,
etc..). For this reason, the populated build caches does not necessarily
coincide with that of a developer workflow.
- Our CI does not upload build-cache to our remote node yet.

In this PR, I propose that we launch a separate build task to populate
the build cache for each PR.
Additionally, I propose that we retry tests in this task so that flaky
tests don't affect build cache upload.

Note that I would like to do some more experiments before actually
enabling caching for everyone. (It can be done manually by setting the
`--build-cache` flag on the CLI)
For this reason, in this PR I don't enable the `org.gradle.caching=true`
option yet. This will be done after experiments are complete.

See the following links for a preview:
- jrhee17#27
-
https://github.com/jrhee17/armeria/actions/runs/5960198011/job/16167108190

Modifications:

- Set up credentials for cache upload. These credentials can be
registered at:
  - https://ge.armeria.dev/cache-admin/node/built-in
  - Cache access control
- Set up remote build cache upload configurations. Note that credentials
are only required for writing, as reading can be done anonymously. The
cache will be pushed only at the CI if the proper write credentials are
supplied (i.e. writing to the `main` branch)
- Note that the `gradle-enterprise-postjob.yml` task must be merged to
the `main` branch to take effect.
- Introduced a `failOnPassedAfterRetry` to prevent build cache uploads
tasks from failing due to flaky tests.
- Renamed the github task to `gradle-enterprise-postjob.yml` to better
reflect the new role.

Result:

- Build cache is uploaded from CI for each PR/push to `main` branch

<!--
Visit this URL to learn more about how to write a pull request
description:

https://armeria.dev/community/developer-guide#how-to-write-pull-request-description
-->
  • Loading branch information
jrhee17 authored Aug 25, 2023
1 parent 5f9c1b6 commit ee42213
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 3 deletions.
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Upload Gradle build scans to ge.armeria.dev
name: Upload artifacts to ge.armeria.dev

on:
workflow_run:
Expand All @@ -8,6 +8,7 @@ on:

env:
LC_ALL: "en_US.UTF-8"
BUILD_JDK_VERSION: "19"
GRADLE_ENTERPRISE_ACCESS_KEY: ${{ secrets.GRADLE_ENTERPRISE_ACCESS_KEY }}
RUN_ID: ${{ github.event.workflow_run.id }}
COMMIT_SHA: ${{ github.event.workflow_run.head_sha }}
Expand Down Expand Up @@ -105,3 +106,34 @@ jobs:
env:
BUILD_SCANS: ${{ steps.upload-build-scans.outputs.BUILD_SCANS }}
PR_NUMBER: ${{ steps.get-pr-number.outputs.PR_NUMBER }}

upload-build-cache:
runs-on: ${{ matrix.on }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
on: [ self-hosted, macos-12, windows-latest ]
steps:
- uses: actions/checkout@v3
with:
ref: ${{ env.COMMIT_SHA }}

- id: setup-build-jdk
name: Set up build JDK ${{ env.BUILD_JDK_VERSION }}
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: ${{ env.BUILD_JDK_VERSION }}

- name: Setup Gradle
uses: gradle/gradle-build-action@v2

- name: Build with Gradle
run: |
./gradlew --no-daemon --stacktrace --build-cache build \
${{ (matrix.on == 'self-hosted') && '--max-workers=8' || '--max-workers=2' }} --parallel \
-PbuildJdkVersion=${{ env.BUILD_JDK_VERSION }} \
-Pretry=true -PfailOnPassedAfterRetry=false \
-Porg.gradle.java.installations.paths=${{ steps.setup-build-jdk.outputs.path }}
shell: bash
2 changes: 1 addition & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ allprojects {
retry {
if (rootProject.findProperty('retry') == 'true') {
maxRetries = 3
failOnPassedAfterRetry = true
failOnPassedAfterRetry = rootProject.findProperty('failOnPassedAfterRetry') != 'false'
}
}
}
Expand Down
9 changes: 8 additions & 1 deletion settings.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,11 @@ plugins {

import com.gradle.scan.plugin.PublishedBuildScan

def isCi = System.getenv("CI") != null

gradleEnterprise {
server = "https://ge.armeria.dev"
buildScan {
def isCi = System.getenv("CI") != null
publishIfAuthenticated()
publishAlways()
uploadInBackground = !isCi
Expand Down Expand Up @@ -58,6 +59,12 @@ gradleEnterprise {
}
}

buildCache {
remote(gradleEnterprise.buildCache) {
push = isCi
}
}

rootProject.name = 'armeria'

apply from: "${rootDir}/gradle/scripts/settings-flags.gradle"
Expand Down

0 comments on commit ee42213

Please sign in to comment.