Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Add separate cache for long-running tests #3293

Merged
merged 6 commits into from
Jul 21, 2023
Merged

🌱 Add separate cache for long-running tests #3293

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
276ec9d
Add separate cache for unit tests.
spencerschrock Jul 21, 2023
a16b0e7
share cache with gitlab tests too.
spencerschrock Jul 21, 2023
151b28e
share cache with github integration tests.
spencerschrock Jul 21, 2023
ab76f22
explicitly download modules in unit test job
spencerschrock Jul 21, 2023
52c878f
checkout needs to be before the go.mod is read.
spencerschrock Jul 21, 2023
701e044
checkout needs to be before the go.sum files are hashed.
spencerschrock Jul 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 20 additions & 4 deletions .github/workflows/gitlab.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ on:
branches:
- main

env:
GO_VERSION_FILE: go.mod # no good way of getting a mutual version between go.mod and tools/go.mod

jobs:
gitlab-integration-trusted:
runs-on: ubuntu-latest
Expand All @@ -33,17 +36,30 @@ jobs:
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v2.4.1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- name: Clone the code
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }} # head SHA if PR, else fallback to push SHA

- name: setup-go
- name: Setup Go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1
with:
go-version: '1.19'
go-version-file: ${{ env.GO_VERSION_FILE }}
check-latest: true
cache: false # we manually manage caches below
- id: go-cache-paths
run: |
echo "go-build=$(go env GOCACHE)" >> "$GITHUB_OUTPUT"
echo "go-mod=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT"
- name: Cache builds
# https://github.com/mvdan/github-actions-golang#how-do-i-set-up-caching-between-builds
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 #v3.3.1
with:
path: |
${{ steps.go-cache-paths.outputs.go-build }}
${{ steps.go-cache-paths.outputs.go-mod }}
key: ${{ runner.os }}-go-tests-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-tests-

- name: Prepare test env
run: |
Expand Down
36 changes: 26 additions & 10 deletions .github/workflows/integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ on:
permissions:
contents: read

env:
GO_VERSION_FILE: go.mod # no good way of getting a mutual version between go.mod and tools/go.mod

jobs:
approve:
runs-on: ubuntu-latest
Expand All @@ -41,27 +44,40 @@ jobs:
needs: [approve]
steps:
- name: Harden Runner
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v1
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v2.4.1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- name: pull_request actions/checkout
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v2.3.4
- name: Clone the code
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
with:
ref: ${{ github.event.pull_request.head.sha }}

- name: setup-go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v2.2.0
- name: Setup Go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1
with:
go-version: '1.19'
go-version-file: ${{ env.GO_VERSION_FILE }}
check-latest: true
cache: false # we manually manage caches below
- id: go-cache-paths
run: |
echo "go-build=$(go env GOCACHE)" >> "$GITHUB_OUTPUT"
echo "go-mod=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT"
- name: Cache builds
# https://github.com/mvdan/github-actions-golang#how-do-i-set-up-caching-between-builds
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 #v3.3.1
with:
path: |
${{ steps.go-cache-paths.outputs.go-build }}
${{ steps.go-cache-paths.outputs.go-mod }}
key: ${{ runner.os }}-go-tests-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-tests-

- name: Prepare test env
run: |
go mod download

- name: Run GITHUB_TOKEN E2E #using retry because the GitHub token is being throttled.
uses: nick-invision/retry@943e742917ac94714d2f408a0e8320f2d1fcafcd
uses: nick-invision/retry@943e742917ac94714d2f408a0e8320f2d1fcafcd # v2.8.3
env:
GITHUB_AUTH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
Expand All @@ -71,7 +87,7 @@ jobs:
command: make e2e-gh-token

- name: codecov
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # 2.1.0
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # 3.1.4
with:
files: "*e2e-coverage.out"
verbose: true
40 changes: 21 additions & 19 deletions .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,36 +37,38 @@ jobs:
contents: read
steps:
- name: Harden Runner
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v1
uses: step-security/harden-runner@55d479fb1c5bcad5a4f9099a5d9f37c8857b2845 # v2.4.1
with:
egress-policy: audit # TODO: change to 'egress-policy: block' after couple of runs

- name: Clone the code
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
- name: Setup Go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v4.0.1
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true
cache: false # we manually manage caches below
- id: go-cache-paths
run: |
echo "go-build=$(go env GOCACHE)" >> "$GITHUB_OUTPUT"
echo "go-mod=$(go env GOMODCACHE)" >> "$GITHUB_OUTPUT"
- name: Cache builds
# https://github.com/mvdan/github-actions-golang#how-do-i-set-up-caching-between-builds
uses: actions/cache@88522ab9f39a2ea568f7027eddc7d8d8bc9d59c8 #v3.3.1
with:
path: |
~/go/pkg/mod
~/.cache/go-build
~/Library/Caches/go-build
%LocalAppData%\go-build
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
${{ steps.go-cache-paths.outputs.go-build }}
${{ steps.go-cache-paths.outputs.go-mod }}
key: ${{ runner.os }}-go-tests-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: Clone the code
uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v2.3.4
with:
fetch-depth: 0
- name: Setup Go
uses: actions/setup-go@fac708d6674e30b6ba41289acaab6d4b75aa0753 # v2.2.0
with:
go-version: ${{ env.GO_VERSION }}
check-latest: true
cache: true
${{ runner.os }}-go-tests-
- name: Prepare test env
run: |
go mod download
- name: Run unit-tests
run: make unit-test
- name: Upload codecoverage
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # 2.1.0
uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # 3.1.4
with:
files: ./unit-coverage.out
verbose: true
Expand Down