Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Cleanup CI #1117

Merged
merged 72 commits into from
Aug 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
36c2998
wip
AlejandroCabeza Jun 5, 2024
e5b11a5
Deduplicate workflows, cleaner naming
AlejandroCabeza Jun 6, 2024
08bff89
Fix empty include/exclude on workflow
AlejandroCabeza Jun 6, 2024
95f20ef
Fix wrong json.
AlejandroCabeza Jun 6, 2024
3502c88
Fix input default value on non-call
AlejandroCabeza Jun 6, 2024
c146f75
Fix tests job naming
AlejandroCabeza Jun 6, 2024
2d21d54
Add some comments, split coverage by steps
AlejandroCabeza Jun 6, 2024
cb70494
Remove nim2 from base testing
AlejandroCabeza Jun 6, 2024
2e103b0
Update actions versions
AlejandroCabeza Jun 7, 2024
e7fb13b
Add memory management flag to test matrix, naming.
AlejandroCabeza Jun 10, 2024
0372da3
Fix nim_branch object keys.
AlejandroCabeza Jun 10, 2024
9e9ea68
Remove ci.yml again, add nim version 2.0 for base_tests.yml
AlejandroCabeza Jul 10, 2024
9401ab7
Move code formatter to different workflow
AlejandroCabeza Jul 10, 2024
0d692e7
Update dependencies workflow comments.
AlejandroCabeza Jul 17, 2024
1f402c3
Add comment for clarity.
AlejandroCabeza Jul 17, 2024
e6a1b65
Rename formatter to linter, and correct names.
AlejandroCabeza Jul 17, 2024
e016767
Only run linter on PR.
AlejandroCabeza Jul 17, 2024
577f6ec
Remove commented out code.
AlejandroCabeza Jul 17, 2024
b772766
Enable codecov on all PRs.
AlejandroCabeza Jul 17, 2024
d0ef28f
Remove CI yet again.
AlejandroCabeza Jul 17, 2024
761dd4e
Undo dependencies target ref misshap.
AlejandroCabeza Jul 17, 2024
bec1611
Remove unused step from codecov.
AlejandroCabeza Jul 17, 2024
c4bfddf
Improve naming on dependencies workflow.
AlejandroCabeza Jul 17, 2024
aa361b4
Temporarily enable daily runs on pull request event.
AlejandroCabeza Jul 18, 2024
b322115
Add concurrency groups for daily jobs
AlejandroCabeza Jul 18, 2024
d526d51
wip
AlejandroCabeza Jul 18, 2024
1cc7d86
wip
AlejandroCabeza Jul 18, 2024
ecb6e20
wip
AlejandroCabeza Jul 18, 2024
e3e2be5
wip
AlejandroCabeza Jul 18, 2024
e5fccf1
wip
AlejandroCabeza Jul 18, 2024
474d282
wip
AlejandroCabeza Jul 18, 2024
43133fa
wip
AlejandroCabeza Jul 18, 2024
02939c5
wip
AlejandroCabeza Jul 18, 2024
629e762
wip
AlejandroCabeza Jul 18, 2024
f9dd31e
wip
AlejandroCabeza Jul 19, 2024
e7f551f
wip
AlejandroCabeza Jul 19, 2024
dd21a3e
wip
AlejandroCabeza Jul 19, 2024
90fcc81
wip
AlejandroCabeza Jul 19, 2024
47a5ef8
wip
AlejandroCabeza Jul 19, 2024
29e2868
wip
AlejandroCabeza Jul 19, 2024
57c5786
wip
AlejandroCabeza Jul 19, 2024
c731116
wip
AlejandroCabeza Jul 19, 2024
8b08e3c
wip
AlejandroCabeza Jul 19, 2024
ac0265c
wip
AlejandroCabeza Jul 19, 2024
a36dcc0
wip
AlejandroCabeza Jul 19, 2024
142e87e
wip
AlejandroCabeza Jul 19, 2024
e10976b
wip
AlejandroCabeza Jul 19, 2024
943214b
wip
AlejandroCabeza Jul 19, 2024
ed0eada
wip
AlejandroCabeza Jul 19, 2024
249aba9
wip
AlejandroCabeza Jul 19, 2024
9bd17f9
wip
AlejandroCabeza Jul 19, 2024
6be966d
wip
AlejandroCabeza Jul 19, 2024
2409e9e
wip
AlejandroCabeza Jul 19, 2024
6f1c2d9
wip
AlejandroCabeza Jul 19, 2024
2b4dee4
wip
AlejandroCabeza Jul 19, 2024
e06d60e
wip
AlejandroCabeza Jul 19, 2024
7afc435
wip
AlejandroCabeza Jul 19, 2024
5d20e01
wip
AlejandroCabeza Jul 19, 2024
614d357
wip
AlejandroCabeza Jul 22, 2024
18396f5
wip
AlejandroCabeza Jul 22, 2024
31278c7
wip
AlejandroCabeza Jul 22, 2024
c03f4e5
wip
AlejandroCabeza Jul 22, 2024
11bd9de
wip
AlejandroCabeza Jul 22, 2024
c712daf
wip
AlejandroCabeza Jul 22, 2024
6f6a121
wip
AlejandroCabeza Jul 22, 2024
ad3eb64
wip
AlejandroCabeza Jul 22, 2024
2a1d38e
wip
AlejandroCabeza Jul 23, 2024
c5a3632
wip
AlejandroCabeza Jul 26, 2024
183c6a6
Merge branch 'master' into feature/cleanup-ci
AlejandroCabeza Aug 1, 2024
e9b3d24
Fix wrong keys.
AlejandroCabeza Aug 2, 2024
fd32a75
wip
AlejandroCabeza Aug 2, 2024
5d5af4f
Merge branch 'master' into feature/cleanup-ci
kaiserd Aug 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/actions/install_nim/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ runs:
- name: Restore Nim DLLs dependencies (Windows) from cache
if: inputs.os == 'Windows'
id: windows-dlls-cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: external/dlls
key: 'dlls'
Expand Down Expand Up @@ -114,7 +114,7 @@ runs:

- name: Restore Nim from cache
id: nim-cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: '${{ github.workspace }}/nim'
key: ${{ inputs.os }}-${{ inputs.cpu }}-nim-${{ inputs.nim_branch }}-cache-${{ env.cache_nonce }}
Expand Down
65 changes: 24 additions & 41 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: CI
name: Continuous Integration

on:
push:
branches:
Expand All @@ -11,37 +12,41 @@ concurrency:
cancel-in-progress: true

jobs:
build:
test:
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
target:
platform:
- os: linux
cpu: amd64
- os: linux
cpu: i386
- os: ubuntu-24-gcc-14
- os: linux-gcc-14
cpu: amd64
- os: macos
cpu: amd64
- os: windows
cpu: amd64
branch: [version-1-6, version-2-0]
nim:
- branch: version-1-6
memory_management: refc
- branch: version-2-0
memory_management: refc
include:
- target:
- platform:
os: linux
builder: ubuntu-22.04
shell: bash
- target:
os: ubuntu-24-gcc-14
- platform:
os: linux-gcc-14
builder: ubuntu-24.04
shell: bash
- target:
- platform:
os: macos
builder: macos-13
shell: bash
- target:
- platform:
os: windows
builder: windows-2022
shell: msys2 {0}
Expand All @@ -50,25 +55,24 @@ jobs:
run:
shell: ${{ matrix.shell }}

name: '${{ matrix.target.os }}-${{ matrix.target.cpu }} (Nim ${{ matrix.branch }})'
name: '${{ matrix.platform.os }}-${{ matrix.platform.cpu }} (Nim ${{ matrix.nim.branch }})'
runs-on: ${{ matrix.builder }}
continue-on-error: ${{ matrix.branch == 'devel' }}
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
submodules: true

- name: Setup Nim
uses: "./.github/actions/install_nim"
with:
os: ${{ matrix.target.os }}
cpu: ${{ matrix.target.cpu }}
os: ${{ matrix.platform.os }}
cpu: ${{ matrix.platform.cpu }}
shell: ${{ matrix.shell }}
nim_branch: ${{ matrix.branch }}
nim_branch: ${{ matrix.nim.branch }}

- name: Setup Go
uses: actions/setup-go@v2
uses: actions/setup-go@v5
with:
go-version: '~1.15.5'

Expand All @@ -83,15 +87,15 @@ jobs:
path: nimbledeps
# Using nim.branch as a simple way to differentiate between nimble using the "pkgs" or "pkgs2" directories.
# The change happened on Nimble v0.14.0.
key: nimbledeps-${{ matrix.branch }}-${{ hashFiles('.pinned') }} # hashFiles returns a different value on windows
key: nimbledeps-${{ matrix.nim.branch }}-${{ hashFiles('.pinned') }} # hashFiles returns a different value on windows

- name: Install deps
if: ${{ steps.deps-cache.outputs.cache-hit != 'true' }}
run: |
nimble install_pinned

- name: Use gcc 14
if : ${{ matrix.target.os == 'ubuntu-24-gcc-14'}}
if : ${{ matrix.platform.os == 'linux-gcc-14'}}
run: |
# Add GCC-14 to alternatives
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-14 14
Expand All @@ -104,26 +108,5 @@ jobs:
nim --version
nimble --version
gcc --version
NIMFLAGS="${NIMFLAGS} --mm:${{ matrix.nim.memory_management }}"
nimble test

lint:
name: "Lint"
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2 # In PR, has extra merge commit: ^1 = PR, ^2 = base

- name: Check nph formatting
# Pin nph to a specific version to avoid sudden style differences.
# Updating nph version should be accompanied with running the new
# version on the fluffy directory.
run: |
VERSION="v0.5.1"
ARCHIVE="nph-linux_x64.tar.gz"
curl -L "https://github.com/arnetheduck/nph/releases/download/${VERSION}/${ARCHIVE}" -o ${ARCHIVE}
tar -xzf ${ARCHIVE}
shopt -s extglob # Enable extended globbing
./nph examples libp2p tests tools *.@(nim|nims|nimble)
git diff --exit-code
27 changes: 16 additions & 11 deletions .github/workflows/codecov.yml → .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name: nim-libp2p codecov builds
name: Coverage

on:
#On push to common branches, this computes the "bases stats" for PRs
# On push to common branches, this computes the coverage that PRs will use for diff
push:
branches:
- master
Expand All @@ -13,12 +13,13 @@ concurrency:
cancel-in-progress: true

jobs:
Coverage:
codecov:
name: Run coverage and upload to codecov
runs-on: ubuntu-22.04
env:
CICOV: YES
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0

Expand All @@ -31,7 +32,7 @@ jobs:

- name: Restore deps from cache
id: deps-cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: nimbledeps
key: nimbledeps-${{ hashFiles('.pinned') }}
Expand All @@ -41,24 +42,28 @@ jobs:
run: |
nimble install_pinned

- name: Run
- name: Setup coverage
run: |
sudo apt-get update
sudo apt-get install -y lcov build-essential git curl
mkdir coverage

- name: Run test suite with coverage flags
run: |
export NIMFLAGS="--lineDir:on --passC:-fprofile-arcs --passC:-ftest-coverage --passL:-fprofile-arcs --passL:-ftest-coverage"
nimble testnative
nimble testpubsub
nimble testfilter

- name: Run coverage
run: |
find nimcache -name *.c -delete
lcov --capture --directory nimcache --output-file coverage/coverage.info
shopt -s globstar
ls `pwd`/libp2p/{*,**/*}.nim
lcov --extract coverage/coverage.info `pwd`/libp2p/{*,**/*}.nim --output-file coverage/coverage.f.info
genhtml coverage/coverage.f.info --output-directory coverage/output
bash <(curl -s https://codecov.io/bash) -f coverage/coverage.f.info || echo "Codecov did not collect coverage reports"

#- uses: actions/upload-artifact@master
# with:
# name: coverage
# path: coverage
- name: Upload coverage to codecov
run: |
bash <(curl -s https://codecov.io/bash) -f coverage/coverage.f.info || echo "Codecov did not collect coverage reports"
12 changes: 0 additions & 12 deletions .github/workflows/daily.yml

This file was deleted.

14 changes: 14 additions & 0 deletions .github/workflows/daily_amd64.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Daily amd64

on:
schedule:
- cron: "30 6 * * *"
workflow_dispatch:

jobs:
test_amd64:
name: Daily amd64
uses: ./.github/workflows/base_daily_tests.yml
with:
nim: "[{'branch': 'version-1-6', 'memory_management': 'refc'}, {'branch': 'version-2-0', 'memory_management': 'refc'}]"
cpu: "['amd64']"
43 changes: 24 additions & 19 deletions .github/workflows/daily_common.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
name: daily-common
name: Daily Common
# Serves as base workflow for daily tasks, it's not run by itself.

on:
workflow_call:
inputs:
nim-branch:
description: 'Nim branch'
nim:
description: 'Nim Configuration'
required: true
type: string
type: string # Following this format: [{"branch": ..., "memory_management": ...}, ...]
cpu:
description: 'CPU'
required: true
Expand All @@ -17,15 +18,20 @@ on:
type: string
default: "[]"

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
delete-cache:
delete_cache:
name: Delete github action's branch cache
runs-on: ubuntu-latest
steps:
- uses: snnaplab/delete-branch-cache-action@v1

build:
needs: delete-cache
timeout-minutes: 120
test:
needs: delete_cache
timeout-minutes: 90
strategy:
fail-fast: false
matrix:
Expand All @@ -39,17 +45,17 @@ jobs:
- os: windows
builder: windows-2022
shell: msys2 {0}
branch: ${{ fromJSON(inputs.nim-branch) }}
nim: ${{ fromJSON(inputs.nim) }}
cpu: ${{ fromJSON(inputs.cpu) }}
exclude: ${{ fromJSON(inputs.exclude) }}

defaults:
run:
shell: ${{ matrix.platform.shell }}

name: '${{ matrix.platform.os }}-${{ matrix.cpu }} (Nim ${{ matrix.branch }})'
name: '${{ matrix.platform.os }}-${{ matrix.cpu }} (Nim ${{ matrix.nim.branch }})'
runs-on: ${{ matrix.platform.builder }}
continue-on-error: ${{ matrix.branch == 'devel' || matrix.branch == 'version-2-0' }}
continue-on-error: ${{ matrix.nim.branch == 'devel' || matrix.nim.branch == 'version-2-0' }}
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -59,26 +65,25 @@ jobs:
with:
os: ${{ matrix.platform.os }}
shell: ${{ matrix.platform.shell }}
nim_branch: ${{ matrix.branch }}
nim_branch: ${{ matrix.nim.branch }}
cpu: ${{ matrix.cpu }}

- name: Setup Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: '~1.15.5'
cache: false

- name: Install p2pd
run: |
V=1 bash scripts/build_p2pd.sh p2pdCache 124530a3

- name: Install dependencies
run: |
nimble install -y --depsOnly

- name: Run tests
run: |
nim --version
nimble --version
nimble install -y --depsOnly
NIMFLAGS="${NIMFLAGS} --mm:refc" nimble test
if [[ "${{ matrix.branch }}" == "devel" ]]; then
echo -e "\nTesting with '--mm:orc':\n"
NIMFLAGS="${NIMFLAGS} --mm:orc" nimble test
fi
NIMFLAGS="${NIMFLAGS} --mm:${{ matrix.nim.memory_management }}" nimble test
14 changes: 14 additions & 0 deletions .github/workflows/daily_devel.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Daily Nim Devel

on:
schedule:
- cron: "30 6 * * *"
workflow_dispatch:

jobs:
test_nim_devel:
name: Daily Nim Devel
uses: ./.github/workflows/base_daily_tests.yml
with:
nim: "[{'branch': 'devel', 'memory_management': 'orc'}]"
cpu: "['amd64']"
8 changes: 5 additions & 3 deletions .github/workflows/daily_i386.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
name: Daily i386

on:
schedule:
- cron: "30 6 * * *"
workflow_dispatch:

jobs:
call-multi-nim-common:
uses: ./.github/workflows/daily_common.yml
test_i386:
name: Daily i386 (Linux)
uses: ./.github/workflows/base_daily_tests.yml
with:
nim-branch: "['version-1-6','version-2-0', 'devel']"
nim: "[{'branch': 'version-1-6', 'memory_management': 'refc'}, {'branch': 'version-2-0', 'memory_management': 'refc'}, {'branch': 'devel', 'memory_management': 'orc'}]"
cpu: "['i386']"
exclude: "[{'platform': {'os':'macos'}}, {'platform': {'os':'windows'}}]"
12 changes: 0 additions & 12 deletions .github/workflows/daily_nim_devel.yml

This file was deleted.

Loading
Loading