Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: switch to free runners #2319

Merged
merged 46 commits into from
Nov 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
24177fa
feat: hardcoded identity transfers in strategy tests
pauldelucia Nov 4, 2024
dacc6db
fix(drive): uncommitted state if db transaction fails (#2305)
shumkov Nov 4, 2024
306b86c
fix(drive): apply batch is not using transaction in `remove_all_votes…
QuantumExplorer Nov 4, 2024
99fe5fa
add comment
pauldelucia Nov 4, 2024
cee3098
Merge remote-tracking branch 'origin/v1.6-dev' into feat/hardcoded-id…
pauldelucia Nov 4, 2024
0d3e091
comment
pauldelucia Nov 5, 2024
e421514
use into_iter instead of iter
pauldelucia Nov 5, 2024
3d941ec
use current identities instead of hardcoded start identities
pauldelucia Nov 5, 2024
4bc0a65
let transfer keys be any security level or key type
pauldelucia Nov 5, 2024
dc48827
fix
pauldelucia Nov 5, 2024
cafda11
feat: hardcoded identity transfers in strategy tests (#2312)
pauldelucia Nov 5, 2024
b86f4e0
Merge branch 'v1.6-dev' of github.com:dashpay/platform into v1.6-dev
shumkov Nov 6, 2024
ae97f47
ci: run devcontainers workflow only on push to master (#2295)
shumkov Nov 6, 2024
48cca1a
ci: do not run test on push (#2308)
shumkov Nov 6, 2024
5c80069
ci: use prebuilt librocksdb in github actions (#2316)
lklimek Nov 8, 2024
2b9d715
ci: debug layer cache
shumkov Nov 8, 2024
42fdd64
ci: update docker/build-push-action@
shumkov Nov 8, 2024
fe231e2
ci: try gha cache
shumkov Nov 8, 2024
626a632
ci: pass push
shumkov Nov 8, 2024
5a9d214
ci: disable mount cache
shumkov Nov 8, 2024
a333049
build: uncomment code in Dockerfile
shumkov Nov 8, 2024
39e9d43
ci: store cargo and yarn cache mounts
shumkov Nov 8, 2024
d6cbbc2
ci: remove garbage
shumkov Nov 8, 2024
e31d501
ci: fix restore keys
shumkov Nov 8, 2024
63c3d50
ci: specify correct version
shumkov Nov 8, 2024
9ca6ffd
ci: fix expected JSON
shumkov Nov 8, 2024
698a8a8
build: remove random port
shumkov Nov 8, 2024
ba2e477
ci: switch build js to free runner
shumkov Nov 8, 2024
7531fc9
ci: cleanup
shumkov Nov 8, 2024
25c31a6
ci: use platform input for unplugged cache
shumkov Nov 11, 2024
01e09b6
ci: switch to free runners
shumkov Nov 11, 2024
20eed8a
ci: switch more jobs to free runners
shumkov Nov 11, 2024
3036336
Merge branch 'v1.6-dev' into ci/docker-layer-cache
shumkov Nov 11, 2024
3a8b9a1
build: fix sccache error
shumkov Nov 13, 2024
b5e9ea0
ci: remove prefix
shumkov Nov 13, 2024
9855b2a
ci: second try to remove prefix
shumkov Nov 13, 2024
3d70254
ci: fix dashmate path
shumkov Nov 13, 2024
33ad11b
ci: fix sed
shumkov Nov 13, 2024
fa5a188
ci: fix HOME var
shumkov Nov 13, 2024
633e2ab
refactor: remove unused params
shumkov Nov 13, 2024
bd18a7e
ci: remove CHROME_BIN
shumkov Nov 13, 2024
ad83ebb
revert: unnecessary changes
shumkov Nov 13, 2024
738d1a5
Merge branch 'v1.6-dev' into ci/docker-layer-cache
shumkov Nov 13, 2024
a8bd8ff
revert: mount cache
shumkov Nov 13, 2024
df8eb06
Merge remote-tracking branch 'origin/ci/docker-layer-cache' into ci/d…
shumkov Nov 13, 2024
ca7017a
ci: fix home dir
shumkov Nov 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions .github/actions/local-network/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,10 @@ runs:
shell: bash
run: echo "sha=$(git log -1 --format="%h" -- packages/dashmate)" >> $GITHUB_OUTPUT

- name: Set HOME variable to github context
shell: bash
run: echo "HOME=$HOME" >> $GITHUB_ENV

# TODO: Use upload artifacts action instead
- name: Restore local network data
id: local-network-data
Expand All @@ -36,7 +40,7 @@ runs:
bucket: multi-runner-cache-x1xibo9c
root: local-network-data
path: |
/home/ubuntu/.dashmate
${{ env.HOME }}/.dashmate
**/.env
dashmate_volumes_dump
key: local-network-volumes/${{ steps.dashmate-fingerprint.outputs.sha }}
Expand Down Expand Up @@ -73,7 +77,7 @@ runs:
bucket: multi-runner-cache-x1xibo9c
root: local-network-data
path: |
/home/ubuntu/.dashmate
${{ env.HOME }}/.dashmate
**/.env
dashmate_volumes_dump
key: local-network-volumes/${{ steps.dashmate-fingerprint.outputs.sha }}
Expand All @@ -90,9 +94,9 @@ runs:
docker tag ${{ inputs.image_org }}/dashmate-helper:$SHA_TAG dashpay/dashmate-helper:$VERSION

# Replace DAPI and Drive images with new org and tag in dashmate config
sed -i -E "s/dashpay\/(drive|dapi):[^\"]+/${{ inputs.image_org }}\/\1:${SHA_TAG}/g" /home/ubuntu/.dashmate/config.json
sed -i -E "s/dashpay\/(drive|dapi):[^\"]+/${{ inputs.image_org }}\/\1:${SHA_TAG}/g" ${{ env.HOME }}/.dashmate/config.json

cat /home/ubuntu/.dashmate/config.json
cat ${{ env.HOME }}/.dashmate/config.json
shumkov marked this conversation as resolved.
Show resolved Hide resolved

- name: Start local network
shell: bash
Expand Down
10 changes: 7 additions & 3 deletions .github/actions/rust/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -95,16 +95,20 @@ runs:
shell: bash
run: echo "key=$(echo '${{ github.ref_name }}' | sha256sum | cut -d ' ' -f1)" >> $GITHUB_OUTPUT

- name: Set HOME variable to github context
shell: bash
run: echo "HOME=$HOME" >> $GITHUB_ENV

- name: Cache cargo registry (S3 bucket cache)
uses: strophy/actions-cache@opendal-update
if: inputs.cache == 'true'
with:
bucket: multi-runner-cache-x1xibo9c
root: actions-cache
path: |
/home/ubuntu/.cargo/registry/index
/home/ubuntu/.cargo/registry/cache
/home/ubuntu/.cargo/git
${{ env.HOME }}/.cargo/registry/index
${{ env.HOME }}/.cargo/registry/cache
${{ env.HOME }}/.cargo/git
key: ${{ runner.os }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}/cargo/registry/${{ hashFiles('**/Cargo.lock') }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/prebuild-devcontainers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ concurrency:
jobs:
build:
name: Build and push devcontainer
runs-on: ["self-hosted", "linux", "x64", "ubuntu-platform"]
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
- name: Checkout repo
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ concurrency:
jobs:
release-npm:
name: Release NPM packages
runs-on: ["self-hosted", "linux", "arm64", "ubuntu-platform"]
runs-on: ubuntu-24.04
timeout-minutes: 15
if: github.event_name != 'workflow_dispatch'
steps:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/tests-build-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
jobs:
build-image:
name: Build ${{ inputs.name }} image
runs-on: ["self-hosted", "linux", "arm64", "ubuntu-platform"]
runs-on: ubuntu-24.04
steps:
- name: Check out repo
uses: actions/checkout@v4
Expand All @@ -41,7 +41,7 @@ jobs:
image_name: ${{ inputs.image_name }}
image_org: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com
target: ${{ inputs.target }}
platform: linux/arm64
platform: linux/amd64
push_tags: true
dockerhub_username: ${{ secrets.DOCKERHUB_USERNAME }}
dockerhub_token: ${{ secrets.DOCKERHUB_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests-build-js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
jobs:
build-js:
name: Build JS
runs-on: ["self-hosted", "linux", "arm64", "ubuntu-platform"]
runs-on: ubuntu-24.04
steps:
- name: Configure AWS credentials and bucket region
uses: aws-actions/configure-aws-credentials@v4
Expand Down
12 changes: 8 additions & 4 deletions .github/workflows/tests-dashmate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ on:
jobs:
dashmate-test:
name: Run ${{ inputs.name }} tests
# TODO: Try with Github Runner, probably it will be the same time
runs-on: [ "self-hosted", "linux", "arm64", "ubuntu-platform" ]
runs-on: ubuntu-24.04
timeout-minutes: 15
steps:
- name: Check out repo
Expand All @@ -42,6 +41,7 @@ jobs:
- name: Setup Node.JS
uses: ./.github/actions/nodejs

# TODO: Use action artifacts
- name: Restore JS build artifacts
uses: strophy/actions-cache@opendal-update
with:
Expand Down Expand Up @@ -84,14 +84,18 @@ jobs:
run: echo "sha=$(git log -1 --format="%h" -- packages/dashmate)" >> $GITHUB_OUTPUT
if: inputs.restore_local_network_data == true

- name: Set HOME variable to github context
shell: bash
run: echo "HOME=$HOME" >> $GITHUB_ENV

Comment on lines +87 to +90
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix potential path handling issue

The current HOME variable setting could have issues with paths containing spaces.

Apply this fix to prevent word splitting:

- run: echo "HOME=$HOME" >> $GITHUB_ENV
+ run: echo "HOME=\"$HOME\"" >> $GITHUB_ENV
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Set HOME variable to github context
shell: bash
run: echo "HOME=$HOME" >> $GITHUB_ENV
- name: Set HOME variable to github context
shell: bash
run: echo "HOME=\"$HOME\"" >> $GITHUB_ENV
🧰 Tools
🪛 actionlint

89-89: shellcheck reported issue in this script: SC2086:info:1:22: Double quote to prevent globbing and word splitting

(shellcheck)

- name: Use test suite local network data to speed up dashmate local network tests
id: local-network-data
uses: strophy/actions-cache/restore@opendal-update
with:
bucket: multi-runner-cache-x1xibo9c
root: local-network-data
path: |
/home/ubuntu/.dashmate
${{ env.HOME }}/.dashmate
dashmate_volumes_dump
key: local-network-volumes/${{ steps.dashmate-fingerprint.outputs.sha }}
if: inputs.restore_local_network_data == true
Expand All @@ -105,7 +109,7 @@ jobs:
env:
DEBUG: 1
DASHMATE_E2E_TESTS_SKIP_IMAGE_BUILD: true
DASHMATE_E2E_TESTS_LOCAL_HOMEDIR: /home/ubuntu/.dashmate
DASHMATE_E2E_TESTS_LOCAL_HOMEDIR: ${{ env.HOME }}/.dashmate
if: steps.local-network-data.outputs.cache-hit == 'true'

- name: Run tests without cache
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/tests-packges-functional.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,9 @@ on:
jobs:
test-functional:
name: Run functional tests
runs-on: [ "self-hosted", "linux", "arm64", "ubuntu-platform" ]
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
CHROME_BIN: /usr/bin/brave-browser
ECR_HOST: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com
steps:
- name: Check out repo
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/tests-test-suite.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,9 @@ on:
jobs:
test-suite:
name: Run ${{ inputs.name }}
runs-on: [ "self-hosted", "linux", "arm64", "ubuntu-platform" ]
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
CHROME_BIN: /usr/bin/brave-browser
ECR_HOST: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com
steps:
- name: Check out repo
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ RUN --mount=type=cache,sharing=shared,id=cargo_registry_index,target=${CARGO_HOM
else \
export FEATURES_FLAG="--features=console,grovedbg" ; \
export OUT_DIRECTORY=debug ; \

fi && \
if [[ -z "${SCCACHE_MEMCACHED}" ]] ; then unset SCCACHE_MEMCACHED ; fi ; \
cargo build \
Expand Down
4 changes: 0 additions & 4 deletions packages/dashmate/src/commands/status/masternode.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,13 @@ export default class MasternodeStatusCommand extends ConfigBaseCommand {
/**
* @param {Object} args
* @param {Object} flags
* @param {DockerCompose} dockerCompose
* @param {createRpcClient} createRpcClient
* @param {Config} config
* @param getMasternodeScope getMasternodeScope
* @return {Promise<void>}
*/
async runWithDependencies(
args,
flags,
dockerCompose,
createRpcClient,
config,
getMasternodeScope,
) {
Expand Down
Loading