Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: upload logs in kind-network-test #9755

Merged
merged 14 commits into from
Nov 5, 2024
6 changes: 5 additions & 1 deletion .github/ensure-tester/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,11 @@ runs:
elif [[ $TYPE == 128core-* ]]; then
SIZE=32xlarge
fi
echo "instance_type=m6a.$SIZE m6in.$SIZE r6a.$SIZE r6i.$SIZE r6in.$SIZE" >> $GITHUB_OUTPUT
INSTANCE_TYPES="m6a.$SIZE m6in.$SIZE r6a.$SIZE r6i.$SIZE r6in.$SIZE"
if [[ $TYPE == *high-memory* ]]; then
INSTANCE_TYPES="r6a.$SIZE r6i.$SIZE r6in.$SIZE"
fi
echo "instance_type=$INSTANCE_TYPES" >> $GITHUB_OUTPUT

- name: Start Tester
uses: ./.github/spot-runner-action
Expand Down
21 changes: 16 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -690,7 +690,7 @@ jobs:
# note: proving disabled
kind-network-test:
needs: [build, configure]
if: contains(github.event.pull_request.labels.*.name, 'network-all') || github.ref_name == 'master'
if: contains(github.event.pull_request.labels.*.name, 'network-all') || (needs.configure.outputs.yarn-project == 'true' && github.ref_name == 'master')
runs-on: ${{ needs.configure.outputs.username }}-x86
strategy:
fail-fast: false
Expand All @@ -699,24 +699,27 @@ jobs:
- test: transfer.test.ts
values: 16-validators
runner_type: 16core-tester-x86
timeout: 60
# TODO(#9736)
# - test: transfer.test.ts
# values: 48-validators
# runner_type: 32core-tester-x86
- test: reorg.test.ts
values: 16-validators
runner_type: 16core-tester-x86
runner_type: 16core-tester-x86-high-memory
timeout: 90
- test: 4epochs.test.ts
values: 16-validators
runner_type: 16core-tester-x86
timeout: 60
steps:
- uses: actions/checkout@v4
with: { ref: "${{ env.GIT_COMMIT }}" }
- uses: ./.github/ci-setup-action
with:
concurrency_key: kind-network-${{ matrix.config.test }}
- name: Setup and KIND Network Test
timeout-minutes: 60
timeout-minutes: ${{ matrix.config.timeout }}
uses: ./.github/ensure-tester-with-images
env:
USERNAME: ${{ needs.configure.outputs.username }}
Expand All @@ -729,13 +732,21 @@ jobs:
builder_images_to_copy: aztecprotocol/aztec:${{ env.GIT_COMMIT }} aztecprotocol/end-to-end:${{ env.GIT_COMMIT }}
# command to produce the images in case they don't exist
builder_command: scripts/earthly-ci ./yarn-project+export-e2e-test-images
tester_ttl: 60
# TODO(#9640): use 16-validators.yaml
tester_ttl: ${{ matrix.config.timeout }}
run: |
cd yarn-project/end-to-end
echo ${{ secrets.DOCKERHUB_PASSWORD }} | docker login -u aztecprotocolci --password-stdin
test=${{ matrix.config.test }}
NAMESPACE="${test%.test.ts}" FRESH_INSTALL=true VALUES_FILE=${{ matrix.config.values }}.yaml ./scripts/network_test.sh ./src/spartan/$test
- name: Copy Network Logs
if: always() # Run on failure too
run: copy_from_tester /home/ubuntu/run-${{ env.RUN_ID }}/yarn-project/end-to-end/scripts/network-test.log network-test.log
- name: Upload Network Logs
if: always() # Run on failure too
uses: actions/upload-artifact@v4
with:
name: kind-network-test-${{ matrix.config.values }}-${{ matrix.config.test }}.log
path: network-test.log

l1-contracts-test:
needs: [build, configure]
Expand Down
6 changes: 5 additions & 1 deletion yarn-project/end-to-end/scripts/network_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,10 @@ if [ "$FRESH_INSTALL" = "true" ]; then
kubectl delete namespace "$NAMESPACE" --ignore-not-found=true --wait=true --now --timeout=10m
fi

function copy_stern_to_log() {
stern spartan -n $NAMESPACE > $SCRIPT_DIR/network-test.log
}

function show_status_until_pxe_ready() {
set +x # don't spam with our commands
sleep 15 # let helm upgrade start
Expand Down Expand Up @@ -99,6 +103,7 @@ handle_network_shaping() {
return 0
}

copy_stern_to_log &
show_status_until_pxe_ready &

function cleanup() {
Expand Down Expand Up @@ -146,7 +151,6 @@ if ! handle_network_shaping; then
fi
fi


docker run --rm --network=host \
-v ~/.kube:/root/.kube \
-e K8S=true \
Expand Down
Loading