Skip to content
This repository has been archived by the owner on Aug 2, 2022. It is now read-only.

[develop] Add Trigger for LRTs and Multiversion Tests Post PR #7901

Merged
merged 52 commits into from
Sep 13, 2019
Merged

[develop] Add Trigger for LRTs and Multiversion Tests Post PR #7901

Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
defa006
Create trigger for LRTs.
scottarnette Sep 11, 2019
4abf858
Create trigger for LRTs.
scottarnette Sep 11, 2019
9279b69
Create trigger for LRTs.
scottarnette Sep 11, 2019
3e04af1
Remove LRT steps from generate-pipeline in favor of triggering eosio-…
scottarnette Sep 11, 2019
bd7be1e
Eliminate second "Pipeline Upload
kj4ezj Sep 11, 2019
ca85cb7
Fix triggered slug.
scottarnette Sep 11, 2019
52fb312
Remove LRT skip defaulting to true.
scottarnette Sep 11, 2019
6355998
Testing containing trigger steps to same generate-pipeline.sh.
scottarnette Sep 11, 2019
2fe0912
Debugging.
scottarnette Sep 11, 2019
63bc734
Revert "Debugging."
scottarnette Sep 11, 2019
0ff6084
Debugging.
scottarnette Sep 11, 2019
35b7a03
Debugging.
scottarnette Sep 11, 2019
0bb4293
Debugging.
scottarnette Sep 11, 2019
ff90b80
Debugging.
scottarnette Sep 11, 2019
420973d
Debugging.
scottarnette Sep 11, 2019
0e925b4
Debugging.
scottarnette Sep 11, 2019
664d7e2
Debugging.
scottarnette Sep 11, 2019
6748ec5
Add logic to only trigger if not already triggered.
scottarnette Sep 11, 2019
4ee5f53
Add logic to only trigger if not already triggered.
scottarnette Sep 11, 2019
98f9135
Add logic to only trigger if not already triggered.
scottarnette Sep 11, 2019
5797e55
Add logic to only trigger if not already triggered.
scottarnette Sep 11, 2019
f5db605
Adjust trigger logic.
scottarnette Sep 11, 2019
c2865fa
Adjust trigger logic.
scottarnette Sep 12, 2019
6aaeaf8
Adjust trigger logic.
scottarnette Sep 12, 2019
57be1bf
Adjust trigger logic.
scottarnette Sep 12, 2019
3fc1fcb
Removed separate script in favor of conditional logic in generate-pi…
scottarnette Sep 12, 2019
03b6258
Removed vars in favor of Terraform applying at pipeline level.
scottarnette Sep 12, 2019
5976484
Logic prep for only triggering on PRs.
scottarnette Sep 12, 2019
3506d39
Logic prep for only triggering on PRs.
scottarnette Sep 12, 2019
5707017
Switch logic to run on PRs.
scottarnette Sep 12, 2019
00db51d
Switch logic to run on PRs.
scottarnette Sep 12, 2019
04970a4
Remove slug check in favor of only using trigger check.
scottarnette Sep 12, 2019
b150508
Remove old if statement.
scottarnette Sep 12, 2019
2d05d36
Fix multiversion trigger label.
scottarnette Sep 12, 2019
0787091
Pass pinned/unpinned variable for LRTs triggered by eosio-unnpinned p…
scottarnette Sep 12, 2019
804002c
Remove newline.
scottarnette Sep 12, 2019
1e25597
Multiversion shouldn't be called on unpinned build trigger.
scottarnette Sep 12, 2019
5953402
Trigger multiversion async so we don't wait for it.
scottarnette Sep 12, 2019
5eb4eff
Set LRTs as async as well.
scottarnette Sep 12, 2019
dc8c625
Unpinned builds don't need to run LRTs on every PR.
scottarnette Sep 12, 2019
dc9d1c4
Unpinned builds don't need to run LRTs on every PR.
scottarnette Sep 12, 2019
4eca0e7
Concurrency attempt.
scottarnette Sep 13, 2019
7bff72d
Concurrency attempt.
scottarnette Sep 13, 2019
80762d1
Trigger builds don't conflict with scheduled concurrency groups.
scottarnette Sep 13, 2019
76e763e
Fix BUILD_SOURCE on triggered builds.
scottarnette Sep 13, 2019
7618963
Only one concurrency group for Linux.
scottarnette Sep 13, 2019
d802cbe
Empty commit.
scottarnette Sep 13, 2019
1296499
Remove concurrency and trigger on PR in favor of triggering post PR.
scottarnette Sep 13, 2019
ae747ce
Finish resetting source checks for concurrency.
scottarnette Sep 13, 2019
b6e1c1d
Fix mac queues being moved around.
scottarnette Sep 13, 2019
edbfeab
Support double-digit major versions.
scottarnette Sep 13, 2019
02dd009
Added back pipeline tail comment.
scottarnette Sep 13, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 63 additions & 12 deletions .cicd/generate-pipeline.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,15 @@ for FILE in $(ls $CICD_DIR/platforms); do
"ICON": env.ICON
}]')
done
# set build_source whether triggered or not
if [[ ! -z ${BUILDKITE_TRIGGERED_FROM_BUILD_ID} ]]; then
export BUILD_SOURCE="--build \$BUILDKITE_TRIGGERED_FROM_BUILD_ID"
fi
export BUILD_SOURCE=${BUILD_SOURCE:---build \$BUILDKITE_BUILD_ID}
# set trigger_job if master/release/develop branch and webhook
if [[ $BUILDKITE_BRANCH =~ ^release/[0-9]+\.[0-9]+\.x$ || $BUILDKITE_BRANCH =~ ^master$ || $BUILDKITE_BRANCH =~ ^develop$ ]]; then
[[ $BUILDKITE_SOURCE == 'webhook' ]] && export TRIGGER_JOB=true
fi
oIFS="$IFS"
IFS=$''
nIFS=$IFS # fix array splitting (\n won't work)
Expand Down Expand Up @@ -102,7 +111,7 @@ echo $PLATFORMS_JSON_ARRAY | jq -cr '.[]' | while read -r PLATFORM_JSON; do
agents:
queue: "$BUILDKITE_AGENT_QUEUE"
timeout: ${TIMEOUT:-180}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_BUILD}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_BUILD}

EOF
else
Expand All @@ -127,10 +136,11 @@ EOF
wait-network: true
timeout: ${TIMEOUT:-180}
agents: "queue=mac-anka-large-node-fleet"
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_BUILD}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_BUILD}

EOF
fi
if [ "$BUILDKITE_SOURCE" = 'schedule' ]; then
if [ "$BUILDKITE_SOURCE" = "schedule" ]; then
scottarnette marked this conversation as resolved.
Show resolved Hide resolved
cat <<EOF
concurrency: ${CONCURRENCY}
concurrency_group: ${CONCURRENCY_GROUP}
Expand Down Expand Up @@ -162,7 +172,7 @@ for ROUND in $(seq 1 $ROUNDS); do
agents:
queue: "$BUILDKITE_AGENT_QUEUE"
timeout: ${TIMEOUT:-10}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_UNIT_TESTS}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_UNIT_TESTS}

EOF
else
Expand All @@ -185,7 +195,8 @@ EOF
wait-network: true
timeout: ${TIMEOUT:-20}
agents: "queue=mac-anka-node-fleet"
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_UNIT_TESTS}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_UNIT_TESTS}

EOF
fi
if [ "$BUILDKITE_SOURCE" = "schedule" ]; then
Expand Down Expand Up @@ -216,7 +227,7 @@ EOF
agents:
queue: "$BUILDKITE_AGENT_QUEUE"
timeout: ${TIMEOUT:-20}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_SERIAL_TESTS}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_SERIAL_TESTS}

EOF
else
Expand All @@ -239,8 +250,7 @@ EOF
wait-network: true
timeout: ${TIMEOUT:-20}
agents: "queue=mac-anka-node-fleet"
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_SERIAL_TESTS}

skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_SERIAL_TESTS}
EOF
fi
if [ "$BUILDKITE_SOURCE" = "schedule" ]; then
Expand All @@ -265,15 +275,15 @@ EOF
cat <<EOF
- label: "$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - $TEST_NAME"
command:
- "buildkite-agent artifact download build.tar.gz . --step '$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - Build' && tar -xzf build.tar.gz"
- "buildkite-agent artifact download build.tar.gz . --step '$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - Build' ${BUILD_SOURCE} && tar -xzf build.tar.gz"
- "./.cicd/test.sh scripts/long-running-test.sh $TEST_NAME"
env:
IMAGE_TAG: $(echo "$PLATFORM_JSON" | jq -r .FILE_NAME)
BUILDKITE_AGENT_ACCESS_TOKEN:
agents:
queue: "$BUILDKITE_AGENT_QUEUE"
timeout: ${TIMEOUT:-180}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_LONG_RUNNING_TESTS:-true}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_LONG_RUNNING_TESTS:-true}

EOF
else
Expand All @@ -283,7 +293,7 @@ EOF
- label: "$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - $TEST_NAME"
command:
- "git clone \$BUILDKITE_REPO eos && cd eos && git checkout \$BUILDKITE_COMMIT && git submodule update --init --recursive"
- "cd eos && buildkite-agent artifact download build.tar.gz . --step '$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - Build' && tar -xzf build.tar.gz"
- "cd eos && buildkite-agent artifact download build.tar.gz . --step '$(echo "$PLATFORM_JSON" | jq -r .ICON) $(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_FULL) - Build' ${BUILD_SOURCE} && tar -xzf build.tar.gz"
- "cd eos && ./.cicd/test.sh scripts/long-running-test.sh $TEST_NAME"
plugins:
- chef/anka#v0.5.1:
Expand All @@ -296,7 +306,7 @@ EOF
wait-network: true
timeout: ${TIMEOUT:-180}
agents: "queue=mac-anka-node-fleet"
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}\${SKIP_LONG_RUNNING_TESTS:-true}
skip: \${SKIP_$(echo "$PLATFORM_JSON" | jq -r .PLATFORM_NAME_UPCASE)_$(echo "$PLATFORM_JSON" | jq -r .VERSION_MAJOR)$(echo "$PLATFORM_JSON" | jq -r .VERSION_MINOR)}${SKIP_LONG_RUNNING_TESTS:-true}
EOF
fi
if [ "$BUILDKITE_SOURCE" = "schedule" ]; then
Expand All @@ -315,6 +325,47 @@ EOF
echo ''
fi
done
# trigger eosio-lrt post pr
if [[ -z $BUILDKITE_TRIGGERED_FROM_BUILD_ID && $TRIGGER_JOB == "true" ]]; then
if ( [[ ! $PINNED == false || $UNPINNED == true ]] ); then
cat <<EOF
- label: ":pipeline: Trigger Long Running Tests"
trigger: "eosio-lrt"
async: true
build:
message: "${BUILDKITE_MESSAGE}"
commit: "${BUILDKITE_COMMIT}"
branch: "${BUILDKITE_BRANCH}"
env:
BUILDKITE_PULL_REQUEST: "${BUILDKITE_PULL_REQUEST}"
BUILDKITE_PULL_REQUEST_BASE_BRANCH: "${BUILDKITE_PULL_REQUEST_BASE_BRANCH}"
BUILDKITE_PULL_REQUEST_REPO: "${BUILDKITE_PULL_REQUEST_REPO}"
scottarnette marked this conversation as resolved.
Show resolved Hide resolved
SKIP_BUILD: "true"
PINNED: "${PINNED}"
UNPINNED: "${UNPINNED}"

EOF
fi
fi
# trigger multiversion post pr
if [[ -z $BUILDKITE_TRIGGERED_FROM_BUILD_ID && $TRIGGER_JOB = "true" ]]; then
if ( [[ ! $PINNED == false || $UNPINNED == true ]] ); then
cat <<EOF
- label: ":pipeline: Trigger Multiversion Test"
trigger: "eos-multiversion-tests"
async: true
build:
message: "${BUILDKITE_MESSAGE}"
commit: "${BUILDKITE_COMMIT}"
branch: "${BUILDKITE_BRANCH}"
env:
BUILDKITE_PULL_REQUEST: "${BUILDKITE_PULL_REQUEST}"
BUILDKITE_PULL_REQUEST_BASE_BRANCH: "${BUILDKITE_PULL_REQUEST_BASE_BRANCH}"
BUILDKITE_PULL_REQUEST_REPO: "${BUILDKITE_PULL_REQUEST_REPO}"

EOF
fi
fi
# pipeline tail
cat <<EOF
- wait:
Expand Down