Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove hardcoded module definitions in CI #39506

Merged
merged 2 commits into from
May 13, 2024
Merged

Remove hardcoded module definitions in CI #39506

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
800ff80
Remove hardcoded module definitions in CI
dliappis May 10, 2024
dc4a9d1
Merge branch 'main' into remove-hardcoded-modules
dliappis May 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .buildkite/metricbeat/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet metricbeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We remove references to the env var MODULE because if unset, it fails the command due to undefined variable checks (set -e)

echo "~~~ Running tests"

# TODO move this section to base image / pre-command hook
echo "~~~ Installing kind"
Expand Down Expand Up @@ -90,7 +90,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

# TODO move this section to base image / pre-command hook
echo "~~~ Installing kind"
Expand Down
16 changes: 4 additions & 12 deletions .buildkite/scripts/changesets.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,17 +56,9 @@ function DefineModuleFromTheChangeSet($projectPath) {
}
}

# TODO: remove this conditional when issue https://github.com/elastic/ingest-dev/issues/2993 gets resolved
if(!$changedModules) {
if($Env:BUILDKITE_PIPELINE_SLUG -eq 'beats-xpack-metricbeat') {
$Env:MODULE = "aws"
}
else {
$Env:MODULE = "kubernetes"
}
}
else {
# TODO: once https://github.com/elastic/ingest-dev/issues/2993 gets resolved, this should be the only thing we export
$Env:MODULE = $changedModules
if ($changedModules) {
$env:MODULE = $changedModules
Write-Output "~~~ Set env var MODULE to [$env:MODULE]"
Write-Output "~~~ Resuming commands"
}
}
13 changes: 5 additions & 8 deletions .buildkite/scripts/changesets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,10 @@ defineModuleFromTheChangeSet() {
fi
done

if [[ -z "$changed_modules" ]]; then # TODO: remove this conditional when issue https://github.com/elastic/ingest-dev/issues/2993 gets resolved
if [[ "$BUILDKITE_PIPELINE_SLUG" == "beats-xpack-metricbeat" ]]; then
export MODULE="aws"
else
export MODULE="kubernetes"
fi
else
export MODULE="${changed_modules}" # TODO: once https://github.com/elastic/ingest-dev/issues/2993 gets resolved, this should be the only thing we export
# export MODULE="" leads to an infinite loop https://github.com/elastic/ingest-dev/issues/2993
if [[ ! -z $changed_modules ]]; then
export MODULE="${changed_modules}"
echo "~~~ Set env var MODULE to [$MODULE]"
echo "~~~ Resuming commands"
fi
}
2 changes: 1 addition & 1 deletion .buildkite/x-pack/pipeline.xpack.auditbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/auditbeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/auditbeat
mage update build test
retry:
Expand Down
4 changes: 1 addition & 3 deletions .buildkite/x-pack/pipeline.xpack.dockerlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,11 +56,9 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/dockerlogbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/dockerlogbeat
mage goIntegTest
env:
MODULE: $MODULE
retry:
automatic:
- limit: 3
Expand Down
8 changes: 4 additions & 4 deletions .buildkite/x-pack/pipeline.xpack.filebeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/filebeat && mage goIntegTest
retry:
automatic:
Expand All @@ -78,7 +78,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/filebeat && mage pythonIntegTest
retry:
automatic:
Expand Down Expand Up @@ -272,7 +272,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

.buildkite/scripts/cloud_tests.sh
env:
Expand Down Expand Up @@ -301,7 +301,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/filebeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
.buildkite/scripts/cloud_tests.sh
env:
ASDF_TERRAFORM_VERSION: 1.0.2
Expand Down
8 changes: 4 additions & 4 deletions .buildkite/x-pack/pipeline.xpack.metricbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Will run tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/metricbeat && mage goIntegTest
retry:
automatic:
Expand All @@ -78,7 +78,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
cd x-pack/metricbeat && mage pythonIntegTest
retry:
automatic:
Expand Down Expand Up @@ -257,7 +257,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"

.buildkite/scripts/cloud_tests.sh
env:
Expand Down Expand Up @@ -288,7 +288,7 @@ steps:
# defines the MODULE env var based on what's changed in a PR
source .buildkite/scripts/changesets.sh
defineModuleFromTheChangeSet x-pack/metricbeat
echo "~~~ Running tests with env var MODULE=$$MODULE"
echo "~~~ Running tests"
.buildkite/scripts/cloud_tests.sh
env:
ASDF_TERRAFORM_VERSION: 1.0.2
Expand Down
4 changes: 1 addition & 3 deletions .buildkite/x-pack/pipeline.xpack.winlogbeat.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,9 @@ steps:
command: |
Import-Module ./.buildkite/scripts/changesets.psm1
defineModuleFromTheChangeSet 'x-pack/winlogbeat'
Write-Output "~~~ Will run tests with env var MODULE=$$Env:MODULE"
Write-Output "~~~ Running tests"
Set-Location -Path x-pack/winlogbeat
mage build unitTest
env:
MODULE: $MODULE
retry:
automatic:
- limit: 3
Expand Down
Loading