Skip to content

Commit

Permalink
feat: Skip test folder name for ptn (#1622)
Browse files Browse the repository at this point in the history
## Description

Closes #1470 

- If the module is of type `ptn`, skip Pester test checking for
`defaults` and `waf-aligned` test folders
- Updating workflow job conditions accordingly.

PsRule jobs running only on `defaults `and `waf-aligned` folders, need
to be skipped if the value of `psRuleModuleTestFilePaths `is empty.
`psRuleModuleTestFilePaths `is the variable calculated by the initialize
pipeline job, hosting the path to the deploy test

Test performed in a different branch (main) where a "fake" pattern
module is created with no `defaults` or `waf-aligned` test folders.
Pester tests for telemetry and orphaned modules have been commented
during testing. PR 1620 contains, for reference, also the "fake" files
used to test this update.

1. push to main - complete, success (publish runs)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/75589a6d-30fe-40e4-b0f6-e24e813c8e16)

1. manual run - only static, success (publish is skipped)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/7f28195a-b414-4f01-b5e4-62d43a282081)

1. manual run - only deployment (publish is skipped)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/2e76bb48-3819-42a3-be65-637b46c48620)

1. push to main - complete, static fails (deployment and publish are
skipped)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/a09bc275-94d5-4d74-8496-2e4c82439537)

1. push to main - cancel pipeline while running static (deployment and
publish are skipped)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/c1bc0559-c084-42b4-9473-8b5cf0ee8d5b)

1. manual run - complete, success (publish runs)

![image](https://github.com/Azure/bicep-registry-modules/assets/56914614/8502e41a-41ed-4a6f-9681-4dbac97c36f8)


## Pipeline Reference

<!-- Insert your Pipeline Status Badge below -->

| Pipeline |
| -------- |
|
[![avm.ptn.batch.batch-account-ptn](https://github.com/eriqua/bicep-registry-modules/actions/workflows/avm.ptn.batch.batch-account-ptn.yml/badge.svg)](https://github.com/eriqua/bicep-registry-modules/actions/workflows/avm.ptn.batch.batch-account-ptn.yml)
(test ptn) |
|
[![avm.res.key-vault.vault](https://github.com/eriqua/bicep-registry-modules/actions/workflows/avm.res.key-vault.vault.yml/badge.svg)](https://github.com/eriqua/bicep-registry-modules/actions/workflows/avm.res.key-vault.vault.yml)
(test res) |

## Type of Change

<!-- Use the check-boxes [x] on the options that are relevant. -->

- [x] Update to CI Environment or utlities (Non-module effecting
changes)
- [ ] Azure Verified Module updates:
- [ ] Bugfix containing backwards compatible bug fixes, and I have NOT
bumped the MAJOR or MINOR version in `version.json`:
- [ ] Someone has opened a bug report issue, and I have included "Closes
#{bug_report_issue_number}" in the PR description.
- [ ] The bug was found by the module author, and no one has opened an
issue to report it yet.
- [ ] Feature update backwards compatible feature updates, and I have
bumped the MINOR version in `version.json`.
- [ ] Breaking changes and I have bumped the MAJOR version in
`version.json`.
  - [ ] Update to documentation

## Checklist

- [x] I'm sure there are no other open Pull Requests for the same
update/change
- [ ] I have run `Set-AVMModule` locally to generate the supporting
module files.
- [x] My corresponding pipelines / checks run clean and green without
any errors or warnings

<!-- Please keep up to day with the contribution guide at
https://aka.ms/avm/contribute/bicep -->
  • Loading branch information
eriqua authored Apr 10, 2024
1 parent 805c167 commit 3bf7e40
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 18 deletions.
15 changes: 11 additions & 4 deletions .github/workflows/avm.template.module.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
job_psrule_test: # Note: Please don't change this job name. It is used by the setEnvironment action to define which PS modules to install on runners.
name: "PSRule [${{ matrix.testCases.name }}]"
runs-on: ubuntu-latest
if: (fromJson(inputs.workflowInput)).staticValidation == 'true'
if: ${{ inputs.psRuleModuleTestFilePaths != '' && (fromJson(inputs.workflowInput)).staticValidation == 'true' }}
strategy:
fail-fast: false
matrix:
Expand All @@ -74,7 +74,7 @@ jobs:
job_psrule_test_waf_reliability: # Note: Please don't change this job name. It is used by the setEnvironment action to define which PS modules to install on runners.
name: "PSRule - WAF Reliability [${{ matrix.testCases.name }}]"
runs-on: ubuntu-latest
if: (fromJson(inputs.workflowInput)).staticValidation == 'true'
if: ${{ inputs.psRuleModuleTestFilePaths != '' && (fromJson(inputs.workflowInput)).staticValidation == 'true' }}
strategy:
fail-fast: false
matrix:
Expand All @@ -101,7 +101,9 @@ jobs:
runs-on: ubuntu-latest
if: |
!cancelled() &&
(fromJson(inputs.workflowInput)).deploymentValidation == 'true' && needs.job_module_static_validation.result != 'failure' && needs.job_psrule_test_waf_reliability.result != 'failure'
(fromJson(inputs.workflowInput)).deploymentValidation == 'true' &&
needs.job_module_static_validation.result != 'failure' &&
needs.job_psrule_test_waf_reliability.result != 'failure'
needs:
- job_module_static_validation
- job_psrule_test_waf_reliability
Expand Down Expand Up @@ -136,8 +138,13 @@ jobs:
job_publish_module: # Note: Please don't change this job name. It is used by the setEnvironment action to define which PS modules to install on runners.
name: "Publishing"
runs-on: ubuntu-latest
if: github.ref == 'refs/heads/main' && success()
# Note: Below always() required in condition due to psrule jobs being skipped for ptn modules not having defaults or waf-aligned folders
if: github.ref == 'refs/heads/main' &&
always() &&
needs.job_module_static_validation.result == 'success' &&
needs.job_module_deploy_validation.result == 'success'
needs:
- job_module_static_validation
- job_module_deploy_validation
steps:
- name: "Checkout"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Describe 'File/folder tests' -Tag 'Modules' {

$moduleFolderTestCases = [System.Collections.ArrayList] @()
foreach ($moduleFolderPath in $moduleFolderPaths) {
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$moduleFolderTestCases += @{
moduleFolderName = $resourceTypeIdentifier
moduleFolderPath = $moduleFolderPath
Expand Down Expand Up @@ -146,11 +146,13 @@ Describe 'File/folder tests' -Tag 'Modules' {

$topLevelModuleTestCases = [System.Collections.ArrayList]@()
foreach ($moduleFolderPath in $moduleFolderPaths) {
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$moduleTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[1] # 'avm/res|ptn/<provider>/<resourceType>' would return 'res|ptn'
if (($resourceTypeIdentifier -split '[\/|\\]').Count -eq 2) {
$topLevelModuleTestCases += @{
moduleFolderName = $moduleFolderPath.Replace('\', '/').Split('/avm/')[1]
moduleFolderPath = $moduleFolderPath
moduleFolderName = $moduleFolderPath.Replace('\', '/').Split('/avm/')[1]
moduleFolderPath = $moduleFolderPath
moduleTypeIdentifier = $moduleTypeIdentifier
}
}
}
Expand Down Expand Up @@ -185,7 +187,7 @@ Describe 'File/folder tests' -Tag 'Modules' {
$pathExisting | Should -Be $true
}

It '[<moduleFolderName>] Module should contain a [` tests/e2e/*waf-aligned `] folder.' -TestCases $topLevelModuleTestCases {
It '[<moduleFolderName>] Module should contain a [` tests/e2e/*waf-aligned `] folder.' -TestCases ($topLevelModuleTestCases | Where-Object { $_.moduleTypeIdentifier -eq 'res' }) {

param(
[string] $moduleFolderPath
Expand All @@ -195,7 +197,7 @@ Describe 'File/folder tests' -Tag 'Modules' {
$wafAlignedFolder | Should -Not -BeNullOrEmpty
}

It '[<moduleFolderName>] Module should contain a [` tests/e2e/*defaults `] folder.' -TestCases $topLevelModuleTestCases {
It '[<moduleFolderName>] Module should contain a [` tests/e2e/*defaults `] folder.' -TestCases ($topLevelModuleTestCases | Where-Object { $_.moduleTypeIdentifier -eq 'res' }) {

param(
[string] $moduleFolderPath
Expand Down Expand Up @@ -227,7 +229,7 @@ Describe 'Pipeline tests' -Tag 'Pipeline' {
$pipelineTestCases = [System.Collections.ArrayList] @()
foreach ($moduleFolderPath in $moduleFolderPaths) {

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$relativeModulePath = Join-Path 'avm' ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}')[1]

$isTopLevelModule = ($resourceTypeIdentifier -split '[\/|\\]').Count -eq 2
Expand Down Expand Up @@ -282,7 +284,7 @@ Describe 'Module tests' -Tag 'Module' {

foreach ($moduleFolderPath in $moduleFolderPaths) {

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$templateFilePath = Join-Path $moduleFolderPath 'main.bicep'

$readmeFileTestCases += @{
Expand Down Expand Up @@ -340,7 +342,7 @@ Describe 'Module tests' -Tag 'Module' {
continue
}

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'

$armTemplateTestCases += @{
moduleFolderName = $resourceTypeIdentifier
Expand Down Expand Up @@ -392,7 +394,7 @@ Describe 'Module tests' -Tag 'Module' {
$templateFilePath = Join-Path $moduleFolderPath 'main.bicep'
$templateFileContent = $builtTestFileMap[$templateFilePath]

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'

# Test file setup
$moduleFolderTestCases += @{
Expand Down Expand Up @@ -618,7 +620,7 @@ Describe 'Module tests' -Tag 'Module' {
$udtSpecificTestCases = [System.Collections.ArrayList] @() # Specific UDT test cases for singular UDTs (e.g. tags)
foreach ($moduleFolderPath in $moduleFolderPaths) {

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$templateFilePath = Join-Path $moduleFolderPath 'main.bicep'
$templateFileContent = $builtTestFileMap[$templateFilePath]

Expand Down Expand Up @@ -1102,7 +1104,7 @@ Describe 'Governance tests' {
$governanceTestCases = [System.Collections.ArrayList] @()
foreach ($moduleFolderPath in $moduleFolderPaths) {

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'
$relativeModulePath = Join-Path 'avm' ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}')[1]

$isTopLevelModule = ($resourceTypeIdentifier -split '[\/|\\]').Count -eq 2
Expand Down Expand Up @@ -1138,7 +1140,6 @@ Describe 'Governance tests' {
$moduleLine | Should -Be $expectedEntry -Because 'the module should match the expected format as documented [here](https://azure.github.io/Azure-Verified-Modules/specs/shared/#codeowners-file).'
}


It '[<moduleFolderName>] Module identifier should be listed in issue template in the correct alphabetical position.' -TestCases $governanceTestCases {

param(
Expand Down Expand Up @@ -1194,7 +1195,7 @@ Describe 'Test file tests' -Tag 'TestTemplate' {
$testFilePaths = (Get-ChildItem -Path $moduleFolderPath -Recurse -Filter 'main.test.bicep').FullName | Sort-Object
foreach ($testFilePath in $testFilePaths) {
$testFileContent = Get-Content $testFilePath
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # 'avm/res|ptn/<provider>/<resourceType>' would return '<provider>/<resourceType>'

$deploymentTestFileTestCases += @{
testName = Split-Path (Split-Path $testFilePath) -Leaf
Expand Down

0 comments on commit 3bf7e40

Please sign in to comment.