Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added post-deployment test support #1036

Merged
Merged
Show file tree
Hide file tree
Changes from 115 commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
20ac8b7
Added skeleton
AlexanderSehr Nov 7, 2023
4939b4d
Implemented first new tests
AlexanderSehr Nov 7, 2023
4e1d86f
Changed to warning
AlexanderSehr Nov 8, 2023
94fff0b
Expanded test verbosity & added additional tets
AlexanderSehr Nov 8, 2023
c18ce6b
Finalized further tests
AlexanderSehr Nov 8, 2023
a343694
Update to latest
AlexanderSehr Nov 8, 2023
3a6f0a2
Update to latest
AlexanderSehr Nov 8, 2023
4ab6a5b
Added module name + small fix
AlexanderSehr Nov 8, 2023
9bb64bd
Fixed kvlt
AlexanderSehr Nov 8, 2023
68e2bd2
Small caching fix
AlexanderSehr Nov 8, 2023
1a013c2
Small fix and enabled passthru of warnings to GH
AlexanderSehr Nov 8, 2023
1c2fc8a
More updates
AlexanderSehr Nov 8, 2023
d9c6dd5
Update to latest
AlexanderSehr Nov 8, 2023
dd0c343
Updated test names
AlexanderSehr Nov 8, 2023
2ebd458
JSON rollback
AlexanderSehr Nov 8, 2023
f6caf7e
Updated regex
AlexanderSehr Nov 8, 2023
f56f66a
Removed assertion
AlexanderSehr Nov 8, 2023
282b640
Update to latest
AlexanderSehr Nov 8, 2023
c025f52
Update avm/utilities/pipelines/staticValidation/compliance/module.tes…
AlexanderSehr Nov 9, 2023
405901a
Update avm/utilities/pipelines/staticValidation/compliance/module.tes…
AlexanderSehr Nov 9, 2023
904c82f
Merge branch 'main' into users/alsehr/testFileCompliance
AlexanderSehr Nov 9, 2023
ef24d1f
Rollback of unrelated changes
AlexanderSehr Nov 9, 2023
9393aa1
Merge branch 'Azure:main' into main
AlexanderSehr Nov 9, 2023
cb39086
Merge branch 'Azure:main' into main
AlexanderSehr Nov 9, 2023
89eac40
Merge branch 'Azure:main' into main
AlexanderSehr Nov 10, 2023
712a673
Merge branch 'Azure:main' into main
AlexanderSehr Nov 13, 2023
55e095f
Resolved conflict
AlexanderSehr Nov 14, 2023
3121335
Update to latest
AlexanderSehr Nov 14, 2023
e9d736c
Merge branch 'Azure:main' into main
AlexanderSehr Nov 15, 2023
abfa1b9
Merge branch 'Azure:main' into main
AlexanderSehr Nov 15, 2023
c6707f6
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
246f9ba
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
6e9729d
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
26ea9f6
Merge branch 'Azure:main' into main
AlexanderSehr Nov 17, 2023
c2d183b
Merge branch 'Azure:main' into main
AlexanderSehr Nov 18, 2023
0c6650e
Merge branch 'Azure:main' into main
AlexanderSehr Nov 19, 2023
65094a5
Merge branch 'Azure:main' into main
AlexanderSehr Nov 21, 2023
968cd9a
Merge branch 'Azure:main' into main
AlexanderSehr Nov 21, 2023
3c9589b
Merge branch 'Azure:main' into main
AlexanderSehr Nov 24, 2023
eb34ddf
Merge branch 'Azure:main' into main
AlexanderSehr Nov 25, 2023
3d4f5c6
Added pipeline to main for testing
AlexanderSehr Nov 26, 2023
70154f3
Merge branch 'Azure:main' into main
AlexanderSehr Nov 27, 2023
18bd2ad
Merge branch 'Azure:main' into main
AlexanderSehr Nov 27, 2023
940b34f
Merge branch 'Azure:main' into main
AlexanderSehr Nov 28, 2023
fbb840f
Merge branch 'Azure:main' into main
AlexanderSehr Nov 29, 2023
381e366
Merge branch 'Azure:main' into main
AlexanderSehr Nov 29, 2023
06bcda3
Merge branch 'Azure:main' into main
AlexanderSehr Dec 1, 2023
0858beb
Merge branch 'Azure:main' into main
AlexanderSehr Dec 1, 2023
72b5fbb
Merge branch 'Azure:main' into main
AlexanderSehr Dec 1, 2023
917a761
TEst disable pipe
AlexanderSehr Dec 4, 2023
1b89452
Merge branch 'Azure:main' into main
AlexanderSehr Dec 4, 2023
feb981e
Update to latest
AlexanderSehr Dec 4, 2023
dda85a8
Merge branch 'Azure:main' into main
AlexanderSehr Dec 4, 2023
0373f46
Merge branch 'Azure:main' into main
AlexanderSehr Dec 6, 2023
80247e4
Merge branch 'Azure:main' into main
AlexanderSehr Dec 9, 2023
2b5a791
Merge branch 'Azure:main' into main
AlexanderSehr Dec 10, 2023
f1bab34
Merge branch 'Azure:main' into main
AlexanderSehr Dec 19, 2023
ef12ee5
Merge branch 'Azure:main' into main
AlexanderSehr Dec 21, 2023
ec1722c
Merge branch 'Azure:main' into main
AlexanderSehr Dec 21, 2023
f4136ec
Added SA workflow
AlexanderSehr Dec 22, 2023
e55e732
Added workflow
AlexanderSehr Jan 1, 2024
5c6aa5e
Update to latest
AlexanderSehr Jan 1, 2024
086c1a8
Merge branch 'Azure:main' into main
AlexanderSehr Jan 2, 2024
a9821fb
Merge branch 'Azure:main' into main
AlexanderSehr Jan 3, 2024
62b326f
Merge branch 'Azure:main' into main
AlexanderSehr Jan 3, 2024
c159269
Merge branch 'Azure:main' into main
AlexanderSehr Jan 4, 2024
6e1f59a
Merge branch 'Azure:main' into main
AlexanderSehr Jan 5, 2024
fb7befc
Merge branch 'Azure:main' into main
AlexanderSehr Jan 5, 2024
cb72681
Merge branch 'Azure:main' into main
AlexanderSehr Jan 8, 2024
f3befb7
Merge branch 'Azure:main' into main
AlexanderSehr Jan 10, 2024
f938cd8
Merge branch 'Azure:main' into main
AlexanderSehr Jan 12, 2024
d0924b6
Merge branch 'Azure:main' into main
AlexanderSehr Jan 14, 2024
315522f
Merge branch 'Azure:main' into main
AlexanderSehr Jan 17, 2024
4012fdd
Update to latest
AlexanderSehr Jan 18, 2024
b18ab70
Merge branch 'Azure:main' into main
AlexanderSehr Jan 20, 2024
686dc83
Merge branch 'Azure:main' into main
AlexanderSehr Jan 22, 2024
3f225f9
Removed outdated metadata
AlexanderSehr Jan 23, 2024
f2bd197
Update to latest
AlexanderSehr Jan 23, 2024
4cd0a6f
Merge branch 'Azure:main' into main
AlexanderSehr Jan 26, 2024
01f3a38
Merged latest main
AlexanderSehr Jan 26, 2024
18c794a
Merge branch 'Azure:main' into users/alsehr/testCleanup
AlexanderSehr Jan 26, 2024
732a605
Merge branch 'main' into users/alsehr/testCleanup
AlexanderSehr Jan 29, 2024
c0105fe
Merge branch 'Azure:main' into main
AlexanderSehr Jan 29, 2024
c7d09f2
Merge branch 'main' into users/alsehr/testCleanup
AlexanderSehr Jan 29, 2024
bea9408
Merge branch 'users/alsehr/testCleanup' of https://github.com/Alexand…
AlexanderSehr Jan 29, 2024
9d063ce
Refereshed docs
AlexanderSehr Jan 29, 2024
db61282
Merge branch 'main' into users/alsehr/testCleanup
AlexanderSehr Jan 29, 2024
4ec79f8
Merge branch 'Azure:main' into main
AlexanderSehr Jan 29, 2024
c26b307
Merge branches 'users/alsehr/testCleanup' and 'main' of https://githu…
AlexanderSehr Jan 29, 2024
e3759a8
Merge branch 'Azure:main' into main
AlexanderSehr Feb 2, 2024
86869a1
Merge branch 'main' of https://github.com/AlexanderSehr/bicep-registr…
AlexanderSehr Feb 2, 2024
f8da6b9
Merge branch 'Azure:main' into main
AlexanderSehr Feb 4, 2024
ece4faa
Merge branch 'main' of https://github.com/AlexanderSehr/bicep-registr…
AlexanderSehr Feb 4, 2024
8884290
Merge branch 'Azure:main' into main
AlexanderSehr Feb 9, 2024
6ed614f
Merge branch 'Azure:main' into main
AlexanderSehr Feb 10, 2024
e307d2a
Merge branch 'Azure:main' into main
AlexanderSehr Feb 11, 2024
96589bb
Merge branch 'Azure:main' into main
AlexanderSehr Feb 12, 2024
75ade55
Merge branch 'Azure:main' into main
AlexanderSehr Feb 13, 2024
c2637f3
Update to latest
AlexanderSehr Feb 16, 2024
01f414b
Disabled other tests
AlexanderSehr Feb 16, 2024
64900a1
Updated ref
AlexanderSehr Feb 16, 2024
2d1e1b7
Update to latest
AlexanderSehr Feb 17, 2024
4345ba6
Update to latest
AlexanderSehr Feb 17, 2024
7e70c08
Update to latest
AlexanderSehr Feb 17, 2024
96e5539
Update to latest
AlexanderSehr Feb 17, 2024
37aad84
Small fix
AlexanderSehr Feb 17, 2024
0b297f7
Aded output
AlexanderSehr Feb 17, 2024
dd8b19d
Update to latest
AlexanderSehr Feb 17, 2024
cfe99c8
Update to latest
AlexanderSehr Feb 17, 2024
b849bc8
Update to latest
AlexanderSehr Feb 17, 2024
8f2e826
Enabled additional deploy test
AlexanderSehr Feb 17, 2024
420a40a
Added job conditions
AlexanderSehr Feb 17, 2024
fcf2c98
Update to latest
AlexanderSehr Feb 17, 2024
7d8a44c
Update to latest
AlexanderSehr Feb 17, 2024
9de254a
Update to latest
AlexanderSehr Feb 17, 2024
56e7afb
Merge branch 'Azure:main' into main
AlexanderSehr Feb 21, 2024
f89b946
Merge branch 'Azure:main' into main
AlexanderSehr Feb 21, 2024
c861111
Merge branch 'Azure:main' into main
AlexanderSehr Feb 22, 2024
9511716
Merge branch 'Azure:main' into main
AlexanderSehr Feb 26, 2024
f28816d
Merge branch 'Azure:main' into main
AlexanderSehr Feb 27, 2024
2d53a8a
Merge branch 'main' into users/alsehr/postDeploymentTest
ChrisSidebotham Feb 28, 2024
61e7138
Merge branch 'main' into users/alsehr/postDeploymentTest
ChrisSidebotham Mar 4, 2024
40972b5
Merge branch 'main' into users/alsehr/postDeploymentTest
ChrisSidebotham Mar 4, 2024
059e5fb
Merge branch 'main' into users/alsehr/postDeploymentTest
AlexanderSehr Mar 4, 2024
515baba
Update to latest
AlexanderSehr Mar 4, 2024
7fcdff4
Updated title
AlexanderSehr Mar 4, 2024
1eceb4b
Update to latest
AlexanderSehr Mar 4, 2024
383a12a
Merge branch 'main' into users/alsehr/postDeploymentTest
ChrisSidebotham Mar 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -264,14 +264,9 @@ runs:
Write-Output ('{0}={1}' -f 'deploymentNames', ($res.deploymentNames | ConvertTo-Json -Compress)) >> $env:GITHUB_OUTPUT

# Populate further outputs
$deploymentOutputHashTable = @{}
$deploymentOutput = $res.deploymentOutput | ConvertTo-Json -Depth 99 -Compress
Write-Output ('{0}={1}' -f 'deploymentOutput', $deploymentOutput) >> $env:GITHUB_OUTPUT

foreach ($outputKey in $res.deploymentOutput.Keys) {
Write-Output ('{0}={1}' -f 'outputKey', $res.deploymentOutput[$outputKey].Value) >> $env:GITHUB_OUTPUT
$deploymentOutputHashTable.add($outputKey, $res.deploymentOutput[$outputKey].Value)
}

$deploymentOutput = $deploymentOutputHashTable | ConvertTo-Json -Compress -Depth 100
Write-Verbose "Deployment output: $deploymentOutput" -Verbose

if ($res.ContainsKey('exception')) {
Expand All @@ -281,6 +276,94 @@ runs:

Write-Output '::endgroup::'

# [Post-Deployment test] task(s)
# ------------------------------
- name: "Run post-deployment Pester tests"
id: pester_run_step
shell: pwsh
run: |
# Grouping task logs
Write-Output '::group::Run Pester tests'

# Load used functions
. (Join-Path $env:GITHUB_WORKSPACE 'avm' 'utilities' 'pipelines' 'staticValidation' 'compliance' 'Set-PesterGitHubOutput.ps1')

# Set repo root path
$repoRootPath = $env:GITHUB_WORKSPACE

$moduleTestFilePath = Join-Path $env:GITHUB_WORKSPACE '${{ inputs.templateFilePath }}'
$moduleTestFolderPath = Split-Path $moduleTestFilePath
$moduleTestFolderName = Split-Path $moduleTestFolderPath -Leaf

$deploymentOutputs = '${{ steps.deploy_step.outputs.deploymentOutput }}' | ConvertFrom-Json -AsHashTable

# --------------------- #
# Invoke Pester test(s) #
# --------------------- #

$pesterConfiguration = @{
Run = @{
Container = New-PesterContainer -Path $moduleTestFolderPath -Data @{
TestInputData = @{
DeploymentOutputs = $deploymentOutputs # Passing in in case we want to directly validate data via the deployment's output
ModuleTestFolderPath = $moduleTestFolderPath # Passing in in case we want to access any data in the test folder / files
}
}
PassThru = $true
}
Output = @{
Verbosity = 'Detailed'
}
}

Write-Verbose 'Invoke test with' -Verbose
$foundTestPaths = $pesterConfiguration.Run.Container | Where-Object { -not [String]::IsNullOrEmpty($_.Item.FullName) } | Foreach-Object { ($_.Item.FullName -split '[\\|\/]tests[\\|\/]e2e[\\|\/]')[1] }

if($foundTestPaths.Count -gt 0) {
Write-Verbose ('Path(s): {0}' -f ($foundTestPaths | Convertto-Json)) -Verbose
Write-Verbose ('Data: {0}' -f ($pesterConfiguration.Run.Container.Data[0] | ConvertTo-Json -Depth 3)) -Verbose

$testResults = Invoke-Pester -Configuration $pesterConfiguration

# ----------------------------------------- #
# Create formatted Pester Test Results File #
# ----------------------------------------- #

$functionInput = @{
PesterTestResults = $testResults
OutputFilePath = Join-Path $env:GITHUB_WORKSPACE 'avm' "$moduleTestFolderName-Pester-output.md"
GitHubRepository = $env:GITHUB_REPOSITORY
BranchName = $env:GITHUB_REF
}

Write-Verbose 'Invoke Pester formatting function with' -Verbose
Write-Verbose ($functionInput | ConvertTo-Json -Depth 0 | Out-String) -Verbose

Set-PesterGitHubOutput @functionInput -Verbose

Write-Output ('{0}={1}' -f 'formattedPesterResultsPath', $functionInput.outputFilePath) >> $env:GITHUB_OUTPUT
} else {
Write-Verbose 'Found no Pester test files (*.test.ps1) in test folder' -Verbose
Write-Output ('{0}={1}' -f 'formattedPesterResultsPath', '') >> $env:GITHUB_OUTPUT
}
- name: "Output to GitHub job summaries"
if: steps.pester_run_step.outputs.formattedPesterResultsPath != ''
shell: pwsh
run: |
# Grouping task logs
Write-Output '::group::Output to GitHub job summaries'

$mdPesterOutputFilePath = '${{ steps.pester_run_step.outputs.formattedPesterResultsPath }}'

if (-not (Test-Path $mdPesterOutputFilePath)) {
Write-Warning ('Input file [{0}] not found. Please check if the previous task threw an error and try again.' -f $mdPesterOutputFilePath)
} else {
Get-Content $mdPesterOutputFilePath >> $env:GITHUB_STEP_SUMMARY
Write-Verbose ('Successfully printed out file [{0}] to Job Summaries' -f $mdPesterOutputFilePath) -Verbose
}

Write-Output '::endgroup::'

# [Deployment removal] task(s)
# ----------------------------
- name: "Remove deployed resources"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,3 +99,5 @@ module testDeployment '../../../main.bicep' = [for iteration in [ 'init', 'idem'
enablePurgeProtection: false
}
}]

output resourceId string = testDeployment[1].outputs.resourceId
AlexanderSehr marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
######################################
## Additional post-deployment tests ##
######################################
##
## You can add any custom post-deployment validation tests you want here, or add them spread accross multiple test files in the test case folder.
##
###########################

param (
[Parameter(Mandatory = $false)]
[hashtable] $TestInputData = @{}
)

Describe 'Validate Key Vault' {

It 'Public endpoint should be disabled' {

$keyVaultResourceId = $TestInputData.DeploymentOutputs.resourceId.Value

$deployedResource = Get-AzResource -ResourceId $keyVaultResourceId

$deployedResource.Properties.publicNetworkAccess | Should -Be 'Disabled'
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
######################################
## Additional post-deployment tests ##
######################################
##
## You can add any custom post-deployment validation tests you want here, or add them spread accross multiple test files in the test case folder.
##
###########################

param (
[Parameter(Mandatory = $false)]
[hashtable] $TestInputData = @{}
)

Describe 'Validate private endpoint deployment' {

Context 'Validate sucessful deployment' {

It "Private endpoints should be deployed in resource group" {

$keyVaultResourceId = $TestInputData.DeploymentOutputs.resourceId.Value
$testResourceGroup = ($keyVaultResourceId -split '\/')[4]
$deployedPrivateEndpoints = Get-AzPrivateEndpoint -ResourceGroupName $testResourceGroup
$deployedPrivateEndpoints.Count | Should -BeGreaterThan 0
}

It 'Private endpoint should have role assignment' {

$keyVaultResourceId = $TestInputData.DeploymentOutputs.resourceId.Value
$testResourceGroup = ($keyVaultResourceId -split '\/')[4]
$deployedPrivateEndpoints = Get-AzPrivateEndpoint -ResourceGroupName $testResourceGroup

$firstPrivateEndpointResourceId = $deployedPrivateEndpoints[0].Id
$firstPrivateEndpointName = ($firstPrivateEndpointResourceId -split '\/')[-1]

$roleAssignments = Get-AzRoleAssignment -ResourceName $firstPrivateEndpointName -ResourceType 'Microsoft.Network/privateEndpoints' -ResourceGroupName $testResourceGroup
$roleAssignments.Count | Should -BeGreaterThan 0
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,6 @@ function Set-PesterGitHubOutput {
)

if ($failedTests.Count -gt 0) {
Write-Verbose 'Adding failed tests'
$fileContent += [System.Collections.ArrayList]@(
'| Name | Error | Source |',
'| :-- | :-- | :-- |'
Expand Down
Loading