Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added URL fetch error handling to test script #642

Merged
merged 36 commits into from
Nov 17, 2023
Merged
Changes from 1 commit
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
20ac8b7
Added skeleton
AlexanderSehr Nov 7, 2023
4939b4d
Implemented first new tests
AlexanderSehr Nov 7, 2023
4e1d86f
Changed to warning
AlexanderSehr Nov 8, 2023
94fff0b
Expanded test verbosity & added additional tets
AlexanderSehr Nov 8, 2023
c18ce6b
Finalized further tests
AlexanderSehr Nov 8, 2023
a343694
Update to latest
AlexanderSehr Nov 8, 2023
3a6f0a2
Update to latest
AlexanderSehr Nov 8, 2023
4ab6a5b
Added module name + small fix
AlexanderSehr Nov 8, 2023
9bb64bd
Fixed kvlt
AlexanderSehr Nov 8, 2023
68e2bd2
Small caching fix
AlexanderSehr Nov 8, 2023
1a013c2
Small fix and enabled passthru of warnings to GH
AlexanderSehr Nov 8, 2023
1c2fc8a
More updates
AlexanderSehr Nov 8, 2023
d9c6dd5
Update to latest
AlexanderSehr Nov 8, 2023
dd0c343
Updated test names
AlexanderSehr Nov 8, 2023
2ebd458
JSON rollback
AlexanderSehr Nov 8, 2023
f6caf7e
Updated regex
AlexanderSehr Nov 8, 2023
f56f66a
Removed assertion
AlexanderSehr Nov 8, 2023
282b640
Update to latest
AlexanderSehr Nov 8, 2023
c025f52
Update avm/utilities/pipelines/staticValidation/compliance/module.tes…
AlexanderSehr Nov 9, 2023
405901a
Update avm/utilities/pipelines/staticValidation/compliance/module.tes…
AlexanderSehr Nov 9, 2023
904c82f
Merge branch 'main' into users/alsehr/testFileCompliance
AlexanderSehr Nov 9, 2023
ef24d1f
Rollback of unrelated changes
AlexanderSehr Nov 9, 2023
9393aa1
Merge branch 'Azure:main' into main
AlexanderSehr Nov 9, 2023
cb39086
Merge branch 'Azure:main' into main
AlexanderSehr Nov 9, 2023
89eac40
Merge branch 'Azure:main' into main
AlexanderSehr Nov 10, 2023
712a673
Merge branch 'Azure:main' into main
AlexanderSehr Nov 13, 2023
55e095f
Resolved conflict
AlexanderSehr Nov 14, 2023
3121335
Update to latest
AlexanderSehr Nov 14, 2023
e9d736c
Merge branch 'Azure:main' into main
AlexanderSehr Nov 15, 2023
abfa1b9
Merge branch 'Azure:main' into main
AlexanderSehr Nov 15, 2023
c6707f6
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
246f9ba
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
8c7e316
Improved robustness of tests
AlexanderSehr Nov 16, 2023
50696b1
Added proper error handling to web-request
AlexanderSehr Nov 16, 2023
6e9729d
Merge branch 'Azure:main' into main
AlexanderSehr Nov 16, 2023
27d0f1d
Merge branch 'main' into users/alsehr/errorHandling
AlexanderSehr Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Expanded test verbosity & added additional tets
  • Loading branch information
AlexanderSehr committed Nov 8, 2023
commit 94fff0ba27001ed731cd18daee02d1b06b7cef21
Original file line number Diff line number Diff line change
Expand Up @@ -910,7 +910,8 @@ Describe 'Module tests' -Tag 'Module' {

Context 'User-defined-types tests' -Tag 'UDT' {

$udtTestCases = [System.Collections.ArrayList] @()
$udtTestCases = [System.Collections.ArrayList] @() # General UDT tests (e.g. param should exist)
$udtSpecificTestCases = [System.Collections.ArrayList] @() # Specific UDT test cases for singular UDTs (e.g. tags)
foreach ($moduleFolderPath in $moduleFolderPaths) {

$resourceTypeIdentifier = ($moduleFolderPath -split '[\/|\\]{1}avm[\/|\\]{1}(res|ptn)[\/|\\]{1}')[2] -replace '\\', '/' # avm/res/<provider>/<resourceType>
Expand All @@ -935,34 +936,47 @@ Describe 'Module tests' -Tag 'Module' {
$templateContent = $convertedTemplates[$moduleFolderPathKey].templateContent
}

$udtSpecificTestCases += @{
moduleFolderName = $resourceTypeIdentifier
templateFileContent = $templateContent
templateFileContentBicep = Get-Content $templateFilePath
}

# Setting expected URL only for those that doen't have multiple different variants
$avmInterfaceSpecsBase = 'https://raw.githubusercontent.com/Azure/Azure-Verified-Modules/main/docs/static/includes/interfaces'
$avmInterfaceSpecsTemplateBase = 'https://raw.githubusercontent.com/Azure/Azure-Verified-Modules/main/docs/static/includes/interfaces'
$avmInterfaceSpecsBase = 'https://azure.github.io/Azure-Verified-Modules/specs/shared/interfaces'
$udtCases = @(
@{
parameterName = 'diagnosticSettings'
udtName = 'diagnosticSettingType'
link = "$avmInterfaceSpecsBase#diagnostic-settings"
}
@{
parameterName = 'roleAssignments'
udtName = 'roleAssignmentType'
udtExpectedUrl = "$avmInterfaceSpecsBase/int.rbac.udt.schema.bicep"
udtExpectedUrl = "$avmInterfaceSpecsTemplateBase/int.rbac.udt.schema.bicep"
link = "$avmInterfaceSpecsBase#role-assignments"
}
@{
parameterName = 'lock'
udtName = 'lockType'
udtExpectedUrl = "$avmInterfaceSpecsBase/int.locks.udt.schema.bicep"
udtExpectedUrl = "$avmInterfaceSpecsTemplateBase/int.locks.udt.schema.bicep"
link = "$avmInterfaceSpecsBase#resource-locks"
}
@{
parameterName = 'managedIdentities'
udtName = 'managedIdentitiesType'
link = "$avmInterfaceSpecsBase#managed-identities"
}
@{
parameterName = 'privateEndpoints'
udtName = 'privateEndpointType'
link = "$avmInterfaceSpecsBase#private-endpoints"
}
@{
parameterName = 'customerManagedKey'
udtName = 'customerManagedKeyType'
link = "$avmInterfaceSpecsBase#customer-managed-keys"
}
)

Expand All @@ -979,19 +993,20 @@ Describe 'Module tests' -Tag 'Module' {
}


It "[<moduleFolderName>] If template has [<parameterName>] parameter, it should implement the expected user-defined type [<udtName>]" -TestCases $udtTestCases {
It "[<moduleFolderName>] If template has [<parameterName>] parameter, it should implement the user-defined type [<udtName>]" -TestCases $udtTestCases {

param(
[hashtable] $templateFileContent,
[string[]] $templateFileContentBicep,
[string] $parameterName,
[string] $udtName,
[string] $expectedUdtUrl
[string] $expectedUdtUrl,
[string] $link
)

if ($templateFileContent.parameters.Keys -contains $parameterName) {
$templateFileContent.parameters.$parameterName.Keys | Should -Contain '$ref' -Because "the [$parameterName] parameter should use a user-defined type."
$templateFileContent.parameters.$parameterName.'$ref' | Should -Be "#/definitions/$udtName" -Because "the [$parameterName] parameter should use a user-defined type [$udtName]."
$templateFileContent.parameters.$parameterName.Keys | Should -Contain '$ref' -Because "the [$parameterName] parameter should use a user-defined type. For for information please review the [AVM Specs]($link)."
$templateFileContent.parameters.$parameterName.'$ref' | Should -Be "#/definitions/$udtName" -Because "the [$parameterName] parameter should use a user-defined type [$udtName]. For for information please review the [AVM Specs]($link)."

if (-not [String]::IsNullOrEmpty($expectedUdtUrl)) {
$implementedSchemaStartIndex = $templateFileContentBicep.IndexOf("type $udtName = {")
Expand Down Expand Up @@ -1025,17 +1040,39 @@ Describe 'Module tests' -Tag 'Module' {
}

if ($formattedDiff.Count -gt 0) {
Write-Warning ("The implemented user-defined type should be the same as the expected user-defined type of url [{0}] and should not have diff`n{1}" -f $expectedUdtUrl, ($formattedDiff | Out-String))
Write-Warning ("The implemented user-defined type is not the same as the expected [user-defined type]({0}) defined in the [AVM specs]({1}) and should not have diff`n{2}" -f $expectedUdtUrl, $link, ($formattedDiff | Out-String))
}
}
} else {
Set-ItResult -Skipped -Because "the module template has no [$parameterName] parameter."
}
}

It "[<moduleFolderName>] If a [managedIdentitiesType] UDT definition exists and supports system-assigned-identities, the template should have an output for its principal ID." -TestCases $udtSpecificTestCases {

param(
[hashtable] $templateFileContent
)

if ($templateFileContent.definitions.Keys -contains 'managedIdentitiesType' -and $templateFileContent.definitions.managedIdentitiesType.properties.keys -contains 'systemAssigned') {
$templateFileContent.outputs.Keys | Should -Contain 'systemAssignedMIPrincipalId' -Because 'The AVM specs require a this output. For for information please review the [AVM Specs](https://azure.github.io/Azure-Verified-Modules/specs/shared/interfaces#managed-identities).'
} else {
Set-ItResult -Skipped -Because "the module template has no [managedIdentitiesType] UDT definition or does not support system-assigned-identities."
}
}

It "[<moduleFolderName>] If a [tags] parameter exists it should be nullable." -TestCases $udtTestCases {

# TODO Add test for tags
# TODO add tests for msi principal id output
param(
[hashtable] $templateFileContent
)

if ($templateFileContent.parameters.Keys -contains 'tags') {
$templateFileContent.parameters.tags.nullable | Should -Be $true -Because 'The AVM specs require a specific format. For for information please review the [AVM Specs](https://azure.github.io/Azure-Verified-Modules/specs/shared/interfaces#tags).'
} else {
Set-ItResult -Skipped -Because "the module template has no [tags] parameter."
}
}
}
}

Expand Down