Skip to content

Commit

Permalink
feat: New utility module - AVM Common Types (Azure#3397)
Browse files Browse the repository at this point in the history
## Description

- Implemented AVM common types module which can be used to reduce
duplicated code in the code base
- Reduced the tests in `main.test.bicep` for UDTs as they are not
compatible with using a module reference
- Added a quotation-escape to the deployment action as it turned out
that it breaks if you have single quotes in your deployment output

Example implementation in KeyVault module:

> **Note:** The type should always be implemented by name as a `*`
implementation causes all types to be added to the combiled `main.json`


![image](https://github.com/user-attachments/assets/76837be7-121b-4a76-a2f8-89c924abbcd8)


![image](https://github.com/user-attachments/assets/10507e09-a289-4b9e-b97f-c6e8241cd3ca)


## Pipeline Reference

<!-- Insert your Pipeline Status Badge below -->

| Pipeline |
| -------- |
|
[![avm.ptn.types.avm-common-type](https://github.com/AlexanderSehr/bicep-registry-modules/actions/workflows/avm.utl.types.avm-common-type.yml/badge.svg?branch=users%2Falsehr%2F1396_commonTypes)](https://github.com/AlexanderSehr/bicep-registry-modules/actions/workflows/avm.utl.types.avm-common-type.yml)
|
|
[![avm.res.key-vault.vault](https://github.com/AlexanderSehr/bicep-registry-modules/actions/workflows/avm.res.key-vault.vault.yml/badge.svg?branch=users%2Falsehr%2F1396_commonTypes)](https://github.com/AlexanderSehr/bicep-registry-modules/actions/workflows/avm.res.key-vault.vault.yml)
[run using local
ref](https://github.com/AlexanderSehr/bicep-registry-modules/actions/runs/11164377047)
|

## Type of Change

<!-- Use the checkboxes [x] on the options that are relevant. -->

- [ ] Update to CI Environment or utilities (Non-module affecting
changes)
- [x] Azure Verified Module updates:
- [ ] Bugfix containing backwards-compatible bug fixes, and I have NOT
bumped the MAJOR or MINOR version in `version.json`:
- [ ] Someone has opened a bug report issue, and I have included "Closes
#{bug_report_issue_number}" in the PR description.
- [ ] The bug was found by the module author, and no one has opened an
issue to report it yet.
- [ ] Feature update backwards compatible feature updates, and I have
bumped the MINOR version in `version.json`.
- [ ] Breaking changes and I have bumped the MAJOR version in
`version.json`.
  - [ ] Update to documentation
  • Loading branch information
AlexanderSehr authored Oct 15, 2024
1 parent 6fd1411 commit da7e5cb
Show file tree
Hide file tree
Showing 10 changed files with 2,053 additions and 73 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -171,4 +171,5 @@
/avm/res/web/serverfarm/ @Azure/avm-res-web-serverfarm-module-owners-bicep @Azure/avm-module-reviewers-bicep
/avm/res/web/site/ @Azure/avm-res-web-site-module-owners-bicep @Azure/avm-module-reviewers-bicep
/avm/res/web/static-site/ @Azure/avm-res-web-staticsite-module-owners-bicep @Azure/avm-module-reviewers-bicep
/avm/utl/types/avm-common-types/ @Azure/avm-utl-types-avmcommontypes-module-owners-bicep @Azure/avm-module-reviewers-bicep
*avm.core.team.tests.ps1 @Azure/avm-core-team-technical-bicep
1 change: 1 addition & 0 deletions .github/ISSUE_TEMPLATE/avm_module_issue.yml
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,7 @@ body:
- "avm/res/web/serverfarm"
- "avm/res/web/site"
- "avm/res/web/static-site"
- "avm/utl/types/avm-common-types"
validations:
required: true
- type: input
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -327,7 +327,7 @@ runs:
Write-Output ('{0}={1}' -f 'deploymentNames', ($res.deploymentNames | ConvertTo-Json -Compress)) >> $env:GITHUB_OUTPUT
# Populate further outputs
$deploymentOutput = $res.deploymentOutput | ConvertTo-Json -Depth 99 -Compress
$deploymentOutput = ($res.deploymentOutput | ConvertTo-Json -Depth 99 -Compress) -replace "'", "''" # Escaping single quotes for resilient access in subsequent steps
Write-Output ('{0}={1}' -f 'deploymentOutput', $deploymentOutput) >> $env:GITHUB_OUTPUT
Write-Verbose "Deployment output: $deploymentOutput" -Verbose
Expand Down
89 changes: 89 additions & 0 deletions .github/workflows/avm.utl.types.avm-common-types.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
name: "avm.utl.types.avm-common-types"

on:
workflow_dispatch:
inputs:
staticValidation:
type: boolean
description: "Execute static validation"
required: false
default: false # true
deploymentValidation:
type: boolean
description: "Execute deployment validation"
required: false
default: true
removeDeployment:
type: boolean
description: "Remove deployed module"
required: false
default: true
customLocation:
type: string
description: "Default location overwrite (e.g., eastus)"
required: false
default: 'northeurope'
push:
branches:
- main
paths:
- ".github/actions/templates/avm-**"
- ".github/workflows/avm.template.module.yml"
- ".github/workflows/avm.utl.types.avm-common-types.yml"
- "avm/utl/types/avm-common-types/**"
- "avm/utilities/pipelines/**"
- "!avm/utilities/pipelines/platform/**"
- "!*/**/README.md"

env:
modulePath: "avm/utl/types/avm-common-types"
workflowPath: ".github/workflows/avm.utl.types.avm-common-types.yml"

concurrency:
group: ${{ github.workflow }}

jobs:
###########################
# Initialize pipeline #
###########################
job_initialize_pipeline:
runs-on: ubuntu-latest
name: "Initialize pipeline"
steps:
- name: "Checkout"
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: "Set input parameters to output variables"
id: get-workflow-param
uses: ./.github/actions/templates/avm-getWorkflowInput
with:
workflowPath: "${{ env.workflowPath}}"
- name: "Get module test file paths"
id: get-module-test-file-paths
uses: ./.github/actions/templates/avm-getModuleTestFiles
with:
modulePath: "${{ env.modulePath }}"
outputs:
workflowInput: ${{ steps.get-workflow-param.outputs.workflowInput }}
moduleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.moduleTestFilePaths }}
psRuleModuleTestFilePaths: ${{ steps.get-module-test-file-paths.outputs.psRuleModuleTestFilePaths }}
modulePath: "${{ env.modulePath }}"

##############################
# Call reusable workflow #
##############################
call-workflow-passing-data:
name: "Run"
permissions:
id-token: write # For OIDC
contents: write # For release tags
needs:
- job_initialize_pipeline
uses: ./.github/workflows/avm.template.module.yml
with:
workflowInput: "${{ needs.job_initialize_pipeline.outputs.workflowInput }}"
moduleTestFilePaths: "${{ needs.job_initialize_pipeline.outputs.moduleTestFilePaths }}"
psRuleModuleTestFilePaths: "${{ needs.job_initialize_pipeline.outputs.psRuleModuleTestFilePaths }}"
modulePath: "${{ needs.job_initialize_pipeline.outputs.modulePath}}"
secrets: inherit
Original file line number Diff line number Diff line change
Expand Up @@ -667,34 +667,26 @@ Describe 'Module tests' -Tag 'Module' {
$udtCases = @(
@{
parameterName = 'diagnosticSettings'
udtName = 'diagnosticSettingType'
link = "$interfaceBase/diagnostic-settings"
}
@{
parameterName = 'roleAssignments'
udtName = 'roleAssignmentType'
udtExpectedUrl = "$interfaceBase/role-assignments/udt-schema"
link = "$interfaceBase/role-assignments"
}
@{
parameterName = 'lock'
udtName = 'lockType'
udtExpectedUrl = "$interfaceBase/resource-locks/udt-schema"
link = "$interfaceBase/resource-locks"
}
@{
parameterName = 'managedIdentities'
udtName = 'managedIdentitiesType'
link = "$interfaceBase/managed-identities"
}
@{
parameterName = 'privateEndpoints'
udtName = 'privateEndpointType'
link = "$interfaceBase/private-endpoints"
}
@{
parameterName = 'customerManagedKey'
udtName = 'customerManagedKeyType'
link = "$interfaceBase/customer-managed-keys"
}
)
Expand All @@ -705,86 +697,29 @@ Describe 'Module tests' -Tag 'Module' {
templateFileContent = $templateFileContent
templateFileContentBicep = Get-Content $templateFilePath
parameterName = $udtCase.parameterName
udtName = $udtCase.udtName
expectedUdtUrl = $udtCase.udtExpectedUrl ? $udtCase.udtExpectedUrl : ''
link = $udtCase.link
}
}
}

It '[<moduleFolderName>] If template has a parameter [<parameterName>], it should implement the user-defined type [<udtName>]' -TestCases $udtTestCases {
It '[<moduleFolderName>] If template has a parameter [<parameterName>], it should implement AVM''s corresponding user-defined type.' -TestCases $udtTestCases {

param(
[hashtable] $templateFileContent,
[string[]] $templateFileContentBicep,
[string] $parameterName,
[string] $udtName,
[string] $expectedUdtUrl,
[string] $link
)

if ($templateFileContent.parameters.Keys -contains $parameterName) {
$templateFileContent.parameters.$parameterName.Keys | Should -Contain '$ref' -Because "the [$parameterName] parameter should use a user-defined type. For information please review the [AVM Specs]($link)."
$templateFileContent.parameters.$parameterName.'$ref' | Should -Be "#/definitions/$udtName" -Because "the [$parameterName] parameter should use a user-defined type [$udtName]. For information please review the [AVM Specs]($link)."

if (-not [String]::IsNullOrEmpty($expectedUdtUrl)) {
$implementedSchemaStartIndex = $templateFileContentBicep.IndexOf("type $udtName = {")
$implementedSchemaEndIndex = $implementedSchemaStartIndex + 1
while ($templateFileContentBicep[$implementedSchemaEndIndex] -notmatch '^\}.*' -and $implementedSchemaEndIndex -lt $templateFileContentBicep.Length) {
$implementedSchemaEndIndex++
}
if ($implementedSchemaEndIndex -eq $templateFileContentBicep.Length) {
throw "Failed to identify [$udtName] user-defined type in template."
}
$implementedSchema = $templateFileContentBicep[$implementedSchemaStartIndex..$implementedSchemaEndIndex]

try {
$rawResponse = Invoke-WebRequest -Uri $expectedUdtUrl
if (($rawResponse.Headers['Content-Type'] | Out-String) -like '*text/plain*') {
$expectedSchemaFull = $rawResponse.Content -split '\n'
} else {
throw "Failed to fetch schema from [$expectedUdtUrl]. Skipping schema check"
}
} catch {
Write-Warning "Failed to fetch schema from [$expectedUdtUrl]. Skipping schema check"
return
}

$expectedSchemaStartIndex = $expectedSchemaFull.IndexOf("type $udtName = {")
$expectedSchemaEndIndex = $expectedSchemaStartIndex + 1
while ($expectedSchemaFull[$expectedSchemaEndIndex] -notmatch '^\}.*' -and $expectedSchemaEndIndex -lt $expectedSchemaFull.Length) {
$expectedSchemaEndIndex++
}
if ($expectedSchemaEndIndex -eq $expectedSchemaFull.Length) {
throw "Failed to identify [$udtName] user-defined type in expected schema at URL [$expectedUdtUrl]."
}
$expectedSchema = $expectedSchemaFull[$expectedSchemaStartIndex..$expectedSchemaEndIndex]

if ($templateFileContentBicep -match '@sys\.([a-zA-Z]+)\(') {
# Handing cases where the template may use the @sys namespace explicitly
$expectedSchema = $expectedSchema | ForEach-Object { $_ -replace '@([a-zA-Z]+)\(', '@sys.$1(' }
}

$formattedDiff = @()
foreach ($finding in (Compare-Object $implementedSchema $expectedSchema)) {
if ($finding.SideIndicator -eq '=>') {
$formattedDiff += ('+ {0}' -f $finding.InputObject)
} elseif ($finding.SideIndicator -eq '<=') {
$formattedDiff += ('- {0}' -f $finding.InputObject)
}
}

if ($formattedDiff.Count -gt 0) {
$warningMessage = "The implemented user-defined type is not the same as the expected user-defined type ({0}) defined in the AVM specs ({1}) and should not have diff`n{2}" -f $expectedUdtUrl, $link, ($formattedDiff | Out-String)
Write-Warning $warningMessage

# Adding also to output to show in GitHub CI
$mdFormattedDiff = ($formattedDiff -join '</br>') -replace '\|', '\|'
$mdFormattedWarningMessage = 'The implemented user-defined type is not the same as the expected [user-defined type]({0}) defined in the [AVM specs]({1}) and should not have diff</br><pre>{2}</pre>' -f $expectedUdtUrl, $link, $mdFormattedDiff
Write-Output @{
Warning = $mdFormattedWarningMessage
}
}
if ($templateFileContent.parameters.$parameterName.Keys -contains 'items') {
# If parameter is an array, the UDT may focus on each element
$templateFileContent.parameters.$parameterName.items.Keys | Should -Contain '$ref' -Because "the [$parameterName] parameter should use a user-defined type. For information please review the [AVM Specs]($link)."
} else {
# If not, the parameter itself should reference a UDT
$templateFileContent.parameters.$parameterName.Keys | Should -Contain '$ref' -Because "the [$parameterName] parameter should use a user-defined type. For information please review the [AVM Specs]($link)."
}
} else {
Set-ItResult -Skipped -Because "the module template has no [$parameterName] parameter."
Expand Down
Loading

0 comments on commit da7e5cb

Please sign in to comment.