Skip to content

Commit

Permalink
[Fixes] Fixed issue with multi-line parameter descriptions (Azure#4488)
Browse files Browse the repository at this point in the history 
* Ported multiline fix from AVM

* Update to latest

Added moved to avm disclaimer to store account

Added moved to avm disclaimer to app container-app

Added Moved to AVM for migrate modules (Azure#4495)

Added MOVED-TO-AVM (Azure#4498)

[Utilities] Enabled Workflow-Trigger function to trigger only for module diff (ported from AVM) (Azure#4499)

* Add support for triggering workflows based on file diff

* Dummy change to test diff

* Dummy change to test diff

* Adding diff module filter

* Removed dummy changes

Moved from CARML to AVM (Azure#4501)

Added missing slash (Azure#4508)

Added MOVED-TO-AVM.md for the machine-learning workspace (Azure#4505)

Added MOVED-TO-AVM.md for the purview account (Azure#4503)

Added MOVED-TO-AVM.md for the DevTest Lab and Network Manager Modules (Azure#4513)

* added dtl

* Update Network Manager modules

Refactor role assignment filtering in Get-RoleAssignmentList.ps1 (Azure#4515)

* Refactor role assignment filtering in Get-RoleAssignmentList.ps1

* fmt

* Add optional parameter to fetch all available roles

* Refactor Get-RoleAssignmentList.ps1 to fetch relevant Role Definitions by default

* Remove unnecessary debug statements in Get-RoleAssignmentList.ps1

Added MOVED-TO-AVM.md for the CDN Profile module (Azure#4512)

Added MOVED-TO-AVM.md for the configuration store (Azure#4506)

Added MOVED-TO-AVM.md for the Signal-R modules (Azure#4517)

[CI Environment] Ported latest AVM pipline changes (e.g. naming) (Azure#4477)

* Ported several changes

* Update to latest

* Update to latest

* Fixed PSRule setup

* Updated pester test file path

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Changed pwsh

* Update to latest

* Update to latest

* Tried alternative path ref

* Added PS Rule ref

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Update to latest

* Renamed paths

* Updated to more specific naming to avoid overlaps with e.g., pwsh path var

* Undid unrelated changes

* Added Az.Resources install to PSRule prep

* Update to latest

* GH PS Rule config fix

* Update to latest

* Update to latest

Added MOVED-TO-AVM.md for the recently migrated modules (Azure#4533)

* Added MOVED-TO-AVM for Relay Namespace

* Added MOVED-TO-AVM for Web Connection

Added MOVED-TO-AVM.md for the recently migrated modules (Azure#4538)

* Added MOVED-TO-AVM.md for the Healthcare APIs Workspace

* Update README.md and MOVED-TO-AVM.md for the Application Gateway Web Application Firewall (WAF) Policies module

CARML - AVM module transition complete (Azure#4541)

* readme updates

* readme update

* new template for MOVED-TO-AVM.md

* remove old notice from README files

* updated notice

* update informational notice in the MOVED-TO-AVM.md files

* update moved-to-avm notice

* moved-to-avm, readme and retired files updated

* readme update

* deleting modules' source code

* delete README files

* rename RETIRED.md and MOVED-TO-AVM.md to README.md

* readme update

* readme updates
  • Loading branch information
@AlexanderSehr
AlexanderSehr authored and Stephane Vallier committed Aug 7, 2024
1 parent e05c0ff commit a5653f7
Show file tree
Hide file tree
Showing 2,543 changed files with 1,900 additions and 488,285 deletions.
55 changes: 41 additions & 14 deletions .azuredevops/pipelineTemplates/jobs.getModuleTestFiles.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ parameters:

# Logic-related parameters
modulePath: '$(modulePath)'
psRuleFilterRegex: '(defaults|waf-aligned)'

##---------------------------------------------##
## TEMPLATE LOGIC ##
Expand All @@ -19,32 +20,58 @@ jobs:
name: ${{ parameters.poolName }}
steps:
- task: PowerShell@2
displayName: 'Get parameter files'
displayName: 'Get module test file paths'
name: getModuleTestFilesTask
inputs:
targetType: inline
pwsh: true
script: |
## ======= ##
## All ##
## ======= ##
# Get the list of parameter file paths
$moduleFolderPath = Join-Path '$(System.DefaultWorkingDirectory)' '${{ parameters.modulePath }}'
$testFilePaths = (Get-ChildItem -Path $moduleFolderPath -Recurse -Filter 'main.test.bicep').FullName | Sort-Object
$deploymentTestPaths = $testFilePaths | ForEach-Object {
$testFilePaths = (Get-ChildItem -Path $moduleFolderPath -Recurse -Filter 'main.test.bicep').FullName | Sort-Object
$testFilePaths = $testFilePaths | ForEach-Object {
$_.Replace($moduleFolderPath, '').Trim('\').Trim('/')
}
Write-Verbose 'Found module test files' -Verbose
$deploymentTestPaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
Write-Verbose 'Found all module test files' -Verbose
$testFilePaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
$testTable = @{}
foreach ($deploymentTestPath in $deploymentTestPaths) {
$deploymentTestFileName = Split-Path (Split-Path $deploymentTestPath -Parent) -Leaf
$testTable[$deploymentTestFileName] += @{
moduleTestFilePath = $deploymentTestPath
}
}
$testFilePaths | ForEach-Object {
$testFileName = Split-Path (Split-Path $_) -Leaf
$testTable[$testFileName] = @{
moduleTestFilePath = $_
moduleTestFileName = $testFileName
}
} | ConvertTo-Json -Compress
$deployCompressedOutput = $testTable | ConvertTo-Json -Compress
Write-Verbose "Publishing output: $deployCompressedOutput" -Verbose
Write-Host ('##vso[task.setVariable variable=moduleTestFilePaths;isOutput=true]{0}' -f $deployCompressedOutput)
$deploymentTestPathsOutput = $testTable | ConvertTo-Json -Compress
## =========== ##
## PS-Rule ##
## =========== ##
$psRuleTestFilePaths = $testFilePaths | Where-Object { $_ -match '${{ parameters.psRuleFilterRegex }}' }
Write-Verbose 'Found PSRule module test files' -Verbose
$psRuleTestFilePaths | ForEach-Object { Write-Verbose "- [$_]" -Verbose }
$psRuleTestTable = @{}
$psRuleTestFilePaths | ForEach-Object {
$testFileName = Split-Path (Split-Path $_) -Leaf
$psRuleTestTable[$testFileName] = @{
moduleTestFilePath = $_
moduleTestFileName = $testFileName
}
}
$psRuleCompressedOutput = $psRuleTestTable | ConvertTo-Json -Compress
Write-Host ('##vso[task.setVariable variable=moduleTests;isOutput=true]{0}' -f ($testTable | ConvertTo-Json -Compress))
Write-Verbose "Module test files: $deploymentTestPathsOutput" -Verbose
Write-Host ('##vso[task.setVariable variable=psRuleModuleTestFilePaths;isOutput=true]{0}' -f $psRuleCompressedOutput)
Write-Verbose "PS Rule publishing output: $psRuleCompressedOutput" -Verbose
16 changes: 5 additions & 11 deletions .azuredevops/pipelineTemplates/jobs.validateModuleDeployment.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@
## | vmImage | '$(vmImage)' | You can provide either a [poolname] or [vmImage] to run the job on. | 'ubuntu20.04' |
## | defaultJobTimeoutInMinutes | 120 | The timeout for the job in this pipeline. | 120 |
## | removeDeployment | 'true' | Set to [true] to flag resources for removal. If not provided, defaults to true. | 'true' |
## | templateFilePath | '' | Path to the template file to deploy. | 'modules/analysis-services/servers/main.bicep' |
## | customTokens | '' | Additional token pairs in json format. | '{"tokenName":"tokenValue"}' |
## | jobDisplayName | '' | The display name of the job. | 'Deploy module' |
## | modulePath | '$(modulePath)' | The path to the module to deploy. | 'c:/KeyVault' |
Expand All @@ -50,7 +49,6 @@ parameters:
defaultJobTimeoutInMinutes: 120
# Logic-related parameters
removeDeployment: false
templateFilePath: ''
customTokens: ''
modulePath: '$(modulePath)'
location: '$(location)'
Expand All @@ -76,7 +74,7 @@ jobs:
dependsOn:
- getModuleTestFiles
strategy:
matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.moduleTests'] ]
matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.moduleTestFilePaths'] ]
##---------------------------------------------##
## TEMPLATE LOGIC ##
##---------------------------------------------##
Expand Down Expand Up @@ -117,20 +115,16 @@ jobs:
# [Agent] Replace tokens
#-----------------------
- task: AzurePowerShell@5
displayName: 'Replace tokens in template file via connection [${{ parameters.serviceConnection }}]'
- task: PowerShell@2
displayName: 'Replace tokens in template files'
inputs:
azureSubscription: ${{ parameters.serviceConnection }}
azurePowerShellVersion: 'latestVersion'
preferredAzurePowerShellVersion: ''
ScriptType: InlineScript
targetType: inline
pwsh: true
inline: |
script: |
# Load used functions
. (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFileList.ps1')
. (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'sharedScripts' 'Get-LocallyReferencedFileList.ps1')
# Get target files
$moduleTestFilePath = Join-Path '$(System.DefaultWorkingDirectory)' '$(modulePath)' '$(moduleTestFilePath)'
Expand Down
144 changes: 144 additions & 0 deletions .azuredevops/pipelineTemplates/jobs.validateModulePSRule.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
#########################################################
## 'Validate module with Pester' Pipeline Template ##
#########################################################
##
## This pipeline template contains the logic to validate a module using a set of Pester tests
##
## Enabled levels of validation
## - Resource-Group-Level
## - Subscription-Level
## - Management-Group-Level
## - Tenant-Level
##
#########################################################
##
##---------------------------------------------##
## TEMPLATE PARAMETERS ##
##---------------------------------------------##
##
## By default it uses the variables specified in the below [parameters] section. However, you can overwrite these variables in the
## referencing pipeline by providing the parameter explicitly.
##
## NOTE: If you don't need to overwrite a shared value, you can IGNORE this section
##
## |==============================================================================================================================================================================================================================================|
## | Parameter | Default Value | Description | Example |
## |----------------------------|-----------------------------------------------|-------------------------------------------------------------------------------------------------------|---------------------------------------------------------|
## | poolName | '$(poolName)' | You can provide either a [poolname] or [vmImage] to run the job on | 'Custom Deployment Pool' |
## | vmImage | '$(vmImage)' | You can provide either a [poolname] or [vmImage] to run the job on | 'ubuntu20.04' |
## | defaultJobTimeoutInMinutes | 120 | The timeout for the job in this pipeline | 120 |
## | modulePath | '$(modulePath)' | The path to the module to deploy. | 'c:/KeyVault' |
## | psrulePath | 'utilities/pipelines/staticValidation/psrule' | The path to the PS-Rule configuration | 'utilities/pipelines/staticValidation/module.tests.ps1' |
## | location | '$(location)' | The location to validate with | 'France Central' |
## | subscriptionId | '$(ARM_SUBSCRIPTION_ID)' | The id of the subscription to validate with when using a Management group service connection | 'aed7c000-6387-412e-bed0-24dfddf4bbc6' |
## | managementGroupId | '$(ARM_MGMTGROUP_ID)' | The id of the management group to validate with. Required only for Management-Group-Level validations | '477c9620-cb01-454f-9ebc-fc6b1df48c14' |
## |==============================================================================================================================================================================================================================================|
##
##---------------------------------------------##

parameters:
# Pipeline-related parameters
poolName: '$(poolName)'
vmImage: '$(vmImage)'
defaultJobTimeoutInMinutes: 120
# Logic-related parameters
modulePath: '$(modulePath)'
psrulePath: 'utilities/pipelines/staticValidation/psrule'
location: '$(location)'
subscriptionId: '$(ARM_SUBSCRIPTION_ID)'
managementGroupId: '$(ARM_MGMTGROUP_ID)'

##---------------------------------------------##
## TEMPLATE LOGIC ##
##---------------------------------------------##
jobs:
- template: /.azuredevops/pipelineTemplates/jobs.getModuleTestFiles.yml
- job:
displayName: Run PSRule tests
timeoutInMinutes: ${{ parameters.defaultJobTimeoutInMinutes }}
pool:
${{ if ne(parameters.vmImage, '') }}:
vmImage: ${{ parameters.vmImage }}
${{ if ne(parameters.poolName, '') }}:
name: ${{ parameters.poolName }}
dependsOn:
- getModuleTestFiles
strategy:
matrix: $[ dependencies.getModuleTestFiles.outputs['getModuleTestFilesTask.psRuleModuleTestFilePaths'] ]
steps:
# [Agent] Replace tokens
#-----------------------
- task: PowerShell@2
displayName: 'Replace tokens in template files'
inputs:
targetType: inline
pwsh: true
script: |
# Load used functions
. (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'tokensReplacement' 'Convert-TokensInFileList.ps1')
. (Join-Path '$(System.DefaultWorkingDirectory)' 'utilities' 'pipelines' 'sharedScripts' 'Get-LocallyReferencedFileList.ps1')
# Get target files
$moduleTestFilePath = Join-Path '$(System.DefaultWorkingDirectory)' '$(modulePath)' '$(moduleTestFilePath)'
# Get target files
$targetFileList = @(
$moduleTestFilePath
)
# Add all module template files as they may contain tokens
$targetFileList += (Get-LocallyReferencedFileList -FilePath $moduleTestFilePath)
$targetFileList = $targetFileList | Sort-Object -Unique
# Construct Token Function Input
$ConvertTokensInputs = @{
FilePathList = $targetFileList
Tokens = @{}
TokenPrefix = '$(tokenPrefix)'
TokenSuffix = '$(tokenSuffix)'
}
# Add enforced tokens
$ConvertTokensInputs.Tokens += @{
subscriptionId = '${{ parameters.subscriptionId }}'
managementGroupId = '${{ parameters.managementGroupId }}'
tenantId = '$(ARM_TENANT_ID)'
}
# Add local (source control) tokens
$tokenMap = @{}
foreach ($token in (Get-ChildItem env: | Where-Object -Property Name -Like "localToken_*")) {
$tokenMap += @{ $token.Name.Replace('localToken_','','OrdinalIgnoreCase') = $token.value }
}
Write-Verbose ('Using local tokens [{0}]' -f ($tokenMap.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens += $tokenMap
# Swap 'namePrefix' token if empty and provided as a Azure DevOps variable
if([String]::IsNullOrEmpty($ConvertTokensInputs.Tokens['namePrefix'])){
Write-Verbose 'Using [namePrefix] token from Azure DevOps Variable Groups' -Verbose
$ConvertTokensInputs.Tokens['namePrefix'] = "$(TOKEN_NAMEPREFIX)"
}
# Add custom tokens (passed in via the pipeline)
if(-not [String]::IsNullOrEmpty('${{ parameters.customTokens }}')) {
$customTokens = '${{ parameters.customTokens }}' | ConvertFrom-Json -AsHashTable
Write-Verbose ('Using custom parameter file tokens [{0}]' -f ($customTokens.Keys -join ', ')) -Verbose
$ConvertTokensInputs.Tokens += $customTokens
}
Write-Verbose "Convert Tokens Input:`n $($ConvertTokensInputs | ConvertTo-Json -Depth 10)" -Verbose
# Invoke Token Replacement Functionality [For Module]
$null = Convert-TokensInFileList @ConvertTokensInputs
- task: ps-rule-assert@2
displayName: Analyze Azure template files
inputs:
inputType: inputPath
modules: 'PSRule.Rules.Azure'
inputPath: '$(System.DefaultWorkingDirectory)/$(modulePath)/$(moduleTestFilePath)'
outputFormat: Csv
option: '${{ parameters.psrulePath}}/ps-rule.yaml' # Path to PSRule configuration options file
source: '${{ parameters.psrulePath}}/.ps-rule/' # Path to folder containing suppression rules to use for analysis.
outputPath: '$(System.DefaultWorkingDirectory)/$(modulePath)/$(moduleTestFilePath)-PSRule-output.csv'
continueOnError: true
Loading

0 comments on commit a5653f7

Please sign in to comment.