[Bug Report]: Using VNET Bicep Module in Azure Quickstarts fails validation

[dciborow]

Module path

modules/network/virtual-network/

Bicep version

0.12

Describe the bug

While using the virtualNetworks bicep module to in the Azure Quickstarts repo, 'best practice violations' are produced by armttk v0.12.

To reproduce

Azure/azure-quickstart-templates#12704

Code snippet

please see PR

Relevant log output

NestedTemplate [parameters('vnetName')] [ Lines 456 - 854 ]
    [?] apiVersions Should Be Recent (514 ms) 
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings

   NestedTemplate [format('{0}-virtualNetworkPeering-local-{1}', uniqueString(deployment().name, parameters('location')), copyIndex())] [ Lines 747 - 1145 ]
    [?] apiVersions Should Be Recent (514 ms) 
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings

   NestedTemplate [format('{0}-virtualNetworkPeering-remote-{1}', uniqueString(deployment().name, parameters('location')), copyIndex())] [ Lines 901 - 1299 ]
    [?] apiVersions Should Be Recent (514 ms) 
##[warning]        The apiVersion 2021-05-01 was not found for the resource type: Microsoft.Network/virtualNetworks/virtualNetworkPeerings

  Parameter Types Should Be Consistent
    [-] Parameter Types Should Be Consistent (357 ms) 
##[error]        Type Mismatch: Parameter 'roleDefinitionIdOrName' in nested template '[format('{0}-VNet-Rbac-{1}', uniqueString(deployment().name, parameters('location')), copyIndex())]' is defined as string, but the parent template defines it as array). Line: 1065, Column: 30
##[error]        Type Mismatch: Parameter 'localVnetName' in nested template '[format('{0}-virtualNetworkPeering-remote-{1}', uniqueString(deployment().name, parameters('location')), copyIndex())]' is defined as string, but the parent template defines it as array). Line: 919, Column: 30
##[error]        Type Mismatch: Parameter 'remoteVirtualNetworkId' in nested template '[format('{0}-virtualNetworkPeering-local-{1}', uniqueString(deployment().name, parameters('location')), copyIndex())]' is defined as string, but the parent template defines it as array). Line: 771, Column: 30
##[error]        Type Mismatch: Parameter 'addressPrefixes' in nested template '[parameters('vnetName')]' is defined as array, but the parent template defines it as string). Line: 480, Column: 22

    [-] Template Should Not Contain Blanks (472 ms) 
##[error]        Empty property:  null Line: 621, Column: 39
##[error]        Empty property:  null Line: 691, Column: 39

[alex-frankel]

I assume we can ignore the warnings? It looks like TTK is not aware of the most recent API version.

@bmoore-msft -- the type mismatch error is confusing. If there was a type mismatch, then bicep would be catching it as an error. Any idea what's happening there?

@MrMCake any insight into these TTK issues? I know the version in Azure/ResourceModules is more up to date, so if that one addresses the TTK issues then we can avoid using the module registry version.

[bmoore-msft]

re: type mismatch - my thoughts exactly - I'm guessing that's a bug in the TTK not accounting for the array element syntax in the param. The reference of the element turns the param into a simple type (that we assume matches a string) which should be fine. I don't think TTK pays attention to the [] after the param.

re: the apiVersion - vnetPeerings resource is not defined in the manifest so there are no apiVersions "published" (i.e. since we didn't return any from /providers). I thought we fell back to the parent type (and apiVersion) in that case but that might be on the backlog still...

[AlexanderSehr]

Hey @alex-frankel,

I experimented with ARM TTK a few times back in the IaCS days as well as with CARML as we wanted to make it a permanent part of our testing pipeline - and in either case it unfortunately became very quickly evident that it isn't made for templates beyond a certain complexity.

Regarding the API versions I guess we can all agree it's based on the age of the module in this repository - and would need to be updated to the latest in CARML.

By the way, we're currently experimenting with PSRule as an alternative to ARM TTK and hope it will yield better results. :)

[alex-frankel]

I wouldn't worry too much about TTK compliance given we are now (or very close to) parity with the bicep linter. I think I'm going to close this as there is no requirement that bicep modules be TTK compliant. I think there are larger issues with the networking module in particular as it was not usable with some of the other modules that @dciborow has submitted. It would be great to get to a point where we can take a shared dependency on a networking module.