Loops suggestions and discussion

[slavizh]

The proposed specs for loops implementation is available here:
https://github.com/Azure/bicep/blob/master/docs/spec/loops.md

Based on that I would like to open a few things for discussion:

• One of the statements in the docs are not true: "Multiple loops may be nested inside each other.”. This is not true due to limit in ARM Templates that needs to be fixed. Basically ARM templates does not support nested loops. The only way they are supported if you loop on the resource and then loop on properties but you cannot have looping on one property and then looping on sub property of that property. This kind of looping is needed in many resources. One example is Application Gateway RP https://docs.microsoft.com/en-us/azure/templates/microsoft.network/applicationgateways. There we have property that is urlPathMaps and it is array. Inside that property we have pathRules that is also array. I would like to be able to loop on both of these. Currently we do this with collect and transform linked deployments as I do not know a way for being possible in ARM Templates. Basically we need ARM templates to support large number of nested loops to enable these scenarios. I say large number as we need to cover any current APIs and see at what lowest level they have arrays within arrays and for any future RP.

• The syntax for loops for me personally looks a little bit unfamiliar. At some point I can get used to it but what I am proposing and looking if more people would agree or express may be completely different opinion is to have a loop syntax that is familiar to other languages. The current loop proposed syntax you first define the resources as array, type and API version and then your loop is defined. In many other languages (like PowerShell for example) you first define the loop and than you define the resource. A simple mock-up of how that could look is below:

// array of storage account names
param storageAccounts array

for (storageName in storageAccounts)
{
  resource storageAccountResource + storageName 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageName
  location: resourceGroup().location
  properties: {
    supportsHttpsTrafficOnly: true
    accessTier: 'Hot'
    encryption: {
      keySource: 'Microsoft.Storage'
      services: {
        blob: {
          enabled: true
        }
        file: {
          enabled: true
        }
      }
    }
  }
  kind: StorageV2
  sku: {
    name: 'Standard_LRS'
  }
}

This is just a mock-up that requires some things to be resolved like:

• What will be the exact key word and syntax for loop?

• How the name of the resource should be build?

• Any other suggestion on improving or completely changing the syntax?

• Last topic is actually a question to the PG. If we take the current proposed syntax and the last example from the doc will we be possible to do something like this:

var defaultObject = {
                enabled: false
}

resource[] parentResources 'Microsoft.Example/examples@2020-06-06' = [for parent in parents where union(defaultObject, parent).enabled: {
  name: parent.name
  properties: {
    children: [for child in parent.children where parent.includeChildren && child.enabled: {
      name: child.name
      setting: child.settingValue
    }]
  }
}]

Note the union function in there.

• With the proposed loop syntax can we still do loops in variables and outputs?

[alex-frankel]

We played around with a syntax like this, but we got stuck on how do I reference things in the loop outside of the loop? e.g. I want to get the blob uri for just the first storage account in the loop. There's not really an equivalent paradigm in imperative programming languages because a loop defines a scope and things created in that scope don't exist outside of the scope.

We played around with a list keyword with a syntax like:

list storageResources for account in storageAccounts {
  resources stg '...' {...}

  // potentially could have other resources in this loop
}

output blobUri string = storageResources.stg[0].properties.primaryEndpoints.blob

Lots of context in #6 that may be helpful.

[slavizh]

@alex-frankel ok. I see. It is difficult problem as you also have the loop in properties which looks a little bit different but at the same time is the same like the one for resource. The 'as' proposal is not so bad. Let' say it stays the current syntax the '=' makes it a little bit better in terms of readability. Strangely that the '=' looks better in loops then in single resource for me. In case we stay with the current syntax is there something that can be done for readability? Something like may be the whole 'for' be on separate line as it will be longer when you have conditions and for.

Besides the syntax discussion there are two questions (the last two) around what is possible with loops. Can they be answered currently or it is something we have to wait and see?

[alex-frankel]

Something like may be the whole 'for' be on separate line as it will be longer when you have conditions and for.

Yes, we will allow you to break this into multiple lines. Bicep being sensitive to line-breaks is a temporary limitation that will go away by 0.3.

Besides the syntax discussion there are two questions (the last two) around what is possible with loops. Can they be answered currently or it is something we have to wait and see?

Loops will work with variables and outputs.

Is the other question about nested property loops? I think our goal would be to update the IL to support nested property loops and then allow it in bicep. That being said, we haven't done any investigations on this, so I may be missing something on the complexities of making this work. @bmoore-msft can you confirm we don't support this today?

[bmoore-msft]

Nested loops are not supported inline, but you can workaround some cases, with nested deployments or variables... Not sure how many scenarios there are for this, @slavizh would love to see a template or snippet.

Re: looping one of the things to keep in mind is that our "programmer instinct" wants this to look and behave like an imperative programming language. ARM (and bicep) is still declarative and there are some subtle differences when you describe "what" (I want n copies of this thing) vs. "how" (run these things through a loop). It is subtle, but causing a bit of consternation.

I wish we could do something as simple as a copies property and it just works, but too many other property values depend on the index/counter.

[slavizh]

The other questions was around using functions like union() in the 'for' loop definition. Would that be possible.

@bmoore-msft Nested loops within properties are quite often as there are many APIs that have array properties within array properties and honestly it is better to support this natively rather we creating linked deployments just for that. It results in longer deployment times and more resources being consumed on your end just because we start deployments. I will send a few examples in private.

[bmoore-msft]

ack - in order for us to support in bicep, we'll need to also support in the json... I suspect this is not a simple change... I agree the work around is less elegant than anyone would like it's just odd that it hasn't come up much if common. Will take a look at the examples and see if I can figure them out ;)

[slavizh]

@bmoore-msft using union() in loops is supported and we use it. I gave it just to verify if you will not miss that scenario.

[MCKLMT]

I'm agree with @slavizh. This syntax feels more friendly because it looks like C#.

// array of storage account names
param storageAccounts array

for (storageName in storageAccounts)
{
  resource storageAccountResource + storageName 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: storageName
  location: resourceGroup().location
  properties: {
    supportsHttpsTrafficOnly: true
    accessTier: 'Hot'
    encryption: {
      keySource: 'Microsoft.Storage'
      services: {
        blob: {
          enabled: true
        }
        file: {
          enabled: true
        }
      }
    }
  }
  kind: StorageV2
  sku: {
    name: 'Standard_LRS'
  }
}

Also, it would be cool to support for syntax and not only foreach.
With the mockup above, it could looks like:

// array of storage account names
param storageCount int = 5

for (i = 0; i < storageCount; i++)
{
  resource storageAccountResource + i 'Microsoft.Storage/storageAccounts@2019-06-01' = {
  name: 'storageAccount' + i
  location: resourceGroup().location
  properties: {
    supportsHttpsTrafficOnly: true
    accessTier: 'Hot'
    encryption: {
      keySource: 'Microsoft.Storage'
      services: {
        blob: {
          enabled: true
        }
        file: {
          enabled: true
        }
      }
    }
  }
  kind: StorageV2
  sku: {
    name: 'Standard_LRS'
  }
}

Using the current specification:

// array of storage account names
param storageCount int

resource[] storageAccountResources 'Microsoft.Storage/storageAccounts@2019-06-01' = [for i = 0; i < storageCount; i++: {
  name: 'storageAccount' + i
  location: resourceGroup().location
  properties: {
    supportsHttpsTrafficOnly: true
    accessTier: 'Hot'
    encryption: {
      keySource: 'Microsoft.Storage'
      services: {
        blob: {
          enabled: true
        }
        file: {
          enabled: true
        }
      }
    }
  }
  kind: StorageV2
  sku: {
    name: 'Standard_LRS'
  }
}]

What do you think?

[MCKLMT]

Any idea when this will be implemented? It is a blocker for a ton of templates. Thanks

[alex-frankel]

We're planning to have looping support released sometime in Jan

[asyncoder]

Hi @alex-frankel, any chance if this feature will be released in the coming releases? Thanks.

[anthony-c-martin]

Hi @alex-frankel, any chance if this feature will be released in the coming releases? Thanks.

It’ll be in the 0.3 release - the PR is in review at the moment.

[MCKLMT]

Is "for i =0; i < count; i++" implemented also in 0.3.1?

[anthony-c-martin]

Is "for i =0; i < count; i++" implemented also in 0.3.1?

You can accomplish this with the range() function - e.g.:

resource myRes 'My.Rp/myTyp@2020-01-01' = [for i in range(0, count): {
  name: 'res${i}'
  ...
}]

[alex-frankel]

Going to close this since loops are implemented. Feel free to keep commenting with feedback here if it makes sense or create a separate issue where appropriate.