Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Aligned AKS interface to AVM specs & added UDT #3506

Merged
merged 16 commits into from
Oct 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
577 changes: 468 additions & 109 deletions avm/res/container-service/managed-cluster/README.md

Large diffs are not rendered by default.

22 changes: 15 additions & 7 deletions avm/res/container-service/managed-cluster/agent-pool/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,14 +47,14 @@ This module deploys an Azure Kubernetes Service (AKS) Managed Cluster Agent Pool
| [`minCount`](#parameter-mincount) | int | The minimum number of nodes for auto-scaling. |
| [`mode`](#parameter-mode) | string | A cluster must have at least one "System" Agent Pool at all times. For additional information on agent pool restrictions and best practices, see: /azure/aks/use-system-pools. |
| [`nodeLabels`](#parameter-nodelabels) | object | The node labels to be persisted across all nodes in agent pool. |
| [`nodePublicIpPrefixId`](#parameter-nodepublicipprefixid) | string | ResourceId of the node PublicIPPrefix. |
| [`nodePublicIpPrefixResourceId`](#parameter-nodepublicipprefixresourceid) | string | ResourceId of the node PublicIPPrefix. |
| [`nodeTaints`](#parameter-nodetaints) | array | The taints added to new nodes during node pool create and scale. For example, key=value:NoSchedule. |
| [`orchestratorVersion`](#parameter-orchestratorversion) | string | As a best practice, you should upgrade all node pools in an AKS cluster to the same Kubernetes version. The node pool version must have the same major version as the control plane. The node pool minor version must be within two minor versions of the control plane version. The node pool version cannot be greater than the control plane version. For more information see upgrading a node pool (https://learn.microsoft.com/en-us/azure/aks/use-multiple-node-pools#upgrade-a-node-pool). |
| [`osDiskSizeGB`](#parameter-osdisksizegb) | int | OS Disk Size in GB to be used to specify the disk size for every machine in the master/agent pool. If you specify 0, it will apply the default osDisk size according to the vmSize specified. |
| [`osDiskType`](#parameter-osdisktype) | string | The default is "Ephemeral" if the VM supports it and has a cache disk larger than the requested OSDiskSizeGB. Otherwise, defaults to "Managed". May not be changed after creation. For more information see Ephemeral OS (https://learn.microsoft.com/en-us/azure/aks/cluster-configuration#ephemeral-os). |
| [`osSku`](#parameter-ossku) | string | Specifies the OS SKU used by the agent pool. The default is Ubuntu if OSType is Linux. The default is Windows2019 when Kubernetes <= 1.24 or Windows2022 when Kubernetes >= 1.25 if OSType is Windows. |
| [`osType`](#parameter-ostype) | string | The operating system type. The default is Linux. |
| [`podSubnetId`](#parameter-podsubnetid) | string | Subnet ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}. |
| [`podSubnetResourceId`](#parameter-podsubnetresourceid) | string | Subnet resource ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}. |
| [`proximityPlacementGroupResourceId`](#parameter-proximityplacementgroupresourceid) | string | The ID for the Proximity Placement Group. |
| [`scaleDownMode`](#parameter-scaledownmode) | string | Describes how VMs are added to or removed from Agent Pools. See [billing states](https://learn.microsoft.com/en-us/azure/virtual-machines/states-billing). |
| [`scaleSetEvictionPolicy`](#parameter-scalesetevictionpolicy) | string | The eviction policy specifies what to do with the VM when it is evicted. The default is Delete. For more information about eviction see spot VMs. |
Expand All @@ -64,7 +64,7 @@ This module deploys an Azure Kubernetes Service (AKS) Managed Cluster Agent Pool
| [`tags`](#parameter-tags) | object | Tags of the resource. |
| [`type`](#parameter-type) | string | The type of Agent Pool. |
| [`vmSize`](#parameter-vmsize) | string | VM size. VM size availability varies by region. If a node contains insufficient compute resources (memory, cpu, etc) pods might fail to run correctly. For more details on restricted VM sizes, see: /azure/aks/quotas-skus-regions. |
| [`vnetSubnetId`](#parameter-vnetsubnetid) | string | Node Subnet ID. If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}. |
| [`vnetSubnetResourceId`](#parameter-vnetsubnetresourceid) | string | Node Subnet ID. If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}. |
| [`workloadRuntime`](#parameter-workloadruntime) | string | Determines the type of workload a node can run. |

### Parameter: `name`
Expand All @@ -87,6 +87,14 @@ The list of Availability zones to use for nodes. This can only be specified if t

- Required: No
- Type: array
- Default:
```Bicep
[
1
2
3
]
```

### Parameter: `count`

Expand Down Expand Up @@ -202,7 +210,7 @@ The node labels to be persisted across all nodes in agent pool.
- Required: No
- Type: object

### Parameter: `nodePublicIpPrefixId`
### Parameter: `nodePublicIpPrefixResourceId`

ResourceId of the node PublicIPPrefix.

Expand Down Expand Up @@ -276,9 +284,9 @@ The operating system type. The default is Linux.
]
```

### Parameter: `podSubnetId`
### Parameter: `podSubnetResourceId`

Subnet ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.
Subnet resource ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.

- Required: No
- Type: string
Expand Down Expand Up @@ -370,7 +378,7 @@ VM size. VM size availability varies by region. If a node contains insufficient
- Type: string
- Default: `'Standard_D2s_v3'`

### Parameter: `vnetSubnetId`
### Parameter: `vnetSubnetResourceId`

Node Subnet ID. If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ param managedClusterName string
param name string

@description('Optional. The list of Availability zones to use for nodes. This can only be specified if the AgentPoolType property is "VirtualMachineScaleSets".')
param availabilityZones array?
param availabilityZones int[] = [1, 2, 3]

@description('Optional. Desired Number of agents (VMs) specified to host docker containers. Allowed values must be in the range of 0 to 1000 (inclusive) for user pools and in the range of 1 to 1000 (inclusive) for system pools. The default value is 1.')
@minValue(0)
Expand Down Expand Up @@ -63,7 +63,7 @@ param mode string?
param nodeLabels object?

@description('Optional. ResourceId of the node PublicIPPrefix.')
param nodePublicIpPrefixId string?
param nodePublicIpPrefixResourceId string?

@description('Optional. The taints added to new nodes during node pool create and scale. For example, key=value:NoSchedule.')
param nodeTaints array?
Expand Down Expand Up @@ -98,8 +98,8 @@ param osSku string?
])
param osType string = 'Linux'

@description('Optional. Subnet ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.')
param podSubnetId string?
@description('Optional. Subnet resource ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.')
param podSubnetResourceId string?

@description('Optional. The ID for the Proximity Placement Group.')
param proximityPlacementGroupResourceId string?
Expand Down Expand Up @@ -141,7 +141,7 @@ param maxSurge string?
param vmSize string = 'Standard_D2s_v3'

@description('Optional. Node Subnet ID. If this is not specified, a VNET and subnet will be generated and used. If no podSubnetID is specified, this applies to nodes and pods, otherwise it applies to just nodes. This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}.')
param vnetSubnetId string?
param vnetSubnetResourceId string?

@description('Optional. Determines the type of workload a node can run.')
param workloadRuntime string?
Expand All @@ -154,7 +154,7 @@ resource agentPool 'Microsoft.ContainerService/managedClusters/agentPools@2023-0
name: name
parent: managedCluster
properties: {
availabilityZones: availabilityZones
availabilityZones: map(availabilityZones ?? [], zone => '${zone}')
count: count
creationData: !empty(sourceResourceId)
? {
Expand All @@ -173,14 +173,14 @@ resource agentPool 'Microsoft.ContainerService/managedClusters/agentPools@2023-0
minCount: minCount
mode: mode
nodeLabels: nodeLabels
nodePublicIPPrefixID: nodePublicIpPrefixId
nodePublicIPPrefixID: nodePublicIpPrefixResourceId
nodeTaints: nodeTaints
orchestratorVersion: orchestratorVersion
osDiskSizeGB: osDiskSizeGB
osDiskType: osDiskType
osSKU: osSku
osType: osType
podSubnetID: podSubnetId
podSubnetID: podSubnetResourceId
proximityPlacementGroupID: proximityPlacementGroupResourceId
scaleDownMode: scaleDownMode
scaleSetEvictionPolicy: scaleSetEvictionPolicy
Expand All @@ -192,7 +192,7 @@ resource agentPool 'Microsoft.ContainerService/managedClusters/agentPools@2023-0
maxSurge: maxSurge
}
vmSize: vmSize
vnetSubnetID: vnetSubnetId
vnetSubnetID: vnetSubnetResourceId
workloadRuntime: workloadRuntime
}
}
Expand Down
29 changes: 18 additions & 11 deletions avm/res/container-service/managed-cluster/agent-pool/main.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
"metadata": {
"_generator": {
"name": "bicep",
"version": "0.29.47.4906",
"templateHash": "2004205618690542488"
"version": "0.30.23.60470",
"templateHash": "13856766172443517827"
},
"name": "Azure Kubernetes Service (AKS) Managed Cluster Agent Pools",
"description": "This module deploys an Azure Kubernetes Service (AKS) Managed Cluster Agent Pool.",
Expand All @@ -27,7 +27,14 @@
},
"availabilityZones": {
"type": "array",
"nullable": true,
"items": {
"type": "int"
},
"defaultValue": [
1,
2,
3
],
"metadata": {
"description": "Optional. The list of Availability zones to use for nodes. This can only be specified if the AgentPoolType property is \"VirtualMachineScaleSets\"."
}
Expand Down Expand Up @@ -139,7 +146,7 @@
"description": "Optional. The node labels to be persisted across all nodes in agent pool."
}
},
"nodePublicIpPrefixId": {
"nodePublicIpPrefixResourceId": {
"type": "string",
"nullable": true,
"metadata": {
Expand Down Expand Up @@ -203,11 +210,11 @@
"description": "Optional. The operating system type. The default is Linux."
}
},
"podSubnetId": {
"podSubnetResourceId": {
"type": "string",
"nullable": true,
"metadata": {
"description": "Optional. Subnet ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}."
"description": "Optional. Subnet resource ID for the pod IPs. If omitted, pod IPs are statically assigned on the node subnet (see vnetSubnetID for more details). This is of the form: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Network/virtualNetworks/{virtualNetworkName}/subnets/{subnetName}."
}
},
"proximityPlacementGroupResourceId": {
Expand Down Expand Up @@ -285,7 +292,7 @@
"description": "Optional. VM size. VM size availability varies by region. If a node contains insufficient compute resources (memory, cpu, etc) pods might fail to run correctly. For more details on restricted VM sizes, see: /azure/aks/quotas-skus-regions."
}
},
"vnetSubnetId": {
"vnetSubnetResourceId": {
"type": "string",
"nullable": true,
"metadata": {
Expand All @@ -312,7 +319,7 @@
"apiVersion": "2023-07-02-preview",
"name": "[format('{0}/{1}', parameters('managedClusterName'), parameters('name'))]",
"properties": {
"availabilityZones": "[parameters('availabilityZones')]",
"availabilityZones": "[map(coalesce(parameters('availabilityZones'), createArray()), lambda('zone', format('{0}', lambdaVariables('zone'))))]",
"count": "[parameters('count')]",
"creationData": "[if(not(empty(parameters('sourceResourceId'))), createObject('sourceResourceId', parameters('sourceResourceId')), null())]",
"enableAutoScaling": "[parameters('enableAutoScaling')]",
Expand All @@ -327,14 +334,14 @@
"minCount": "[parameters('minCount')]",
"mode": "[parameters('mode')]",
"nodeLabels": "[parameters('nodeLabels')]",
"nodePublicIPPrefixID": "[parameters('nodePublicIpPrefixId')]",
"nodePublicIPPrefixID": "[parameters('nodePublicIpPrefixResourceId')]",
"nodeTaints": "[parameters('nodeTaints')]",
"orchestratorVersion": "[parameters('orchestratorVersion')]",
"osDiskSizeGB": "[parameters('osDiskSizeGB')]",
"osDiskType": "[parameters('osDiskType')]",
"osSKU": "[parameters('osSku')]",
"osType": "[parameters('osType')]",
"podSubnetID": "[parameters('podSubnetId')]",
"podSubnetID": "[parameters('podSubnetResourceId')]",
"proximityPlacementGroupID": "[parameters('proximityPlacementGroupResourceId')]",
"scaleDownMode": "[parameters('scaleDownMode')]",
"scaleSetEvictionPolicy": "[parameters('scaleSetEvictionPolicy')]",
Expand All @@ -346,7 +353,7 @@
"maxSurge": "[parameters('maxSurge')]"
},
"vmSize": "[parameters('vmSize')]",
"vnetSubnetID": "[parameters('vnetSubnetId')]",
"vnetSubnetID": "[parameters('vnetSubnetResourceId')]",
"workloadRuntime": "[parameters('workloadRuntime')]"
},
"dependsOn": [
Expand Down
Loading
Loading