Skip to content

Commit

Permalink
Introduce Azure Accelerated Networking
Browse files Browse the repository at this point in the history
for machine type/os image combinations which support it.
  • Loading branch information
dkistner committed Apr 7, 2020
1 parent 82b4bab commit 4b49aa1
Show file tree
Hide file tree
Showing 12 changed files with 142 additions and 25 deletions.
12 changes: 8 additions & 4 deletions charts/internal/machineclass/templates/machineclass.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ spec:
{{- if hasKey $machineClass "identityID" }}
identityID: {{ $machineClass.identityID }}
{{- end }}
{{- if hasKey $machineClass.network "acceleratedNetworking" }}
networkProfile:
acceleratedNetworking: {{ $machineClass.network.acceleratedNetworking }}
{{- end }}
hardwareProfile:
vmSize: {{ $machineClass.machineType }}
osProfile:
Expand Down Expand Up @@ -65,11 +69,11 @@ spec:
name: {{ $machineClass.name }}
namespace: {{ $.Release.Namespace }}
subnetInfo:
vnetName: {{ $machineClass.vnetName }}
{{- if hasKey $machineClass "vnetResourceGroup" }}
vnetResourceGroup: {{ $machineClass.vnetResourceGroup}}
vnetName: {{ $machineClass.network.vnet }}
{{- if hasKey $machineClass.network "vnetResourceGroup" }}
vnetResourceGroup: {{ $machineClass.network.vnetResourceGroup}}
{{- end }}
subnetName: {{ $machineClass.subnetName }}
subnetName: {{ $machineClass.network.subnet }}
{{- if $machineClass.tags }}
tags:
{{ toYaml $machineClass.tags | indent 4 }}
Expand Down
12 changes: 8 additions & 4 deletions charts/internal/machineclass/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@ machineClasses:
# foo: bar
region: westeurope
resourceGroup: my-resource-group
vnetName: my-vnet
subnetName: my-subnet-in-my-vnet
zone: 1
# identityID: /subscriptions/subscription-id/resourceGroups/resource-group-name/providers/Microsoft.ManagedIdentity/userAssignedIdentities/identity-name
network:
vnet: my-vnet
subnet: my-subnet-in-my-vnet
# vnetResourceGroup: my-vnet-resource-group
# acceleratedNetworking: true
tags:
Name: shoot-crazy-botany
kubernetes.io-cluster-shoot-crazy-botany: "1"
Expand All @@ -29,8 +32,9 @@ machineClasses:
- name: class-2-availability-set
region: westeurope
resourceGroup: my-resource-group
vnetName: my-vnet
subnetName: my-subnet-in-my-vnet
network:
vnet: my-vnet
subnet: my-subnet-in-my-vnet
availabilitySetID: /subscriptions/subscription-id/resourceGroups/resource-group-name/providers/Microsoft.Compute/availabilitySets/availablity-set-name
tags:
Name: shoot-crazy-botany
Expand Down
6 changes: 6 additions & 0 deletions docs/usage-as-end-user.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,3 +210,9 @@ spec:
nginxIngress:
enabled: true
```

## Miscellaneous

### Azure Accelerated Networking
Machine(s) will be automatically configured to use [Azure Accelerated Networking](https://docs.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli) in case the machine type and machine image version is configured in the CloudProfile to be compatible with Accelerated Networking.
Supported machine types are listed in `.spec.providerConfig.acceleratedNetworingMachineTypes[]` and the supported machine image versions are listed in `.spec.providerConfig.machineImages[].versions[].acceleratedNetworking`.
23 changes: 19 additions & 4 deletions docs/usage-as-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@ In this document we are describing how this configuration looks like for Azure a

## `CloudProfileConfig`

The cloud profile configuration contains information about the update and failure domain counts in the Azure regions you want to offer.
Additionally, it contains the real machine image identifiers in the Azure environment. You can provide either URN or image ID.
You have to map every version that you specify in `.spec.machineImages[].versions` here such that the Azure extension knows the machine image identifiers for every version you want to offer.

An example `CloudProfileConfig` for the Azure extension looks as follows:

```yaml
Expand All @@ -21,15 +17,25 @@ countUpdateDomains:
countFaultDomains:
- region: westeurope
count: 3
acceleratedNetworingMachineTypes:
- Standard_D3_v2
machineImages:
- name: coreos
versions:
- version: 2135.6.0
urn: "CoreOS:CoreOS:Stable:2135.6.0"
acceleratedNetworking: true
- version: 2303.3.0
id: "/Subscriptions/4bfa08b6-bad8-4b8e-aa00-741c0a859e36/Providers/Microsoft.Compute/Locations/westus/Publishers/CoreOS/ArtifactTypes/VMImage/Offers/CoreOS/Skus/Stable/Versions/2303.3.0"
```
The cloud profile configuration contains information about the update via `.countUpdateDomains[]` and failure domain via `.countFaultDomains[]` counts in the Azure regions you want to offer.

Additionally, it contains the real machine image identifiers in the Azure environment. You have to map every image version that you specify in `.spec.machineImages[].versions` such that the Azure extension knows the machine image identifiers for every version you want to offer. You can provide either URN (`.machineImages[].versions[].urn`) or image ID (`.machineImages[].versions[].id`) for each image version. It's also possible to specify for each image version if the version is supporting Azure Accelerated Networking via `.machineImages[].versions[].acceleratedNetworking`.

In `.acceleratedNetworingMachineTypes[]` you can specify the machine types which are compatible with [Azure Accelerated Networking](https://docs.microsoft.com/en-us/azure/virtual-network/create-vm-accelerated-networking-cli).


## Example `CloudProfile` manifest

The possible values for `.spec.volumeTypes[].name` on Azure are `Standard_LRS`, `StandardSSD_LRS` and `Premium_LRS`. There is another volume type called `UltraSSD_LRS` but this type is not supported to use as os disk. If an end user select a volume type whose name is not equal to one of the valid values then the machine will be created with the default volume type which belong to the selected machine type. Therefore it is recommended to configure only the valid values for the `.spec.volumeType[].name` in the `CloudProfile`.
Expand All @@ -53,6 +59,10 @@ spec:
versions:
- version: 2135.6.0
machineTypes:
- name: Standard_D3_v2
cpu: "4"
gpu: "0"
memory: 14Gi
- name: Standard_D4_v3
cpu: "4"
gpu: "0"
Expand All @@ -72,6 +82,8 @@ spec:
providerConfig:
apiVersion: azure.provider.extensions.gardener.cloud/v1alpha1
kind: CloudProfileConfig
acceleratedNetworingMachineTypes:
- Standard_D3_v2
countUpdateDomains:
- region: westeurope
count: 5
Expand All @@ -81,6 +93,9 @@ spec:
machineImages:
- name: coreos
versions:
- version: 2303.3.0
urn: CoreOS:CoreOS:Stable:2303.3.0
acceleratedNetworking: true
- version: 2135.6.0
urn: "CoreOS:CoreOS:Stable:2135.6.0"
```
22 changes: 22 additions & 0 deletions hack/api-reference/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,17 @@ string
logical names and versions to provider-specific identifiers.</p>
</td>
</tr>
<tr>
<td>
<code>acceleratedNetworkingMachineTypes</code></br>
<em>
[]string
</em>
</td>
<td>
<p>AcceleratedNetworkingMachineTypes is a list of machine types which support accelerated networking.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="azure.provider.extensions.gardener.cloud/v1alpha1.ControlPlaneConfig">ControlPlaneConfig
Expand Down Expand Up @@ -726,6 +737,17 @@ string
<p>ID is the VM image ID</p>
</td>
</tr>
<tr>
<td>
<code>acceleratedNetworking</code></br>
<em>
bool
</em>
</td>
<td>
<p>AcceleratedNetworing is an indicator if the image does support Azure accelerated networking.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="azure.provider.extensions.gardener.cloud/v1alpha1.MachineImages">MachineImages
Expand Down
4 changes: 4 additions & 0 deletions pkg/apis/azure/types_cloudprofile.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ type CloudProfileConfig struct {
// MachineImages is the list of machine images that are understood by the controller. It maps
// logical names and versions to provider-specific identifiers.
MachineImages []MachineImages
// AcceleratedNetworkingMachineTypes is a list of machine types which support accelerated networking.
AcceleratedNetworkingMachineTypes []string
}

// DomainCount defines the region and the count for this domain count value.
Expand All @@ -57,4 +59,6 @@ type MachineImageVersion struct {
URN *string
// ID is the image id.
ID *string
// AcceleratedNetworing is an indicator if the image does support Azure accelerated networking.
AcceleratedNetworking *bool
}
4 changes: 4 additions & 0 deletions pkg/apis/azure/v1alpha1/types_cloudprofile.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ type CloudProfileConfig struct {
// MachineImages is the list of machine images that are understood by the controller. It maps
// logical names and versions to provider-specific identifiers.
MachineImages []MachineImages `json:"machineImages"`
// AcceleratedNetworkingMachineTypes is a list of machine types which support accelerated networking.
AcceleratedNetworkingMachineTypes []string `json:"acceleratedNetworkingMachineTypes,omitempty"`
}

// DomainCount defines the region and the count for this domain count value.
Expand Down Expand Up @@ -60,4 +62,6 @@ type MachineImageVersion struct {
//ID is the VM image ID
// +optional
ID *string `json:"id,omitempty"`
// AcceleratedNetworing is an indicator if the image does support Azure accelerated networking.
AcceleratedNetworking *bool `json:"acceleratedNetworking,omitempty"`
}
4 changes: 4 additions & 0 deletions pkg/apis/azure/v1alpha1/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions pkg/apis/azure/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions pkg/apis/azure/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 38 additions & 4 deletions pkg/controller/worker/machines.go
Original file line number Diff line number Diff line change
Expand Up @@ -174,12 +174,13 @@ func (w *workerDelegate) generateMachineConfig(ctx context.Context) error {
Annotations: pool.Annotations,
Taints: pool.Taints,
}

networkConfig = map[string]interface{}{
"vnet": infrastructureStatus.Networks.VNet.Name,
"subnet": nodesSubnet.Name,
}
machineClassSpec = map[string]interface{}{
"region": w.worker.Spec.Region,
"resourceGroup": infrastructureStatus.ResourceGroup.Name,
"vnetName": infrastructureStatus.Networks.VNet.Name,
"subnetName": nodesSubnet.Name,
"tags": map[string]interface{}{
"Name": w.worker.Namespace,
fmt.Sprintf("kubernetes.io-cluster-%s", w.worker.Namespace): "1",
Expand All @@ -198,12 +199,16 @@ func (w *workerDelegate) generateMachineConfig(ctx context.Context) error {
machineClassSpec["vnetResourceGroup"] = *infrastructureStatus.Networks.VNet.ResourceGroup
}

if w.enableAcceleratedNetworking(pool.MachineType, pool.MachineImage.Name, pool.MachineImage.Version) {
networkConfig["acceleratedNetworking"] = true
}
machineClassSpec["network"] = networkConfig

if zone != nil {
machineDeployment.Minimum = worker.DistributeOverZones(zone.index, pool.Minimum, zone.count)
machineDeployment.Maximum = worker.DistributeOverZones(zone.index, pool.Maximum, zone.count)
machineDeployment.MaxSurge = worker.DistributePositiveIntOrPercent(zone.index, pool.MaxSurge, zone.count, pool.Maximum)
machineDeployment.MaxUnavailable = worker.DistributePositiveIntOrPercent(zone.index, pool.MaxUnavailable, zone.count, pool.Minimum)

machineClassSpec["zone"] = zone.name
}
if availabilitySetID != nil {
Expand Down Expand Up @@ -268,3 +273,32 @@ func (w *workerDelegate) generateMachineConfig(ctx context.Context) error {

return nil
}

func (w *workerDelegate) enableAcceleratedNetworking(machineType, machineImageName, machineImageVersion string) bool {
// Check if used operating system and version are supporting accelerated networking.
var osSupport bool
for _, image := range w.cloudProfileConfig.MachineImages {
if image.Name != machineImageName {
continue
}
for _, imageVersion := range image.Versions {
if imageVersion.Version == machineImageVersion {
if imageVersion.AcceleratedNetworking != nil && *imageVersion.AcceleratedNetworking {
osSupport = true
} else {
return false
}
}
}
}

// Check if machine type is in list of accelerated networking supporting machine types.
var machineTypeSupport bool
for _, machType := range w.cloudProfileConfig.AcceleratedNetworkingMachineTypes {
if machType == machineType {
machineTypeSupport = true
break
}
}
return osSupport && machineTypeSupport
}
Loading

0 comments on commit 4b49aa1

Please sign in to comment.