Feature request - Managed Disk Encryption for Virtual Machine

[imcdnzl]

There seem to be other issues around encrypted disk but are not for managed disks that I can find (please close if you consider otherwise or have missed one!!)

A reference for this is how the az cli does it https://docs.microsoft.com/en-us/cli/azure/vm/encryption?view=azure-cli-latest#az_vm_encryption_enable

To implement this I think the machine with disk would have to be created first and then this applied shortly afterwards as it seems to be something you have to apply after the disk has been created (haven't verified personally yet) :-(

[tombuildsstuff]

Upstream issue: Azure/azure-sdk-for-go#2322

[OffColour]

To add to this, this is working for me, at least on Windows machines.

resource "azurerm_virtual_machine_extension" "disk-encryption" {
  name                 = "DiskEncryption"
  location             = "${local.location}"
 resource_group_name = "${azurerm_resource_group.environment-rg.name}"
  virtual_machine_name = "${azurerm_virtual_machine.server.name}"
  publisher            = "Microsoft.Azure.Security"
  type                 = "AzureDiskEncryption"
  type_handler_version = "2.2"

  settings = <<SETTINGS
{
  "EncryptionOperation": "EnableEncryption",
  "KeyVaultURL": "https://${local.vaultname}.vault.azure.net",
  "KeyVaultResourceId": "/subscriptions/${local.subscriptionid}/resourceGroups/${local.vaultresourcegroup}/providers/Microsoft.KeyVault/vaults/${local.vaultname}",
  "KeyEncryptionKeyURL": "https://${local.vaultname}.vault.azure.net/keys/${local.keyname}/${local.keyversion}",
  "KekVaultResourceId": "/subscriptions/${local.subscriptionid}/resourceGroups/${local.vaultresourcegroup}/providers/Microsoft.KeyVault/vaults/${local.vaultname}",
  "KeyEncryptionAlgorithm": "RSA-OAEP",
  "VolumeType": "All"
}
SETTINGS
}

Other thing worth mentioning:
The encryption extension will create an E: drive partition which will become disk 2, so if you've got any subsequent mounting and drive letter assigning steps for a data disk in something like DSC, make sure that you're referencing Disk 3 and at least F: otherwise it'll error.

I've not tried applying it to an existing VM that already has a E: drive. It might just take the next available letter or it might crash.

[whytoe]

@OffColour Did this work for Data and OS Disks because we are experiencing an issue where only the OS disk encrypts using the above extension example.

[OffColour]

@whytoe Yes, it's working for both, however I have seen that occasionally the portal incorrectly reports that the data disk isn't encrypted. This is also true of using Get-AzureRmVMDiskEncryptionStatus. MS have confirmed this is a bug.

However if you go on to the VM and run manage-bde -status driveletter: it reports that it's encrypted.

[mathieu-benoit]

@tombuildsstuff, just curious, is it now supported (i.e. managed disk encryption for VM)? If yes, it could be great to update that example still leveraging an ARM Template?
https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/examples/virtual-machines/encrypt-running-linux-vm/main.tf#L71
Furthermore, what's the official approach to follow to achieve that or what are the differences with this implementation: #399?
Thanks for the clarification!

[ithahn]

Would be great to have another update as this is important to have while using terraform for machine deployment in Azure.

[ghost]

Why is there encryption support documented within the managed_disk resource if its not currently possible?

Maybe I'm missing something here but when configuring the encryption settings terraform returns the following error for managed data disks on apply:

Error: Error waiting for Virtual Machine "<vm>" (Resource Group "<rsg>") to finish updating Disk "<disk>": 
Code="NotSupported" Message="Disk '<disk-id>' contains encryption settings and cannot be used as a data disk. 
In order to use it as a data disk, remove the encryption settings and ensure that the virtual machine OS disk has the applicable encryption settings defined."

[sunnynazar]

I totally agree with @clarkcode .

Encryption of managed data disk using "azurerm_managed_disk" encryption settings does n't work, it should be either disabled or if its an issue with Azure SDK should be explained.

@tombuildsstuff - do you have anymore insights on this please?

[justcallmetrell]

Have been attempting to deploy a VM w/ Manged OS and Data disk following the Terraform Documentation, however, persistent issues as others in the thread are mentioning. For me, I receive

Error: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=400 -- Original Error: Code="InvalidParameter" Message="Parameter 'osProfile' is not allowed." Target="osProfile"

I remove the osProfile block and can have successful deployment, however, its not accessible because the VM doesn't have a computer_name, username, and password.

Is this issue being worked on and/or documentation being updated actively? Any insight or update would be greatly appreciated.

[ymash13]

I am seeing the same issue where the OS disk gets encrypted but not the Data disk. After initializing the data disk in windows, the disk automatically got encrypted after some time, maybe an hour or so.
@OffColour - I saw the same thing. Portal says it's not encrypted, but you see that it is bitlocker encrypted in disk management MMC.

[rapster83]

I have encrypted my OS and data disk using the following:

resource "azurerm_virtual_machine_extension" "vm_encry_win" {
  count                         = "${var.vm_encry_os_type == "Windows" ? 1 : 0}"
  name                          = "${var.vm_encry_name}"
  location                      = "${var.vm_encry_location}"
  resource_group_name           = "${var.vm_encry_rg_name}"
  virtual_machine_name          = "${var.vm_encry_vm_name}"
  publisher                     = "${var.vm_encry_publisher}"
  type                          = "${var.vm_encry_type}"
  type_handler_version          = "${var.vm_encry_type_handler_version == "" ? "2.2" : var.vm_encry_type_handler_version}"
  auto_upgrade_minor_version    = "${var.vm_encry_auto_upgrade_minor_version}"
  tags                          = "${var.vm_encry_tags}"

  settings = <<SETTINGS
                {
                    "EncryptionOperation":      "${var.vm_encry_operation}",
                    "KeyVaultURL":              "${var.vm_encry_kv_vault_uri}",
                    "KeyVaultResourceId":       "${var.vm_encry_kv_vault_id}",
                    "KeyEncryptionKeyURL":      "${var.vm_encry_kv_key_url}",
                    "KekVaultResourceId":       "${var.vm_encry_kv_vault_id}",
                    "KeyEncryptionAlgorithm":   "${var.vm_encry_key_algorithm}",
                    "VolumeType":               "${var.vm_encry_volume_type}"
                }
             SETTINGS
}

And it worked so far. Both disks are encrypted now. Unfortunately, I have the problem, when I re-run terraform plan or terraform apply, because the azurerm_managed_disk wants to replace my data disk now as the follwoing screenshot illustrates.

resource "azurerm_managed_disk" "md" {
    count                   = "${var.md_count}"
    name                    = "${var.md_name}-disk${count.index}"
    location                = "${var.md_location}"
    resource_group_name     = "${var.md_rg_name}"
    storage_account_type    = "${var.md_storage_account_type}"
    create_option           = "${var.md_create_option}"
    source_uri              = "${var.md_create_option == "Import" ? var.md_source_uri : null}"
    source_resource_id      = "${var.md_create_option == "Copy" || var.md_create_option == "Restore" ? var.md_source_resource_id : null}"
    image_reference_id      = "${var.md_create_option == "FromImage" ? var.md_image_reference_id : null}"
    os_type                 = "${var.md_create_option == "Copy" || var.md_create_option == "Import" ? var.md_os_type : null}"
    disk_size_gb            = "${var.md_disk_size_gb != "" ? var.md_disk_size_gb : null}"
    tags                    = "${merge(local.default_tags, var.md_tags)}"
}

Seems like the encryption_settings on azurerm_managed_disk is not working correctly.
Does anyone can help me with this issue? Is there a work around?

[jeff1985]

Hi @stefan-rapp ,

if disk creation works fine, you probably could just tell terraform to ignore the encryption_settings:

  lifecycle {
    ignore_changes = [ encryption_settings ]
  }

Do you want to give it a try?

[rapster83]

Hi @jeff1985 ,
I tried that already:

lifecycle {
        ignore_changes = [  encryption_settings[0].enabled,
                            encryption_settings[0].key_encryption_key[0].key_url,
                            encryption_settings[0].key_encryption_key[0].source_vault_id,
                            encryption_settings[0].disk_encryption_key[0].secret_url,
                            encryption_settings[0].disk_encryption_key[0].source_vault_id]
    } 

But the thing is, that for initial creation of managed disks, I always need encryption settings now.

Normally, we need a fix, that if Microsoft.Azure.Security with AzureDiskEncryption is used as VM Extension to encrypt disks, the parameters of encryption_settings should NOT do a replacement of azurerm_managed_disk, especially if the encryption_settings block is not specified.

[rapster83]

@jeff1985 ,
sorry for the inconvinience, I tried out your option and it worked.

lifecycle {
        ignore_changes = [encryption_settings]
    }

Thx for your quick help. Appreciate it!

[tombuildsstuff]

👋

We're currently working on version 2.0 of the Azure Provider which we previously announced in #2807.

As a part of this we're introducing five new resources which will supersede the existing azurerm_virtual_machine and azurerm_virtual_machine_scale_set resources:

• azurerm_linux_virtual_machine
• azurerm_linux_virtual_machine_scale_set
• azurerm_virtual_machine_scale_set_extension
• azurerm_windows_virtual_machine
• azurerm_windows_virtual_machine_scale_set

We recently opened #5550 which adds support for the new Virtual Machine resources - and I'm able to confirm that encrypting the OS Disk is supported in the new Virtual Machine resources - however unfortunately we have no plans to backport this to the existing azurerm_virtual_machine resource.

In the new resources you can encrypt the OS Disk using a Disk Encryption Set like so:

resource "azurerm_resource_group" "example" {}
resource "azurerm_network_interface" "example" {}
resource "azurerm_key_vault" "example" {}
resource "azurerm_key_vault_key" "example" {}

resource "azurerm_disk_encryption_set" "example" {
  name                = "example-set"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
  key_vault_key_id    = azurerm_key_vault_key.example.id
  
  identity {
    type = "SystemAssigned"
  }
}
resource "azurerm_key_vault_access_policy" "disk-encryption" {
  key_vault_id = azurerm_key_vault.example.id
  tenant_id = azurerm_disk_encryption_set.example.identity.0.tenant_id
  object_id = azurerm_disk_encryption_set.example.identity.0.principal_id
  key_permissions = [
    "get",
    "wrapkey",
    "unwrapkey",
  ]
}

resource "azurerm_role_assignment" "disk-encryption-read-keyvault" {
  scope                = azurerm_key_vault.test.id
  role_definition_name = "Reader"
  principal_id         = azurerm_disk_encryption_set.example.identity.0.principal_id
}

resource "azurerm_linux_virtual_machine" "example" {
  name                = "example-vm"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
  size                = "Standard_F2"
  admin_username      = "adminuser"
  network_interface_ids = [
    azurerm_network_interface.example.id,
  ]
  
  admin_ssh_key {
    username   = "adminuser"
    public_key = file("~/.ssh/id_rsa.pub")
  }
  os_disk {
    caching                = "ReadWrite"
    disk_encryption_set_id = azurerm_disk_encryption_set.example.id
    storage_account_type   = "Standard_LRS"
  }
  source_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }
  depends_on = [
    "azurerm_role_assignment.disk-encryption-read-keyvault",
    "azurerm_key_vault_access_policy.disk-encryption",
  ]
}

Note: Soft Delete is not natively supported on the azurerm_key_vault resource prior to 2.0 - as such this needs to be configured out-of-band - however this will be supported in version 2.0 of the Azure Provider.

In order to get feedback on these new resources we'll be launching support for these new resources as an opt-in Beta in an upcoming 1.x release of the Azure Provider and ultimately release these as "GA" in the upcoming 2.0 release. We'll post an update in #2807 when both the opt-in Beta (1.x) & GA (2.0) are available - as such I'd recommend subscribing to that issue for updates.

This issue's been assigned to the milestone "2.0" since this is where this will ship - however (due to the way that closing Github Issues from PR's works, to be able to track this back for future users) this issue will be closed once the first of the new resources have been merged.

Thanks!

[Dmitry1987]

I just tried the above snippet using my resource group (used the 'data' to import the vault/subnet/key required), it works but the disk shows as 'not encrypted' in Azure UI. How do I know if it actually worked? 😅
It's Ubuntu, and I tried the cryptsetup status xxxxx but no results, the disks don't seem encrypted at the linux level (I know they are by default encrypted at rest, in Azure under the hood).
Terraform run was successful, and the "disk encryption set" exists, and it also has that OS disk. But the VM tab says encryption not enabled, is there a way to check what's the real status of encryption?

[dsedivec]

For the benefit of anyone who stumbles upon this in the future, I'd like to note that this issue's title refers to "managed disk encryption", which I think refers to Azure Server-Side Encryption¹ (SSE). tombuildsstuff's example a couple comments up from this one is indeed, as far as I know, setting up SSE.

However, a couple of the comments in this issue are setting up Azure Disk Encryption (ADE), which uses a VM extension and sets up BitLocker (Windows) or dm-crypt (Linux) via the OS.

SSE and ADE are not the same thing at all.

The upstream issue tombuildsstuff referred to early on also seems to be about ADE, not SSE.

As of this writing, the encrypt-running-linux-vm example also seems to be setting up ADE, though it uses azurerm_template_deployment which is kind of cheating (and I'm betting this can all be done today entirely with resources supplied by this provider).

I propose we blame Microsoft for this confusion.

Finally, I suspect pre-2.0 versions of the azurerm provider were able to configure ADE, but not SSE. I've only just started using Terraform, so I'm starting with the 2.x provider, but from what I've read in this issue and elsewhere, I gather 2.0 added management of SSE. Something like that.

¹ In Microsoft docs, SSE sometimes has different expansions. I think I've seen "Server-Side Encryption" (with and without hyphen), "Server-side Storage Encryption", and "Storage Service Encryption".

[AlphaWong]

resource "azurerm_disk_encryption_set" "example" {
  name                = "example-set"
  resource_group_name = azurerm_resource_group.example.name
  location            = azurerm_resource_group.example.location
  key_vault_key_id    = azurerm_key_vault_key.example.id
  
  identity {
    type = "SystemAssigned"
  }
}

good to see os_disk is supported now. But how about the azurerm_managed_disk ?

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.