error reconfiguring virtual machine: Invalid operation for device '0'.

[n0405]

Community Guidelines

• I have read and agree to the HashiCorp Community Guidelines .
• Vote on this issue by adding a 👍 reaction to the original issue initial description to help the maintainers prioritize.
• Do not leave "+1" or other comments that do not add relevant information or questions.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform

v1.3.6

Terraform Provider

v2.2.0

VMware vSphere

7.0.3

Description

hi ,
I want to provision two virtual machines from template, template is created from alma linux ( cloud init image). But I am getting error : Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.

│   with vsphere_virtual_machine.vm["terraVM1"],
│   on Create_VMs.tf line 46, in resource "vsphere_virtual_machine" "vm":
│   46: resource "vsphere_virtual_machine" "vm" {

When I try to manualy create VM through GUI in vcenter , there is no problem .

Affected Resources or Data Sources

disk {
    label            = local.disk_label
    size             = local.disk_size
    thin_provisioned = local.disk_thin
  }

disk_label          = "disk0"
    disk_size           = "16"
    disk_thin           = "true"

Terraform Configuration

main.tf

provider "vsphere" {
  user                 = var.user
  password             = var.pass
  vsphere_server       = var.vcenter
  allow_unverified_ssl = true

  }

data "vsphere_datacenter" "dc" {
  name = local.dc
}

data "vsphere_compute_cluster" "compute_cluster" {
  name          = local.cluster
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_resource_pool" "pool" {
  name          = local.resource_pool_id
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_datastore" "datastore" {
  name          = local.datastore
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_network" "net" {
  name          = local.vnic1
  datacenter_id = data.vsphere_datacenter.dc.id
}

data "vsphere_virtual_machine" "template" {
  name          = local.template
  datacenter_id = data.vsphere_datacenter.dc.id
}

resource "vsphere_virtual_machine" "vm" {
  for_each = var.vms
  datastore_id     = data.vsphere_datastore.datastore.id
  resource_pool_id = data.vsphere_resource_pool.pool.id
  guest_id         = data.vsphere_virtual_machine.template.guest_id

  network_interface {
    network_id = data.vsphere_network.net.id
  }

  name = each.value.name

  num_cpus = local.cpu_count
  memory   = local.memory

  disk {
    label            = local.disk_label
    size             = local.disk_size
    thin_provisioned = local.disk_thin
  }

  clone {
    template_uuid = data.vsphere_virtual_machine.template.id
  
           # this part is irrelevant , because the same error occurs in both scenarios, without and with it
          customize {
               linux_options {
                       host_name = each.value.name
                      domain    = local.domain
                                   }

            network_interface {
                               ipv4_address    = each.value.ipv4_add_mgt
                              ipv4_netmask    = local.ipv4_net
                             dns_server_list = local.dns
                                 }

           ipv4_gateway = local.ipv4_gate
                }
  }
}

variables.tf

locals {
    dc                  = "DC"
    cluster             = "CL"
    resource_pool_id    = "terra"
    datastore           = "Datastore1"
    vnic1               = "vl888"
    template            = "alma"
    cpu_count           = "2"
    memory              = "2048"
    disk_label          = "disk0"
    disk_size           = "16"
    disk_thin           = "true"
    domain              = "me.local"
    ipv4_net            = "24"
    dns                 = ["8.8.8.8", "8.8.4.4"]
    ipv4_gate           = "192.168.1.1"
 
  }

variable "vms" {
    type = map
    default = {
      terraVM1= {
        name         = "terraVM1"
        ipv4_add_mgt = "192.168.1.101"
        },
      terraVM2 = {
        name         = "terraVM2"
        ipv4_add_mgt = "192.168.1.102"
         },
    }
  }

Debug Output

Panic Output

No response

Expected Behavior

to create two virtal mashines

Actual Behavior

error

Steps to Reproduce

terrafrom plan :

Terraform will perform the following actions:

   vsphere_virtual_machine.vm["terraVM1"] will be created
     + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-16"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 2048
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "terraVM1"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 2
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-19007"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = (known after apply)
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "422d0eb1-9698-d648-70b0-2c2d1bac1140"
          + timeout       = 30
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 16
          + storage_policy_id = (known after apply)
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-19008"
        }
    }

   vsphere_virtual_machine.vm["terraVM2"] will be created
     + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-16"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 2048
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "terraVM2"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 2
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-19007"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = (known after apply)
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "422d0eb1-9698-d648-70b0-2c2d1bac1140"
          + timeout       = 30
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 16
          + storage_policy_id = (known after apply)
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-19008"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

Terraform apply

vsphere_virtual_machine.vm["terraVM2"]: Creating...
vsphere_virtual_machine.vm["terraVM1"]: Creating...
╷
│ Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.
│
│   with vsphere_virtual_machine.vm["terraVM2"],
│   on Create_VMs.tf line 46, in resource "vsphere_virtual_machine" "vm":
│   46: resource "vsphere_virtual_machine" "vm" {
│
╵
╷
│ Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.
│
│   with vsphere_virtual_machine.vm["terraVM1"],
│   on Create_VMs.tf line 46, in resource "vsphere_virtual_machine" "vm":
│   46: resource "vsphere_virtual_machine" "vm" {

Environment Details

No response

Screenshots

No response

References

No response

[tenthirtyam]

May we ask that you please format HCL using the GotHub markdown editor for readability?

[tenthirtyam]

As a test, try changing the disk stanza to something like this and see if the results are the same.

  disk {
    label            = "disk0"
    size             = data.vsphere_virtual_machine.template.disks[0].size
    eagerly_scrub    = data.vsphere_virtual_machine.template.disks[0].eagerly_scrub
    thin_provisioned = data.vsphere_virtual_machine.template.disks[0].thin_provisioned
  }

[n0405]

Hi , this goes through without error . VMs created as same disk size like in template.

[tenthirtyam]

Ok, this narrows the issue down to this stanza for further review.

[tenthirtyam]

When you ran the plan for the stanza change, what was the disk size returned from the datasource - your variable must be the same or greater.

[n0405]

variable is greater then disk size in template, I set to 16 .

disk_size           = "16"

...

disk {
    label            = local.disk_label
    size             = local.disk_size
    thin_provisioned = local.disk_thin
   }

[tenthirtyam]

When you ran the plan for the stanza change, what was the disk size returned from the datasource....

Can you provide the plan output to confirm?

Any change if you add eagerly_scrub?

[n0405]

this is plan output when I use my configuration ( Part regarding for first vm, it's the same for second) :

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm["terraVM1"] will be created
  + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-16"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 2048
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "terraVM1"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 2
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-19007"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = (known after apply)
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "422d42a5-8ede-0153-85f4-0da1608efb53"
          + timeout       = 30

          + customize {
              + ipv4_gateway = "192.168.71.1"
              + timeout      = 10

              + linux_options {
                  + domain       = "me.local"
                  + host_name    = "terraVM1"
                  + hw_clock_utc = true
                }

              + network_interface {
                  + dns_server_list = [
                      + "8.8.8.8",
                      + "8.8.4.4",
                    ]
                  + ipv4_address    = "192.168.1.101"
                  + ipv4_netmask    = 24
                }
            }
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 16
          + storage_policy_id = (known after apply)
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-19008"
        }
    }

This is plan output when I change disk stanza , like you wrote :

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm["terraVM1"] will be created
  + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-16"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 2048
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "terraVM1"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 2
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-19007"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = (known after apply)
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "422d42a5-8ede-0153-85f4-0da1608efb53"
          + timeout       = 30

          + customize {
              + ipv4_gateway = "192.168.1.1"
              + timeout      = 10

              + linux_options {
                  + domain       = "me.local"
                  + host_name    = "terraVM1"
                  + hw_clock_utc = true
                }

              + network_interface {
                  + dns_server_list = [
                      + "8.8.8.8",
                      + "8.8.4.4",
                    ]
                  + ipv4_address    = "192.168.1.102"
                  + ipv4_netmask    = 24
                }
            }
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = true
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 10
          + storage_policy_id = (known after apply)
          + thin_provisioned  = false
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-19008"
        }
    }

[tenthirtyam]

First, this does help to confirm that the source template’s size for disk0 is 10GB and that the resource is requesting 16GB, which should be fine. It does also show that the settings desired for thin_provisioned and eagerly_scrub are both the opposite of the template (which may be related to another issue.. tbd).

As a test, set your size, thin_provisioned and eagerly_scrub to see if you get the same results.

(Keep in mind that Boolean values do not need quotes in your locals.)

locals {
    …
    disk_label = "disk0"
    disk_size = "10"
    disk_thin = true
    disk_eagerly_scrub = true
}

disk {
    label = local.disk_label
    size = local.disk_size
    thin_provisioned = local.disk_thin
    eagerly_scrub = local.disk_eagerly_scrub
  }

Let’s use this test to troubleshoot further.

[n0405]

So i try , like this

locals {
...
disk_label = "disk0"
    disk_size = "10"
    disk_thin = true
    disk_eagerly_scrub = false
}

disk {
    label = local.disk_label
    size = local.disk_size
    thin_provisioned = local.disk_thin
    eagerly_scrub = local.disk_eagerly_scrub
  }

It's create vm, but won't boot VM , on console of VM I can see "Operating system not found...."

apply output

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm["terraVM1"] will be created
  + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-16"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 2048
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "terraVM1"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 2
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-19007"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = (known after apply)
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "422d42a5-8ede-0153-85f4-0da1608efb53"
          + timeout       = 30
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 10
          + storage_policy_id = (known after apply)
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-19008"
        }
    }

I edited template and set disk to thin .

Maybe I should create a new temple.

[tenthirtyam]

Regarding not booting - is the virtual machine template using EFI or EFI with secure boot? If so, keep in mind that firmware will default to bios if not defined. Since you're using rhel8_64Guest that is EFI by default if I recall correctly.

EFI

  firmware                = "efi"
  efi_secure_boot_enabled = false

EFI with Secure Boot

  firmware                = "efi"
  efi_secure_boot_enabled = true

You could use the Packer examples from my other project.

[n0405]

yes , that solvet the problem with booting . But still can't change disk size . I try with second template Ubuntu 22(jammy-server-cloudimg-amd64) same behavior .

I try to provision VM , no use of template .

resource "vsphere_virtual_machine" "vm" {
  name             = "foo"
  resource_pool_id = data.vsphere_compute_cluster.compute_cluster.resource_pool_id
  datastore_id     = data.vsphere_datastore.datastore.id
  num_cpus         = 1
  memory           = 1024
  guest_id         = "other3xLinux64Guest"
  network_interface {
    network_id = data.vsphere_network.net.id
  }
  disk {
    label = "disk0"
    size  = 20
  }
}

but , no luck ,
vsphere_virtual_machine.vm: Creating...
╷
│ Error: error powering on virtual machine: powerOn task for vm "/DC/vm/foo" failed: Unable to enumerate all disks.

[tenthirtyam]

Thanks for confirming this test. I’ll follow up when I have additional time to review.

[driftertest]

Hi, have the same problem, can you help or need to wait version 2.5.0?

Terraform
v1.5.2

Terraform Provider
v2.4.1

VMware vSphere
7.0.3.01400

Datasources

data "vsphere_datacenter" "BMDC" {
  name = "BMDC"
}

data "vsphere_compute_cluster" "cluster" {
  name          = "ecs-bmdc-vmcls01"
  datacenter_id = data.vsphere_datacenter.BMDC.id
}

data "vsphere_datastore" "datastore_001" {
  name          = var.datastore[0]
  datacenter_id = data.vsphere_datacenter.BMDC.id
}

data "vsphere_datastore" "datastore_002" {
  name          = var.datastore[1]
  datacenter_id = data.vsphere_datacenter.BMDC.id
}

data "vsphere_network" "devops_network" {
  name          = "Ecosystem|AP_Ecosystem_Stretched|EPG_ECS_DEVOPS-TOOLS_DC-VLAN_1126"
  datacenter_id = data.vsphere_datacenter.BMDC.id
}

data "vsphere_virtual_machine" "template" {
  name          = local.template
  datacenter_id = data.vsphere_datacenter.BMDC.id
}

data "vsphere_storage_policy" "vm" {
  name = "Test VM Storage Policy"
}

data "vsphere_storage_policy" "disk" {
  name = "Test VM Storage Policy"
}

Variables

locals {
  dc               = "BMDC"
  cluster          = "ecs-bmdc-vmcls01"
  resource_pool_id = "terra"
  datastore        = "Datastore1"
  vnic1            = "vl888"
  template         = "Redhat 8.8 Template"
  cpu_count        = "2"
  memory           = "2048"
  disk_label       = "disk0"
  disk_size        = "100"
  disk_thin        = "true"
  disk_eagerly_scrub = "false"
  domain           = "me.local"
  ipv4_net         = "24"
  dns              = ["8.8.8.8", "8.8.4.4"]
  ipv4_gate        = "10.120.0.1"

}

variable "machines" {
  type = map(any)
  default = {
    "VM1" = {
      "name"         = "k8s-master-11"
      "ipv4_add_mgt" = "10.120.0.160"
      "procs"        = 4
      "ram"          = 8192
      "gateway"      = "10.120.0.1"
    }

    "VM2" = {
      "name"         = "k8s-master-12"
      "ipv4_add_mgt" = "10.120.0.161"
      "procs"        = 4
      "ram"          = 8192
      "gateway"      = "10.120.0.1"
    }
  }
}

variable "datastore" {
  type = list(string)
  default = [
    "pt-bmdc-str-PCHFL2246A006-ES-vv-001",
    "pt-bmdc-str-PCHFL2246A006-ES-vv-002",
    "pt-bmdc-str-PCHFL2246A006-ES-vv-003",
    "pt-bmdc-str-PCHFL2246A006-ES-vv-004",
    "pt-bmdc-str-PCHFL2246A006-ES-vv-005",
    "pt-bmdc-str-PCHFL2246A006-ES-vv-006"
  ]
}

Main

terraform {
  required_providers {
    vsphere = {
      source  = "hashicorp/vsphere"
      version = "~> 2.4"
    }
  }
  required_version = ">= 1.2.0"
}

provider "vsphere" {
  user                 = USERNAME
  password             = PASSWORD
  vsphere_server       = IP_SERVER
  allow_unverified_ssl = true
  client_debug = true
}

resource "vsphere_virtual_machine" "vm" {
  for_each         = var.machines
  datastore_id     = data.vsphere_datastore.datastore_001.id
  resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
  guest_id         = data.vsphere_virtual_machine.template.guest_id

  network_interface {
    network_id = data.vsphere_network.devops_network.id
  }

  name = each.value.name

  num_cpus = each.value.procs
  memory   = each.value.ram

  storage_policy_id = data.vsphere_storage_policy.vm.id

  disk {
    label            = local.disk_label
    size             = local.disk_size
    thin_provisioned = local.disk_thin
    eagerly_scrub = local.disk_eagerly_scrub
    storage_policy_id = data.vsphere_storage_policy.disk.id    
  }

  clone {
    template_uuid = data.vsphere_virtual_machine.template.id

    customize {
      linux_options {
        host_name = each.value.name
        domain    = local.domain
      }
      network_interface {
        ipv4_address    = each.value.ipv4_add_mgt
        ipv4_netmask    = local.ipv4_net
        dns_server_list = local.dns
      }
      ipv4_gateway = local.ipv4_gate
    }
  }
}

When i run plan command, everything all right:

Terraform will perform the following actions:

  # vsphere_virtual_machine.vm["VM1"] will be created
  + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-1022"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + extra_config_reboot_required            = true
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 8192
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "k8s-master-11"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 4
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-9"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = "19ddb546-f1f3-4da7-8058-89af965b9aa3"
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "4203bff4-e6ad-32c3-c23a-b0bbbacfef4a"
          + timeout       = 30

          + customize {
              + ipv4_gateway = "10.120.0.1"
              + timeout      = 10

              + linux_options {
                  + domain       = "me.local"
                  + host_name    = "k8s-master-11"
                  + hw_clock_utc = true
                }

              + network_interface {
                  + dns_server_list = [
                      + "8.8.8.8",
                      + "8.8.4.4",
                    ]
                  + ipv4_address    = "10.120.0.160"
                  + ipv4_netmask    = 24
                }
            }
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 100
          + storage_policy_id = "19ddb546-f1f3-4da7-8058-89af965b9aa3"
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-2036"
        }
    }

  # vsphere_virtual_machine.vm["VM2"] will be created
  + resource "vsphere_virtual_machine" "vm" {
      + annotation                              = (known after apply)
      + boot_retry_delay                        = 10000
      + change_version                          = (known after apply)
      + cpu_limit                               = -1
      + cpu_share_count                         = (known after apply)
      + cpu_share_level                         = "normal"
      + datastore_id                            = "datastore-1022"
      + default_ip_address                      = (known after apply)
      + ept_rvi_mode                            = "automatic"
      + extra_config_reboot_required            = true
      + firmware                                = "bios"
      + force_power_off                         = true
      + guest_id                                = "rhel8_64Guest"
      + guest_ip_addresses                      = (known after apply)
      + hardware_version                        = (known after apply)
      + host_system_id                          = (known after apply)
      + hv_mode                                 = "hvAuto"
      + id                                      = (known after apply)
      + ide_controller_count                    = 2
      + imported                                = (known after apply)
      + latency_sensitivity                     = "normal"
      + memory                                  = 8192
      + memory_limit                            = -1
      + memory_share_count                      = (known after apply)
      + memory_share_level                      = "normal"
      + migrate_wait_timeout                    = 30
      + moid                                    = (known after apply)
      + name                                    = "k8s-master-12"
      + num_cores_per_socket                    = 1
      + num_cpus                                = 4
      + power_state                             = (known after apply)
      + poweron_timeout                         = 300
      + reboot_required                         = (known after apply)
      + resource_pool_id                        = "resgroup-9"
      + run_tools_scripts_after_power_on        = true
      + run_tools_scripts_after_resume          = true
      + run_tools_scripts_before_guest_shutdown = true
      + run_tools_scripts_before_guest_standby  = true
      + sata_controller_count                   = 0
      + scsi_bus_sharing                        = "noSharing"
      + scsi_controller_count                   = 1
      + scsi_type                               = "pvscsi"
      + shutdown_wait_timeout                   = 3
      + storage_policy_id                       = "19ddb546-f1f3-4da7-8058-89af965b9aa3"
      + swap_placement_policy                   = "inherit"
      + tools_upgrade_policy                    = "manual"
      + uuid                                    = (known after apply)
      + vapp_transport                          = (known after apply)
      + vmware_tools_status                     = (known after apply)
      + vmx_path                                = (known after apply)
      + wait_for_guest_ip_timeout               = 0
      + wait_for_guest_net_routable             = true
      + wait_for_guest_net_timeout              = 5

      + clone {
          + template_uuid = "4203bff4-e6ad-32c3-c23a-b0bbbacfef4a"
          + timeout       = 30

          + customize {
              + ipv4_gateway = "10.120.0.1"
              + timeout      = 10

              + linux_options {
                  + domain       = "me.local"
                  + host_name    = "k8s-master-12"
                  + hw_clock_utc = true
                }

              + network_interface {
                  + dns_server_list = [
                      + "8.8.8.8",
                      + "8.8.4.4",
                    ]
                  + ipv4_address    = "10.120.0.161"
                  + ipv4_netmask    = 24
                }
            }
        }

      + disk {
          + attach            = false
          + controller_type   = "scsi"
          + datastore_id      = "<computed>"
          + device_address    = (known after apply)
          + disk_mode         = "persistent"
          + disk_sharing      = "sharingNone"
          + eagerly_scrub     = false
          + io_limit          = -1
          + io_reservation    = 0
          + io_share_count    = 0
          + io_share_level    = "normal"
          + keep_on_remove    = false
          + key               = 0
          + label             = "disk0"
          + path              = (known after apply)
          + size              = 100
          + storage_policy_id = "19ddb546-f1f3-4da7-8058-89af965b9aa3"
          + thin_provisioned  = true
          + unit_number       = 0
          + uuid              = (known after apply)
          + write_through     = false
        }

      + network_interface {
          + adapter_type          = "vmxnet3"
          + bandwidth_limit       = -1
          + bandwidth_reservation = 0
          + bandwidth_share_count = (known after apply)
          + bandwidth_share_level = "normal"
          + device_address        = (known after apply)
          + key                   = (known after apply)
          + mac_address           = (known after apply)
          + network_id            = "dvportgroup-2036"
        }
    }

Plan: 2 to add, 0 to change, 0 to destroy.

But apply gives me Errors:

╷
│ Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.
│
│   with vsphere_virtual_machine.vm["VM1"],
│   on main.tf line 19, in resource "vsphere_virtual_machine" "vm":
│   19: resource "vsphere_virtual_machine" "vm" {
│
╵
╷
│ Error: error reconfiguring virtual machine: error reconfiguring virtual machine: Invalid operation for device '0'.
│
│   with vsphere_virtual_machine.vm["VM2"],
│   on main.tf line 19, in resource "vsphere_virtual_machine" "vm":
│   19: resource "vsphere_virtual_machine" "vm" {
│
╵

[appilon]

Addressed in #1996

[github-actions]

This functionality has been released in v2.4.2 of the Terraform Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.