Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VMCX file format #6102

Closed
Novakov opened this issue Aug 6, 2015 · 24 comments
Closed

VMCX file format #6102

Novakov opened this issue Aug 6, 2015 · 24 comments

Comments

@Novakov
Copy link

Novakov commented Aug 6, 2015

I'm not sure where I should create issue regarding Hyper-V provider, so this is the same issue as https://github.com/MSOpenTech/vagrant-windows-hyperv/issues/73

After installing Windows 10 I wanted to play with vagrant and hyper-v. After creating a box, doing vagrant up failed with error:

C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/config.rb:92:in `block in missing_interpolation_argument_handler': missing interpolation argument :name in "The box you're using with the Hyper-V provider ('%{name}')\nis invalid. A Hyper-V box should contain both a\n\"Virtual Machines\" and a \"Virtual Hard Disks\" folder that are\ncreated as part of exporting a Hyper-V machine.\n\nWithin these directories, Vagrant expects to find the\nvirtual machine configuration as well as the root hard disk.\n\nThe box you're attempting to use is missing one or both of\nthese directories or does not contain the files expected. Verify\nthat you added the correct box. If this problem persists,\nplease contact the creator of the box for assistance." ({:_key=>:box_invalid, :_namespace=>"vagrant_hyperv.errors"} given) (I18n::MissingInterpolationArgument)
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:29:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:29:in `block in interpolate_hash'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:21:in `gsub'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:21:in `interpolate_hash'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:17:in `interpolate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/backend/base.rb:152:in `interpolate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/backend/base.rb:41:in `translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:158:in `block in translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:154:in `catch'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:154:in `translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/errors.rb:103:in `translate_error'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/errors.rb:72:in `initialize'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/plugins/providers/hyperv/action/import.rb:50:in `exception'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/plugins/providers/hyperv/action/import.rb:50:in `raise'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/plugins/providers/hyperv/action/import.rb:50:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builder.rb:116:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in `block in run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/util/busy.rb:19:in `busy'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in `run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/call.rb:53:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/plugins/providers/hyperv/action/check_enabled.rb:18:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/builder.rb:116:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in `block in run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/util/busy.rb:19:in `busy'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/action/runner.rb:66:in `run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:214:in `action_raw'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:191:in `block in action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/environment.rb:516:in `lock'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:178:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/machine.rb:178:in `action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.7.4/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/logging-1.8.2/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'

In the box there is configuration file but not in vmc format but new VMCX format which seems to be unsupported.

Does it mean that it is no longer possible/supported to create Vagrant box from Hyper-V machine?

@sethvargo
Copy link
Contributor

I don't think Vagrant currently works on Windows 10, but this looks like a separate issue. Thanks for reporting!

@Novakov
Copy link
Author

Novakov commented Aug 6, 2015

@sethvargo Before creating box myself, I successfully ran https://atlas.hashicorp.com/serveit/boxes/centos-7, so using boxes based on previous hyper-v version seems to work

@sethvargo sethvargo added the core label Aug 6, 2015
@Novakov
Copy link
Author

Novakov commented Aug 8, 2015

I've created simple PowerShell script that generates XML file that can be understood by vagrant.

Source: https://gist.github.com/Novakov/c640d65f1bf4cc7d7b7a
Usage: .\generate_vm_xml_config.ps1 -VMName <name of VM on local Hyper-V> -OutputPath <path to output XML>

@Fudoshiki
Copy link

#6272

@sopwithpup
Copy link

@Novakov I've run your script and many thanks but where does it go in the box file? Currently I have a the VHD and VM directories and a js manifest file. Tried replacing the js but doesn't work.

Do I just put "boxname".xml in there too?

@Novakov
Copy link
Author

Novakov commented Sep 29, 2015

@sopwithpup It should be in VIrtual Machines folder as box.xml. I've also removed vmcx & vmrs files

@mitchellh
Copy link
Contributor

This is fixed!

@johnpbloch
Copy link

I don't understand how the issue is fixed. The code still requires an XML configuration file when importing a new box. The problem is that Hyper V on Windows 10 does not create such a file, instead they've moved to a (maybe proprietary?) binary VMCX format for the configuration files exported in the Virtual Machines directory.

@mitchellh
Copy link
Contributor

Oh sorry. I thought this had to do with generation 2 VMs. Thanks for pointing out my mistake.

We'll require someone with Hyper-V expertise to come help us fix this, as we don't currently have a maintainer for it.

@mitchellh mitchellh reopened this Nov 19, 2015
@johnpbloch
Copy link

For knowledge sharing purposes, I just found this blog post on reading VMCX files with PowerShell http://blogs.msdn.com/b/virtual_pc_guy/archive/2015/04/10/using-powershell-to-look-at-a-vm-configuration.aspx . I'll have to play with it later when I'm at my computer again.

@avishnyakov
Copy link

+1, same issue with Win10

@ForNeVeR
Copy link

Well, it seems to me like a pretty localized problem. We'll just need to update import_vm.ps1 script to support the new VMCX format (maybe using that blog post mentioned by @johnpbloch), or write a new script for VMCX. It doesn't look like a very hard task.

@evancox10
Copy link

evancox10 commented Jul 27, 2016

I would like to work on this to add support for .vmcx files. I agree that most of the work can be done in import_vm.ps1

Also, it looks import.rb tried to raise an error, but that this call itself failed and didn't pass the right arguments. So there is something going wrong with that too. That should be fixed as well, and may be a good first step to get meaningful output. Will check the Hyper-V errors class and see what may be going wrong.

Step 0: Fix the Raise Error call parameters (trivial)
Step 1: Fix the error checking in import.rb related to config path. It should be updated to look for either a .xml or a .vmcx. (trivial)
Step 2: Update import_vm.ps1 as needed for the new VMCX binary format. (most of the work)

Before starting any work on this, I would like to know from the Vagrant maintainers that I'm going in the right direction :) My thoughts are to leverage MSFT's native Hyper-V VM cmdlets to analyze the newer VMCX-based VMs. That way, we will be transparent to VM Configuration File version changes going forward. We could leave the old code that directly parses the XML in place, as there should be no changes from MSFT for XML-based configuration files. Only newer VM's with .VMCX/.VMRS files will change.

By using Compare-VM, and then taking the hypothetical VM output, as shown in the technet blog post, we should be able to operate on that object natively using all of the standard Get-VM* commands. So it would switch from a text-based approach of parsing XML file contents, to one that uses the native PS Hyper-V objects. Should be a win for future-compatibility, as the burden is now on MSFT to figure out how to parse any future VMs.

If this is the direction that the Vagrant team is OK with, I can give fixing this a shot. Are there any test-cases targeting this code path? Would be good to beef this up too, including error scenarios which don't seem to be covered (thus the problem with the error call).

Some other notes:
I believe the release I'm on was built using this version of import.rb, where line 54 is a call to RaiseError.

Here's the stack trace

C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/config.rb:92:in `block in missing_interpolation_argument_handler': missing interpolation argument :name in "The box you're using with the Hyper-V provider ('%{name}')\nis invalid. A Hyper-V box should contain both a\n\"Virtual Machines\" and a \"Virtual Hard Disks\" folder that are\ncreated as part of exporting a Hyper-V machine.\n\nWithin these directories, Vagrant expects to find the\nvirtual machine configuration as well as the root hard disk.\n\nThe box you're attempting to use is missing one or both of\nthese directories or does not contain the files expected. Verify\nthat you added the correct box. If this problem persists,\nplease contact the creator of the box for assistance." ({:_key=>:box_invalid, :_namespace=>"vagrant_hyperv.errors"} given) (I18n::MissingInterpolationArgument)
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:29:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:29:in `block in interpolate_hash'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:21:in `gsub'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:21:in `interpolate_hash'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/interpolate/ruby.rb:17:in `interpolate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/backend/base.rb:152:in `interpolate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n/backend/base.rb:41:in `translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:158:in `block in translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:154:in `catch'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/i18n-0.7.0/lib/i18n.rb:154:in `translate'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/errors.rb:103:in `translate_error'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/errors.rb:72:in `initialize'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/plugins/providers/hyperv/action/import.rb:54:in `exception'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/plugins/providers/hyperv/action/import.rb:54:in `raise'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/plugins/providers/hyperv/action/import.rb:54:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:95:in `block in finalize_action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/builder.rb:116:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/runner.rb:66:in `block in run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/util/busy.rb:19:in `busy'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/runner.rb:66:in `run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/builtin/call.rb:53:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/builtin/handle_box.rb:56:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/plugins/providers/hyperv/action/check_enabled.rb:18:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/warden.rb:34:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/builder.rb:116:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/runner.rb:66:in `block in run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/util/busy.rb:19:in `busy'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/action/runner.rb:66:in `run'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/machine.rb:225:in `action_raw'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/machine.rb:200:in `block in action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/environment.rb:561:in `lock'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/machine.rb:186:in `call'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/machine.rb:186:in `action'
        from C:/HashiCorp/Vagrant/embedded/gems/gems/vagrant-1.8.4/lib/vagrant/batch_action.rb:82:in `block (2 levels) in run'
PS C:\Users\evan.cox\vagrant_gs\temp>

@bbrala
Copy link
Contributor

bbrala commented Sep 26, 2016

Sorry for highjacking this @evancox10, had some time on my hand, and seemed like a fun thing to do :)

Hmm, i was just looking at this and i'm slightly confused. From what i understand from the script, it reads the format and if some options are set, use the ones configured by the user. Why not just load the file, and just overwrite the few values someone might set on their Vagrantfile. It seems that would result in a clean copy of the configuration, but then as a new VM.

I feel that could make the import command quite a lot simpler. Also it would enable to show incompatibilities, should those be in the machine.

I've started to have a look at importing the VM this way, and seems to work on my Windows 10. Just need to rename the vhd's so they wont collide in name. Currently they keep the old name.

Fun little exercise.

@bbrala
Copy link
Contributor

bbrala commented Oct 2, 2016

@chrisroberts Milestone 2.0? This seems like it really isn't that big an feature, any rationale on why you added the tag?

@chrisroberts
Copy link
Member

Hi @bbrala! The milestone assignment isn't immutable. There are hundreds of issues in the back log that I am getting sorted and queued. Their placement will change over time based on discussions/importance/pull requests/etc. Based on everything else, that's just the bucket this issue is in right now. Cheers! 😃

@chrisroberts chrisroberts modified the milestones: 1.8, 2.0 Oct 3, 2016
@bbrala
Copy link
Contributor

bbrala commented Oct 3, 2016

Ok cool, thinking or will help a lot of people if this gets in. Also combined with the (hopefully) soon to come hyper v integration in packer.

Next thing I might do is make the package command work for hyper v.

@bbrala
Copy link
Contributor

bbrala commented Oct 7, 2016

@evancox10 I ended up not using Compare-VM, I had issues with adjusting certain parts of hardware, and vagrant seems to copy the vhd(x) itself, which made it a bit akward. The PR is ready to go. So now we are just waiting for it to go through process of getting merged.

@evancox10
Copy link

@bbrala Awesome, no problems by me that someone else stepped up. Anything I can do to help test it out? I'll try to checkout the code this weekend and give it a spin.

@bbrala
Copy link
Contributor

bbrala commented Oct 13, 2016

Yeah sure, checking the two PR's (Import and Package of Hyper V) would help loads.

Import VMCX: #7854
Package Hyper V: #7867

Made a gist with instructions with getting up and running on Windows (https://gist.github.com/bbrala/7e96be8a4e26f14834e8bbc21d246c38), but you might already have a running enviroment. :)

@bbrala
Copy link
Contributor

bbrala commented Oct 19, 2016

@evancox10 did you have a go at the two PR's?

@brianretford
Copy link

meant to comment here - I tested these and they both work after my patches, what needs to happen to get these pulled?

@chrisroberts
Copy link
Member

Fixed via: #7854

@ghost
Copy link

ghost commented Apr 3, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Apr 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests