Feature Request: test command

[ghost]

I'm asking for a test command which lets us run some tests against a built image based on some specification.

USE-CASE:
I build an AMI or a Google Cloud machine image, and I want to make sure they're behaving as expected. So, I need to run some tests against my build artifacts.

I wish packer could support a test command that I can tell please verify this image with this test.

A test could be a simple shell script or a sets of shell scripts. Packer spins up a new instance/vm with the built image, waits until it becomes ready, then ssh to it and run the script/scripts and it should see no error. Very similar to the way that packer builds an image using provisioner scripts.

Another option would be specifying tests in a tabular way. For example, we can have a json file in which we have an array of inputs and expected outputs. So, I can say I want to run this command and I'm expecting these strings/regexes to be present in the output.

[mwhooker]

I think this is a great way to test your images, but my feeling is that it falls a little out of scope of packer.

What about running a shell provisioner which runs your tests? If the tests fails, packer would see the build as failing?

[ghost]

Well, what I have right now is a little terraform and a shell script which run my tests. But, because what I'm doing is very similar to what packer does for creating a machine image (spinning up a new instance, wait until it becomes ready, and run a script against it), I thought why not having this feature integrated with Packer.

[mwhooker]

I think you should put your tests in the provisioner section. Ideally, we don't even want to create an artifact if it's broken, and the best way to do that is to error during provisioning.

If you just want to test that the image was created correctly, then I don't think packer is the best place for that. Packer is all about creating images, which is already a large domain, so we want avoid integrating anything beyond that into the packer core.

It might be a good idea to shoot an email to the mailing list to brainstorm ideas. Thank you for suggesting this, but I don't think this is something we're going to work on.