add skip_register_ami option

[jeremy-asher]

This is a cleaner implementation of #3793 which should take into account feedback from @mwhooker. In summary, this allows the AWS builders to be used for additional purposes when the AMI is not the final product of the build. When the "skip_register_ami" option is set, this skips all the steps following provisioning and proceeds straight to instance termination.

Note: this PR doesn't yet modify the newer EBS Surrogate Builder. I wasn't clear on how it works or how it differs from using from_scratch option I added to the Chroot builder. I can add a similar bypass to the Surrogate builder if that makes sense. @jen20 could your clarify?

[mwhooker]

@jeremy-asher thanks very much for keeping up with this. Will target for next release

[jeremy-asher]

@mwhooker should I just add this for the EBS Surrogate Builder as well and add the docs?

[mwhooker]

hmm, not sure it makes sense to add to that builder. Will have to think if it makes sense to add this to all the builders or just 1 or 2.

[jeremy-asher]

Thinking about this a bit more, I don't really see any harm in adding this option to the surrogate builder, but I can't think of any obvious use case. If someone wanted to do something with a blank device, they could use the regular EBS builder with this. I still think it's potentially useful with all of the other three builders.

[mwhooker]

@jeremy-asher This is looking good. Would you please add documentation to

• website/source/docs/builders/amazon-chroot.html.md
• website/source/docs/builders/amazon-ebs.html.md
• website/source/docs/builders/amazon-instance.html.md

describing the new skip registration option and maybe giving a 1 sentence reason for why you might want to use this?

[mwhooker]

don't we want to stop the instance even if we're not creating an AMI?

[jeremy-asher]

This is taken care of by instance termination in the cleanup phase for StepRunSourceInstance:
https://github.com/jeremy-asher/packer/blob/59109d6786f58b18e89362cc7b999c2361469735/builder/amazon/common/step_run_source_instance.go#L347

https://github.com/jeremy-asher/packer/blob/59109d6786f58b18e89362cc7b999c2361469735/builder/amazon/ebs/builder.go#L138

[mwhooker]

👍

[jeremy-asher]

Okay, I'll take care of the documentation.

[jeremy-asher]

Rebasing this real quick. @mwhooker is the desired style now to indent the second and subsequent lines of a multi line list item block an extra two spaces? I see some inconsistencies since rebasing the docs.

[mwhooker]

hey @jeremy-asher, would you please email me at [email protected] when you get a chance? I want to talk about this PR a little bit

[jeremy-asher]

Hey @mwhooker. Pinged you on mail. Do you need anything else on this?

[mwhooker]

Hey @jeremy-asher, sorry, I know I owe you an email, but I really haven't had time to give it the proper amount of thought. I've been going back and forth on your PR with our product manager, and I want to make sure I understand your use-case well enough. The thing that's giving us pause is that the PR, in some ways, radically alters the behavior of packer. We try to keep it as focused on possible on creating artifacts. You mentioned this being useful with the artifice post-processor. Can you give us a little more information about what kinds of artifacts you're producing? Is Packer really a benefit here as opposed to a quick shell script using the aws cli? I saw some people were also interested in using this as a sort of "dry run" invocation. I'm curious, for that use case, if it wouldn't be easier just to set the AMI name to something like "my-image-dry-run" and then set force_deregister_image to true. I know you put a lot of effort in to this PR, and I definitely encouraged you while writing it, so I apologize for not having this discussion earlier. We're still open to merging this, but I did want to just understand if this was the best way to accomplish your goals. thanks, --Matt Hooker

…

On Mon, Jun 12, 2017 at 2:24 PM jeremy-asher ***@***.***> wrote: Hey @mwhooker <https://github.com/mwhooker>. Pinged you on mail. Do you need anything else on this? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4681 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAA9AnCxJab6_8fyMwidCqK7_9J_6MQcks5sDayogaJpZM4MhM0R> .

[jeremy-asher]

@mwhooker I'm currently using this for two purposes:

One is to use the amazon-ebs builder to stand up an instance to run the amazon-chroot builder in. I don't need an AMI from the instance, so it saves time to immediately terminate the instance at the end instead of making AMIs I have to clean up or leave sitting around. I haven't wanted to migrate to a persistent instance due to the maintenance effort and the speed has been good enough for now.

The other use I have is for running test builds. I use the amazon-ebs builder to stand up an instance from an AMI I just built and I use the provision step to test that the instance works as expected. I don't need an AMI built from this either, so it saves time to throw it away at the end also.

Another use case that I was thinking of would be build an instance and use provisioners to produce a file artifact which is then extracted from the instance before terminating it. I don't have a specific use for that right now, but it seems like it would pair well with the artifice post-processor.

It would also be nice if artifice would output an AMI artifact instead of just a file since that could allow my first example to emit the actual AMI product from the inner build, but this isn't really necessary for my purposes.

[mwhooker]

Going to discuss this a bit more, but it's not ready to go out in this release. We're going to be better about releasing every 4 weeks or so, so next release won't be too far away

[jen20]

@jeremy-asher worth noting that for your first use case you can use the EBS Surrogate builder instead - it was designed exactly for that use case. You can find an example of using it here.

For running tests, perhaps Terraform might be more appropriate?

[SwampDragons]

I know there's been a lot of offline communication around this, and I believe that we've come down on solving the issue another way. Feel free to reopen if this isn't the case.

[patrickf55places]

@SwampDragons Could you share the solution? This could be handy for me as well (use case is testing packer configuration without actually creating an AMI).

[lorengordon]

I'm interested in the proposed solution. Posted my use case in the user group to see if anyone else has pointers on testing AMIs, https://groups.google.com/d/topic/packer-tool/Jj7yBpimlSY/discussion.

[jeremy-asher]

For anyone else who runs into this issue, I'm going to be using Terraform to do the testing instead.

If you don't want to do that, this patch isn't necessary to do the workflow I described where your build system executes a second Packer EBS builder run after the first. Instead, you can configure a fixed name (test-image for example) and set force_deregister_image to true. The only disadvantages are that it will take a bit longer to run the build and you will have one image from your last test build sitting around at all times.

[lorengordon]

@jeremy-asher thanks, yeah, that's the approach i decided to take also (force_deregister_image).

[subbu11]

@jeremy-asher @mwhooker

I am not able to find 'skip_register_ami' option in the documentation of ebs-chroot/ebs-surrogate.

I need to use it for skipping snapshot creation and ami creation.

Please help.

[cetex]

If i understood this correct it allows us to skip snapshoting and creating an ami, thats great.

I think It should be possible on ebs builder as well as you may want to build a whole new system, and have a test which tests building a whole new system but not "committing it" to an ami.

[avdhoot07]

@jeremy-asher @mwhooker

I have similar use case. Can you please share the updated code/branch link for github. I clone this branch https://github.com/hashicorp/packer.git but it is not showing me the updated changes.
Thanks in advance.

[willthames]

The simplest way to achieve this is to just exit 1 or equivalent at the end of your provisioning step (I use ansible so I just have a - fail: task at the end of my provisioning playbook). If the provisioning step fails, no AMI is created.

[jerryjvl]

The down-side of the exit 1 approach is that the intentional failure is indistinguishable from a real failure elsewhere in the provisioning / testing process.

I use InSpec to validate the AMI state, which I want to be able to run on my PR branches, but without creation of an AMI artifact until it gets merged to master. If the InSpec fails, I want a normal Packer exit code to indicate this failure... but... I do not want my early exit to avoid creating an AMI to be treated as an error. I cannot see a way to achieve both these goals with this work-around.

Combined this with the fact that Packer does not propagate exit codes, or as far as I can tell provide any mechanism to explicitly control the exit code ... and this turns into a very poor work-around for real-world build pipelines.

[jerryjvl]

@SwampDragons -- is there a credible alternate work-around to distinguish intentional failures that should be suppressed in a Packer config from actual real failures?

Note, valid_exit_codes does not help in this case, because that once again suppresses the exit code within Packer, which is behaviour I want to leverage to avoid creating the actual AMI.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.