Add skip_create_image option to the openstack builder

[fr123k]

Hi Packer team,

i just wanted to have the possibility to verify packer builds in openstack without actually
creating the image as a result.

Then i found the #7209 issue and searched the code base and realized that there is
already a implementation in the googlecloud builder.

https://github.com/hashicorp/packer/blob/master/builder/googlecompute/step_create_image.go#L26

Copied the main code and implemented a skip in all openstack builder steps that either
create an image or reference an already existing one.

• stepAddImageMembers
• stepCreateImage
• stepUpdateImageMinDisk
• stepUpdateImageTags
• stepUpdateImageVisibility

As i didn't found any test code for the googlecloud builder i also didn't
created some.

I performed manual tests with our onpremise Openstack installation.

Closes #7209

[codecov]

Codecov Report

Merging #10496 (6850e77) into master (94e22e6) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

Impacted Files	Coverage Δ
builder/openstack/image_config.go	39.28% <ø> (ø)
builder/openstack/step_add_image_members.go	0.00% <0.00%> (ø)
builder/openstack/step_create_image.go	0.00% <0.00%> (ø)
builder/openstack/step_update_image_mindisk.go	0.00% <0.00%> (ø)
builder/openstack/step_update_image_tags.go	0.00% <0.00%> (ø)
builder/openstack/step_update_image_visibility.go	0.00% <0.00%> (ø)

[fr123k]

This is the debug output from the manual integration test with skip_create_image=true

2021/01/16 21:32:15 [INFO] Packer version: 1.7.0-dev [go1.15.5 linux amd64]
2021/01/16 21:32:15 Checking 'PACKER_CONFIG' for a config file path
2021/01/16 21:32:15 'PACKER_CONFIG' not set; checking the default config file path
2021/01/16 21:32:15 Attempting to open config file: /root/.packerconfig
2021/01/16 21:32:15 [WARN] Config file doesn't exist: /root/.packerconfig
2021/01/16 21:32:15 Setting cache directory: /packer_cache
Running in background, not using a TTY
2021/01/16 21:32:15 Creating plugin client for path: /bin/packer
2021/01/16 21:32:15 Starting plugin: /bin/packer []string{"/bin/packer", "plugin", "packer-builder-openstack"}
2021/01/16 21:32:15 Waiting for RPC address for: /bin/packer
2021/01/16 21:32:15 packer-builder-openstack plugin: [INFO] Packer version: 1.7.0-dev [go1.15.5 linux amd64]
2021/01/16 21:32:15 packer-builder-openstack plugin: Checking 'PACKER_CONFIG' for a config file path
2021/01/16 21:32:15 packer-builder-openstack plugin: 'PACKER_CONFIG' not set; checking the default config file path
2021/01/16 21:32:15 packer-builder-openstack plugin: Attempting to open config file: /root/.packerconfig
2021/01/16 21:32:15 packer-builder-openstack plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2021/01/16 21:32:15 packer-builder-openstack plugin: Setting cache directory: /packer_cache
2021/01/16 21:32:15 packer-builder-openstack plugin: args: []string{"packer-builder-openstack"}
2021/01/16 21:32:15 Received unix RPC address for /bin/packer: addr is /tmp/packer-plugin838028050
2021/01/16 21:32:15 packer-builder-openstack plugin: Plugin address: unix /tmp/packer-plugin838028050
2021/01/16 21:32:15 packer-builder-openstack plugin: Waiting for connection...
2021/01/16 21:32:15 packer-builder-openstack plugin: Serving a plugin connection...
2021/01/16 21:32:15 Creating plugin client for path: /bin/packer
2021/01/16 21:32:15 Starting plugin: /bin/packer []string{"/bin/packer", "plugin", "packer-provisioner-file"}
2021/01/16 21:32:15 Waiting for RPC address for: /bin/packer
2021/01/16 21:32:15 packer-provisioner-file plugin: [INFO] Packer version: 1.7.0-dev [go1.15.5 linux amd64]
2021/01/16 21:32:15 packer-provisioner-file plugin: Checking 'PACKER_CONFIG' for a config file path
2021/01/16 21:32:15 packer-provisioner-file plugin: 'PACKER_CONFIG' not set; checking the default config file path
2021/01/16 21:32:15 packer-provisioner-file plugin: Attempting to open config file: /root/.packerconfig
2021/01/16 21:32:15 packer-provisioner-file plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2021/01/16 21:32:15 packer-provisioner-file plugin: Setting cache directory: /packer_cache
2021/01/16 21:32:15 packer-provisioner-file plugin: args: []string{"packer-provisioner-file"}
2021/01/16 21:32:15 packer-provisioner-file plugin: Plugin address: unix /tmp/packer-plugin886646216
2021/01/16 21:32:15 packer-provisioner-file plugin: Waiting for connection...
2021/01/16 21:32:15 Received unix RPC address for /bin/packer: addr is /tmp/packer-plugin886646216
2021/01/16 21:32:15 packer-provisioner-file plugin: Serving a plugin connection...
2021/01/16 21:32:15 Creating plugin client for path: /bin/packer
2021/01/16 21:32:15 Starting plugin: /bin/packer []string{"/bin/packer", "plugin", "packer-provisioner-shell"}
2021/01/16 21:32:15 Waiting for RPC address for: /bin/packer
2021/01/16 21:32:15 packer-provisioner-shell plugin: [INFO] Packer version: 1.7.0-dev [go1.15.5 linux amd64]
2021/01/16 21:32:15 packer-provisioner-shell plugin: Checking 'PACKER_CONFIG' for a config file path
2021/01/16 21:32:15 packer-provisioner-shell plugin: 'PACKER_CONFIG' not set; checking the default config file path
2021/01/16 21:32:15 packer-provisioner-shell plugin: Attempting to open config file: /root/.packerconfig
2021/01/16 21:32:15 packer-provisioner-shell plugin: [WARN] Config file doesn't exist: /root/.packerconfig
2021/01/16 21:32:15 packer-provisioner-shell plugin: Setting cache directory: /packer_cache
2021/01/16 21:32:15 packer-provisioner-shell plugin: args: []string{"packer-provisioner-shell"}
2021/01/16 21:32:15 Received unix RPC address for /bin/packer: addr is /tmp/packer-plugin536600697
2021/01/16 21:32:15 packer-provisioner-shell plugin: Plugin address: unix /tmp/packer-plugin536600697
2021/01/16 21:32:15 packer-provisioner-shell plugin: Waiting for connection...
2021/01/16 21:32:15 packer-provisioner-shell plugin: Serving a plugin connection...
2021/01/16 21:32:15 Preparing build: openstack
2021/01/16 21:32:16 Build debug mode: false
2021/01/16 21:32:16 Force build: false
2021/01/16 21:32:16 On error: 
2021/01/16 21:32:16 Waiting on builds to complete...
2021/01/16 21:32:16 Starting build run: openstack
2021/01/16 21:32:16 Running builder: openstack
2021/01/16 21:32:16 [INFO] (telemetry) Starting builder openstack
openstack: output will be in this color.

==> openstack: Loading flavor: vc2r4Gd20G
2021/01/16 21:32:16 packer-builder-openstack plugin: [INFO] Loading flavor by ID: vc2r4Gd20G
2021/01/16 21:32:16 packer-builder-openstack plugin: [ERROR] Failed to find flavor by ID: Resource not found
2021/01/16 21:32:16 packer-builder-openstack plugin: [INFO] Loading flavor by name: vc2r4Gd20G
    openstack: Verified flavor. ID: 891998de-e278-49e5-a596-a4ac0c56bb78
==> openstack: Using existing SSH private key
2021/01/16 21:32:16 packer-builder-openstack plugin: Using Image Filters {ID: Limit:0 Marker: Name:sre-packer-debug-base Visibility:public MemberStatus:accepted Owner: Status:active SizeMin:0 SizeMax:0 Sort:created_at:desc SortKey: SortDir: Tags:[] CreatedAtQuery:<nil> UpdatedAtQuery:<nil> ContainerFormat: DiskFormat:}
    openstack: Found Image ID: 0a325324-218c-454c-bb31-73fda9028d38
==> openstack: Creating volume...
==> openstack: Waiting for volume packer_60035b60-3387-1061-2ee9-cc65bbe89ab8 (volume id: 6f62b121-5174-42ae-b6f7-5a98f3e086c6) to become available...
2021/01/16 21:32:17 packer-builder-openstack plugin: Waiting for volume creation status: creating
2021/01/16 21:32:18 [ERR] Checkpoint error: Get "https://checkpoint-api.hashicorp.com/v1/check/packer?arch=amd64&os=linux&signature=4f66c527-1291-70ff-ab65-f6b7a83fa6bc&version=1.7.0-dev": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
    openstack: Volume ID: 6f62b121-5174-42ae-b6f7-5a98f3e086c6
==> openstack: Launching server...
==> openstack: Launching server...
2021/01/16 21:32:21 packer-builder-openstack plugin: server id: bf65ce61-8046-4956-8a21-640e9d8151c6
2021/01/16 21:32:21 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE]
    openstack: Server ID: bf65ce61-8046-4956-8a21-640e9d8151c6
==> openstack: Waiting for server to become ready...
2021/01/16 21:32:21 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE] currently BUILD (0%)
2021/01/16 21:32:24 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE] currently BUILD (0%)
2021/01/16 21:32:26 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE] currently BUILD (0%)
2021/01/16 21:32:28 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE] currently BUILD (0%)
2021/01/16 21:32:31 packer-builder-openstack plugin: Waiting for state to become: [ACTIVE] currently BUILD (0%)
2021/01/16 21:32:34 packer-builder-openstack plugin: [INFO] Not using winrm communicator, skipping get password...
==> openstack: Creating floating IP using network 6a0b9177-6110-4dd6-ad61-89e3ef8568d3 ...
    openstack: Created floating IP: 'd3d0ce57-6637-4bea-9c55-e5e92212067e' (xxx.xxx.xxx.xxx)
==> openstack: Associating floating IP 'd3d0ce57-6637-4bea-9c55-e5e92212067e' (xxx.xxx.xxx.xxx) with instance port...
2021/01/16 21:32:37 packer-builder-openstack plugin: Instance interface: 0: {PortState:ACTIVE FixedIPs:[{SubnetID:eff4b84d-814d-43da-a57f-f241c6fe40ea IPAddress:xx.xx.xx.xx}] PortID:36283d01-4a87-4069-a663-587a7c558382 NetID:506ba3d5-5fda-43d3-bc01-b30b3b216cac MACAddr:xx:xx:xx:xx:xx:xx}
2021/01/16 21:32:39 packer-builder-openstack plugin: [DEBUG] Using floating IP xxx.xxx.xxx.xxx to connect
2021/01/16 21:32:39 packer-builder-openstack plugin: [INFO] Waiting for SSH, up to timeout: 24h0m0s
2021/01/16 21:32:39 packer-builder-openstack plugin: [DEBUG] Using floating IP xxx.xxx.xxx.xxx to connect
    openstack: Added floating IP 'd3d0ce57-6637-4bea-9c55-e5e92212067e' (xxx.xxx.xxx.xxx) to instance!
==> openstack: Using ssh communicator to connect: xxx.xxx.xxx.xxx
==> openstack: Waiting for SSH to become available...
2021/01/16 21:32:50 packer-builder-openstack plugin: [DEBUG] TCP connection to SSH ip/port failed: dial tcp xxx.xxx.xxx.xxx:22: connect: connection refused
2021/01/16 21:32:55 packer-builder-openstack plugin: [DEBUG] Using floating IP xxx.xxx.xxx.xxx to connect
2021/01/16 21:32:55 packer-builder-openstack plugin: [DEBUG] TCP connection to SSH ip/port failed: dial tcp xxx.xxx.xxx.xxx:22: connect: connection refused
2021/01/16 21:33:00 packer-builder-openstack plugin: [DEBUG] Using floating IP xxx.xxx.xxx.xxx to connect
2021/01/16 21:33:00 packer-builder-openstack plugin: [INFO] Attempting SSH connection to xxx.xxx.xxx.xxx:22...
2021/01/16 21:33:00 packer-builder-openstack plugin: [DEBUG] reconnecting to TCP connection for SSH
2021/01/16 21:33:00 packer-builder-openstack plugin: [DEBUG] handshaking with SSH
2021/01/16 21:33:00 packer-builder-openstack plugin: [DEBUG] handshake complete!
2021/01/16 21:33:00 packer-builder-openstack plugin: [INFO] no local agent socket, will not connect agent
2021/01/16 21:33:00 packer-builder-openstack plugin: Running the provision hook
2021/01/16 21:33:00 [INFO] (telemetry) Starting provisioner file
==> openstack: Connected to SSH!
==> openstack: Uploading /packer/sre-packer-debug/provision.sh => /tmp/provision.sh
2021/01/16 21:33:00 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
2021/01/16 21:33:00 packer-provisioner-file plugin: [INFO] 893 bytes written for 'uploadData'
2021/01/16 21:33:00 [INFO] 893 bytes written for 'uploadData'
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Starting remote scp process:  scp -vt /tmp
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Started SCP session, beginning transfers...
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] scp: Uploading provision.sh: perms=C0644 size=893
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] SCP session complete, closing stdin pipe.
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Waiting for SSH session to complete.
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] scp stderr (length 29): Sink: C0644 893 provision.sh
2021/01/16 21:33:02 packer-provisioner-file plugin: closing
2021/01/16 21:33:02 closing
2021/01/16 21:33:02 packer-builder-openstack plugin: closing
2021/01/16 21:33:02 [INFO] (telemetry) ending file
2021/01/16 21:33:02 [INFO] (telemetry) Starting provisioner shell
2021/01/16 21:33:02 packer-provisioner-shell plugin: Opening /tmp/packer-shell429735300 for reading
==> openstack: Provisioning with shell script: /tmp/packer-shell429735300
2021/01/16 21:33:02 packer-provisioner-shell plugin: [INFO] 39 bytes written for 'uploadData'
2021/01/16 21:33:02 [INFO] 39 bytes written for 'uploadData'
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Starting remote scp process:  scp -vt /tmp
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Started SCP session, beginning transfers...
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Copying input data into temporary file so we can read the length
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] scp: Uploading script_1995.sh: perms=C0644 size=39
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] SCP session complete, closing stdin pipe.
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Waiting for SSH session to complete.
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] scp stderr (length 30): Sink: C0644 39 script_1995.sh
2021/01/16 21:33:02 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
2021/01/16 21:33:03 packer-builder-openstack plugin: [DEBUG] starting remote command: chmod 0755 /tmp/script_1995.sh
2021/01/16 21:33:03 packer-builder-openstack plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:33:03 [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:33:03 [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:33:03 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:33:03 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
2021/01/16 21:33:03 packer-builder-openstack plugin: [DEBUG] starting remote command: chmod +x /tmp/script_1995.sh; PACKER_BUILDER_TYPE='openstack' PACKER_BUILD_NAME='openstack'  /tmp/script_1995.sh
    openstack: SSH connection successful
    openstack: Wait 10 minutes for acknowlegdment file to exist /tmp/ack. Stop provisioning after 10 minutes if the file doesnt exists.
    openstack: Wait an other 30 seconds (attempts = 1, max attempts = 20)
2021/01/16 21:34:03 packer-builder-openstack plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] 0 bytes written for 'stderr'
2021/01/16 21:34:03 [INFO] 382 bytes written for 'stdout'
2021/01/16 21:34:03 [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 packer-provisioner-shell plugin: [INFO] 0 bytes written for 'stderr'
2021/01/16 21:34:03 packer-provisioner-shell plugin: [INFO] 382 bytes written for 'stdout'
2021/01/16 21:34:03 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
    openstack: Wait an other 30 seconds (attempts = 2, max attempts = 20)
    openstack: File /tmp/ack found. Wait forever until the file /tmp/shutdown exists.
2021/01/16 21:34:03 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
    openstack: File /tmp/shutdown found. Finish Provisioning
2021/01/16 21:34:03 packer-builder-openstack plugin: [DEBUG] starting remote command: rm -f /tmp/script_1995.sh
2021/01/16 21:34:03 packer-builder-openstack plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:34:03 packer-builder-openstack plugin: [DEBUG] Opening new ssh session
2021/01/16 21:34:03 packer-builder-openstack plugin: [DEBUG] starting remote command: rm -f
2021/01/16 21:34:03 packer-builder-openstack plugin: [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] RPC endpoint: Communicator ended with: 0
2021/01/16 21:34:03 packer-provisioner-shell plugin: [INFO] RPC client: Communicator ended with: 0
2021/01/16 21:34:03 [INFO] (telemetry) ending shell
==> openstack: Stopping server: bf65ce61-8046-4956-8a21-640e9d8151c6 ...
2021/01/16 21:34:04 packer-builder-openstack plugin: Waiting for state to become: [SHUTOFF STOPPED]
    openstack: Waiting for server to stop: bf65ce61-8046-4956-8a21-640e9d8151c6 ...
2021/01/16 21:34:04 packer-builder-openstack plugin: Waiting for state to become: [SHUTOFF STOPPED] currently ACTIVE (0%)
==> openstack: Detaching volume packer_60035b60-3387-1061-2ee9-cc65bbe89ab8 (volume id: 6f62b121-5174-42ae-b6f7-5a98f3e086c6)
==> openstack: Waiting for volume packer_60035b60-3387-1061-2ee9-cc65bbe89ab8 (volume id: 6f62b121-5174-42ae-b6f7-5a98f3e086c6) to become available...
==> openstack: Skipping image creation...
==> openstack: Skipping image update tags...
==> openstack: Skipping image update visibility...
==> openstack: Skipping image add members...
==> openstack: Skipping image update mindisk...
==> openstack: Deleted temporary floating IP 'd3d0ce57-6637-4bea-9c55-e5e92212067e' (xxx.xxx.xxx.xxx)
==> openstack: Terminating the source server: bf65ce61-8046-4956-8a21-640e9d8151c6 ...
2021/01/16 21:34:08 packer-builder-openstack plugin: Waiting for state to become: [DELETED]
2021/01/16 21:34:09 packer-builder-openstack plugin: Waiting for state to become: [DELETED] currently SHUTOFF (0%)
2021/01/16 21:34:11 packer-builder-openstack plugin: Waiting for state to become: [DELETED] currently SHUTOFF (0%)
2021/01/16 21:34:14 packer-builder-openstack plugin: [INFO] 404 on ServerStateRefresh, returning DELETED
==> openstack: Deleting volume: 6f62b121-5174-42ae-b6f7-5a98f3e086c6 ...
2021/01/16 21:34:14 [INFO] (telemetry) ending openstack
Build 'openstack' finished after 1 minute 58 seconds.
==> Wait completed after 1 minute 58 seconds

==> Builds finished but no artifacts were created.
2021/01/16 21:34:14 [INFO] (telemetry) Finalizing.
==> Wait completed after 1 minute 58 seconds

==> Builds finished but no artifacts were created.
2021/01/16 21:34:15 waiting for all plugin processes to complete...
2021/01/16 21:34:15 /bin/packer: plugin process exited

It works as expected.

[sylviamoss]

Thanks, LGTM 👍🏼

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.