Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporary SSH Key files not generated or used in proxmox-iso #301

Open
bl1nk1n opened this issue Nov 19, 2024 · 4 comments · May be fixed by #302
Open

Temporary SSH Key files not generated or used in proxmox-iso #301

bl1nk1n opened this issue Nov 19, 2024 · 4 comments · May be fixed by #302

Comments

@bl1nk1n
Copy link

bl1nk1n commented Nov 19, 2024

Overview of the Issue

I am trying to avoid having to hardcode an SSH password or SSH key that will need to be changed/removed later. In the documentation it says:

If no communicator is defined, an SSH key is generated for use, and is used in the image's Cloud-Init settings for provisioning.

When not specifying any communicator, ssh_username, ssh_password, or ssh_private_key_file, it complains about needing an ssh_username (which matches the above statement about SSH being used by default; and the fact that the variable is required by the SSH communicator).

After supplying it with ssh_username I would expect it to generate a temporary key pair to be given to cloud-init, use those credentials to provision, and then remove them from the system. However, it looks like it tries to authenticate with method "none" and inspecting the server created, the key is not in any authorized_keys files.

Reproduction Steps

In my specific instance I was creating an ubuntu-24.04.1 template from the official ISO and passing a CD with the meta-data and user-data (autoinstall) files (which should include cloud-init options). I'm running Packer in the official Packer docker container with no modifications.

I run init, validate, and build in the docker container, and it works as expected until my above issue:

  • download the image
  • create the ISO containing meta-data and user-data
  • upload both ISOs to Proxmox
  • create a VM from the Ubuntu ISO with the cidata ISO mounted
  • pass boot_command from source to instruct it to use the mounted cidata CD
  • Ubuntu autoinstalls using the mounted CD
  • VM reboots and is accessible by my host running Packer

Software Versions

Docker Version

v4.36.0

Packer Docker Image

light - f64016a1388f8d4b807978369bf09269b1823719562d4c78970a72ae10627b23

Packer Version

$ docker run --rm -e PACKER_LOG=1 hashicorp/packer:light version          
2024/11/19 02:44:15 [INFO] Packer version: 1.12.0-alpha1 [go1.21.12 linux amd64]
...
2024/11/19 02:44:15 machine readable: version []string{"1.12.0"}
2024/11/19 02:44:15 machine readable: version-prelease []string{"alpha1"}
2024/11/19 02:44:15 machine readable: version-commit []string{"9ff6fe7f"}
Packer v1.12.0-alpha1
...

proxmox Plugin Version

v1.2.1_x5.0_linux_amd64

Proxmox Version

pve-manager/8.2.8/a577cfa684c7476d

Simplified Packer Buildfile

Removed everything that didn't seem pertinent to how keys should be created/deployed. Can add more detail later if need be.

packer {
    required_version = "~> 1"
    required_plugins {
        proxmox = {
            version = "~> 1"
            source = "github.com/hashicorp/proxmox"
        }
    }
}
source "proxmox-iso" "ubuntu-template" {
    qemu_agent = true
    additional_iso_files {
        cd_files = [
            "./files/autoinstall/meta-data",
            "./files/autoinstall/user-data"
        ]
        cd_label = "cidata"
        iso_storage_pool = "local"
    }
    cloud_init = true
    cloud_init_storage_pool = "local-lvm"
    boot_wait = "5s"
    boot_command = [
        "<esc><esc><esc><esc><wait>",
        "e<wait>",
        "<down><down><down><end>",
        " autoinstall ds=\"nocloud;\"<wait>",
        "<f10><wait>"
    ]
    # communicator = "ssh"
    ssh_username = "automation"
    # temporary_key_pair_type = "ed25519"
    ssh_timeout = "10m"
}

build {
    sources = ["source.proxmox-iso.ubuntu-template"]

    # Provisioning the VM Template for Cloud-Init Integration
    provisioner "shell" {
        inline = [
            "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done",
            "sudo rm /etc/ssh/ssh_host_*",
            "sudo truncate -s 0 /etc/machine-id",
            "sudo apt -y autoremove --purge",
            "sudo apt -y clean",
            "sudo apt -y autoclean",
            "sudo cloud-init clean",
            "sudo rm -f /etc/cloud/cloud.cfg.d/subiquity-disable-cloudinit-networking.cfg",
            "sudo sync"
        ]
    }
    provisioner "file" {
        source = "files/99-pve.cfg"
        destination = "/tmp/99-pve.cfg"
    }
    provisioner "shell" {
        inline = [ "sudo cp /tmp/99-pve.cfg /etc/cloud/cloud.cfg.d/99-pve.cfg"]
    }
}

Log Fragments and crash.log files

2024/11/19 01:50:11 [INFO] Packer version: 1.12.0-alpha1 [go1.21.12 linux amd64]
...
==> ubuntu-template.proxmox-iso.ubuntu-template: Waiting for SSH to become available...
2024/11/19 01:51:22 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:51:22 [DEBUG] Unable to get address during connection step: 500 QEMU guest agent is not running   
2024/11/19 01:51:22 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:51:22 [INFO] Waiting for SSH, up to timeout: 10m0s
2024/11/19 01:51:25 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:51:25 [DEBUG] Error getting SSH address: 500 QEMU guest agent is not running
...
2024/11/19 01:58:14 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:14 [DEBUG] Error getting SSH address: 500 QEMU guest agent is not running
2024/11/19 01:58:21 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:21 [INFO] Attempting SSH connection to 192.168.209.233:22...
2024/11/19 01:58:21 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:21 [DEBUG] reconnecting to TCP connection for SSH
2024/11/19 01:58:21 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:21 [DEBUG] handshaking with SSH
2024/11/19 01:58:22 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:22 [DEBUG] SSH handshake err: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain
2024/11/19 01:58:22 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 01:58:22 [DEBUG] Detected authentication error. Increasing handshake attempts.
...
2024/11/19 02:01:15 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:15 [INFO] Attempting SSH connection to 192.168.209.233:22...
2024/11/19 02:01:15 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:15 [DEBUG] reconnecting to TCP connection for SSH
2024/11/19 02:01:15 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:15 [DEBUG] handshaking with SSH
2024/11/19 02:01:15 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:15 [DEBUG] SSH handshake err: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain
2024/11/19 02:01:15 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:15 [DEBUG] Detected authentication error. Increasing handshake attempts.
==> ubuntu-template.proxmox-iso.ubuntu-template: Timeout waiting for SSH.
2024/11/19 02:01:22 packer-plugin-proxmox_v1.2.1_x5.0_linux_amd64 plugin: 2024/11/19 02:01:22 [DEBUG] SSH wait cancelled. Exiting loop.
...
2024/11/19 02:01:26 machine readable: error-count []string{"1"}
...
2024/11/19 02:01:26 machine readable: ubuntu-template.proxmox-iso.ubuntu-template,error []string{"Timeout waiting for SSH."}
...
Build 'ubuntu-template.proxmox-iso.ubuntu-template' errored after 11 minutes 12 seconds: Timeout waiting for SSH.
...
--> ubuntu-template.proxmox-iso.ubuntu-template: Timeout waiting for SSH.
...
@bl1nk1n bl1nk1n added the bug label Nov 19, 2024
@bl1nk1n
Copy link
Author

bl1nk1n commented Nov 19, 2024

I'm going to try just setting the ssh_private_key_file to the one I set up for later (to the automation user, but temporary SSH key files would be much more useful.

I would also fix it myself/figure out if it really is a bug if I wasn't a noob to Packer. I dug through a TON of code, but couldn't find anything that made me go "AH here it is."

@bl1nk1n
Copy link
Author

bl1nk1n commented Nov 19, 2024

The only other thing I can think of right now is that maybe it is related to my CD usage for autoinstall instead of HTTP? Perhaps proxmox-iso cannot inject the credentials into the cloud-init/autoinstall config because it is being loaded into an ISO instead of dynamically adding it to the file via HTTP server?

@bl1nk1n
Copy link
Author

bl1nk1n commented Nov 21, 2024

When comparing the Builder.Run() functions between proxmox-iso and amazonebs I noticed that they have the step:

&awscommon.StepKeyPair{
    Debug:        b.config.PackerDebug,
    Comm:         &b.config.RunConfig.Comm,
    IsRestricted: b.config.IsChinaCloud(),
    DebugKeyPath: fmt.Sprintf("ec2_%s.pem", b.config.PackerBuildName),
    Tags:         b.config.RunTags,
    Ctx:          b.config.ctx,
},

before they attempt their connection step:

&communicator.StepConnect{
    Config: &b.config.RunConfig.Comm,
    Host: awscommon.SSHHost(ec2conn, b.config.SSHInterface, b.config.Comm.Host()),
    SSHPort: awscommon.Port(b.config.SSHInterface, b.config.Comm.Port()),
    SSHConfig: b.config.RunConfig.Comm.SSHConfigFunc()
},

The connection step is very very similar to what is in proxmox-iso:

&communicator.StepConnect{
    Config:    comm,
    Host:      commHost((*comm).Host()),
    SSHConfig: (*comm).SSHConfigFunc(),
},

So it seems like we are missing a key generation step here. It looks like they have their own custom key generation step, but the SSH communicator seems to have StepSSHKeyGen we can use:

type StepSSHKeyGen struct {
	CommConf *Config
	SSHTemporaryKeyPair
}

// Run executes the Packer build step that generates SSH key pairs.
// The key pairs are added to the ssh config
func (s *StepSSHKeyGen) Run(ctx context.Context, state multistep.StateBag) multistep.StepAction {
	ui := state.Get("ui").(packersdk.Ui)
	comm := s.CommConf

	if comm.SSHPrivateKeyFile != "" {
		ui.Say("Using existing SSH private key")
		privateKeyBytes, err := comm.ReadSSHPrivateKeyFile()
		if err != nil {
			state.Put("error", err)
			return multistep.ActionHalt
		}

		comm.SSHPrivateKey = privateKeyBytes

		publicKeyBytes, err := sshkey.PublicKeyFromPrivate(privateKeyBytes)
		if err != nil {
			state.Put("error", err)
			return multistep.ActionHalt
		}
		comm.SSHPublicKey = publicKeyBytes

		return multistep.ActionContinue
	}

	algorithm := s.SSHTemporaryKeyPair.SSHTemporaryKeyPairType
	if algorithm == "" {
		algorithm = sshkey.RSA.String()
	}
	a, err := sshkey.AlgorithmString(algorithm)
	if err != nil {
		err := fmt.Errorf("%w: possible algorithm types are `dsa` | `ecdsa` | `ed25519` | `rsa` ( the default )", err)
		state.Put("error", err)
		return multistep.ActionHalt
	}

	ui.Say(fmt.Sprintf("Creating temporary %s SSH key for instance...", strings.ToUpper(a.String())))
	pair, err := sshkey.GeneratePair(a, nil, s.SSHTemporaryKeyPairBits)
	if err != nil {
		err := fmt.Errorf("Error creating temporary SSH key: %s", err)
		state.Put("error", err)
		ui.Error(err.Error())
		return multistep.ActionHalt
	}

	comm.SSHPrivateKey = pair.Private
	comm.SSHPublicKey = pair.Public

	return multistep.ActionContinue
}

@bl1nk1n bl1nk1n linked a pull request Nov 21, 2024 that will close this issue
@bl1nk1n
Copy link
Author

bl1nk1n commented Nov 21, 2024

I think after reading a TON more code I'm starting to understand something; but I have never written Go before, and I've only just touched Packer in the past week. So please be generous with the constructive criticism!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants