hashicorp/packer-plugin-sdk incompatible with zclconf/go-cty v1.11.0

[mschuchard]

Overview of the Issue

If the package github.com/zclconf/go-cty v1.11.0 is used with the Packer SDK, then an error is thrown. I am unsure if this is an error with Packer SDK or go-cty. If the error is with the other package, then please redirect me. I am opening this issue on both trackers.

Reproduction Steps

Do anything with a Packer plugin built with go-cty v1.11.0 as this occurs immediately after Packer loads the plugin.

Packer Plugin SDK/go-cty versions

packer-sdk 0.3.1 and 0.3.2 both throw this error.
go-cty v1.10.0 works perfectly. v1.11.0 throws this error.

Operating system and Environment details

go 1.18

Log Fragments and crash.log files

panic: ConfigSpec failed: gob: type cty.Type has no exported fields [recovered]
	panic: ConfigSpec failed: gob: type cty.Type has no exported fields

goroutine 1 [running]:
log.Panic({0xc000b3c2a8, 0xc000b3c2e8, 0xc000b3c318})
	/opt/hostedtoolcache/go/1.17.11/x64/src/log/log.go:354 +0x65
github.com/hashicorp/packer/packer.(*cmdProvisioner).checkExit(0x90, {0x43e1e60, 0xc00052daa0}, 0x0)
	/home/runner/work/packer/packer/packer/cmd_provisioner.go:47 +0x7f
github.com/hashicorp/packer/packer.(*cmdProvisioner).ConfigSpec.func1()
	/home/runner/work/packer/packer/packer/cmd_provisioner.go:19 +0x39
panic({0x43e1e60, 0xc00052daa0})
	/opt/hostedtoolcache/go/1.17.11/x64/src/runtime/panic.go:1038 +0x215
github.com/hashicorp/packer-plugin-sdk/rpc.(*commonClient).ConfigSpec(0xc0007905e0)
	/home/runner/go/pkg/mod/github.com/hashicorp/[email protected]/rpc/common.go:44 +0x297
github.com/hashicorp/packer/packer.(*cmdProvisioner).ConfigSpec(0xc000b3e0e0)
	/home/runner/work/packer/packer/packer/cmd_provisioner.go:22 +0x65
github.com/hashicorp/packer/hcl2template.decodeHCL2Spec({0x5ce8d10, 0xc000725e60}, 0xc00053e9a8, {0x7f7dd9ff6450, 0xc00053e9a8})
	/home/runner/work/packer/packer/hcl2template/decode.go:17 +0x39
github.com/hashicorp/packer/hcl2template.(*HCL2Provisioner).HCL2Prepare(0xc0005da5a0, 0xc0005da570)
	/home/runner/work/packer/packer/hcl2template/types.hcl_provisioner.go:51 +0x439
github.com/hashicorp/packer/hcl2template.(*PackerConfig).startProvisioner(0xc00056bb80, {{{0xc0005340f7, 0x6}, {0xc0005340fe, 0x6}}, {0x0, 0x0}, {0x5ce8d10, 0xc000724b10}}, 0xc0005526c0, ...)
	/home/runner/work/packer/packer/hcl2template/types.build.provisioners.go:199 +0x6a5
github.com/hashicorp/packer/hcl2template.(*PackerConfig).getCoreBuildProvisioner(0x0, {{{0xc0005340f7, 0x6}, {0xc0005340fe, 0x6}}, {0x0, 0x0}, {0x5ce8d10, 0xc000724b10}}, 0xc0005526c0, ...)
	/home/runner/work/packer/packer/hcl2template/types.packer_config.go:482 +0x105
github.com/hashicorp/packer/hcl2template.(*PackerConfig).getCoreBuildProvisioners(0xc0005340fe, {{{0xc0005340f7, 0x6}, {0xc0005340fe, 0x6}}, {0x0, 0x0}, {0x5ce8d10, 0xc000724b10}}, {0xc0005300e0, ...}, ...)
	/home/runner/work/packer/packer/hcl2template/types.packer_config.go:470 +0x1cb
github.com/hashicorp/packer/hcl2template.(*PackerConfig).GetBuilds(0xc00056bb80, {{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, 0x0, 0x0, {0x0, ...}, ...})
	/home/runner/work/packer/packer/hcl2template/types.packer_config.go:670 +0x14fc
github.com/hashicorp/packer/command.(*BuildCommand).RunContext(0xc000c68ba0, {0x5ce83e0, 0xc000c6c980}, 0xc0007967e0)
	/home/runner/work/packer/packer/command/build.go:182 +0x21e
github.com/hashicorp/packer/command.(*BuildCommand).Run(0xc000c68ba0, {0xc00006e0a0, 0x2, 0x2})
	/home/runner/work/packer/packer/command/build.go:40 +0xc5
github.com/mitchellh/cli.(*CLI).Run(0xc0003bca00)
	/home/runner/go/pkg/mod/github.com/mitchellh/[email protected]/cli.go:262 +0x5f8
main.wrappedMain()
	/home/runner/work/packer/packer/main.go:262 +0xb28
main.realMain()
	/home/runner/work/packer/packer/main.go:49 +0xf3
main.main()
	/home/runner/work/packer/packer/main.go:35 +0x19

Additional Information

I suspect this is referencing the cty.Type from the <plugin>.hcl2spec.go. If I attempt to regenerate that file with go-cty v1.11.0, then the following error is thrown:

mapstructure-to-hcl2: main.[]: Unexpected package creation during export data loading

[mschuchard]

This is confirmed to be an issue in the packer-sdk, but the discussion is occurring in zclconf/go-cty#134.

[mschuchard]

For similar reasons HCL v2 cannot be updated past v2.14.1.

[nywilken]

As folks on the thread may already know, encoding/gob support was removed from github.com/zclconf/[email protected]. The Packer plugin SDK relies on gob for supporting HCL2 templates. The removal of the encoding causes Packer to crash when using HCL2 templates.

The team is working internally to find the best path forward, as a change to the wire protocol from gob will introduce a breaking change and plugin incompatibilities. We are working to notify plugin developers of the incompatible go-cty package and ask that all external plugins pin the versions of the following two packages to those used by the Packer plugin SDK.

github.com/hashicorp/hcl/v2 v2.13.0
github.com/zclconf/go-cty v1.10.0

I'm closing this issue in favor of #135. Please feel free to reach out if you have any questions or suggestions. Cheers.