ci: add yq shim for v3/v4 support

[mkulke]

yq3 (installed in kata builds as a side effect) and yq4 are different in cli and query syntax. the shim should abstract from those differences

[stevenhorsman]

Thanks @mkulke - the code looks good to me and it's nice timing as I've just started to try out ditching the caching and was about to look at this. I will try it in practice building the podvm too now.

[stevenhorsman]

So the workflow is failing with:

Makefile.defaults:6: *** "/work/cloud-api-adaptor/hack/yq-shim.sh not found, consider doing snap install yq".  Stop.

So maybe there is an issue with the $(ROOT_PATH)/hack/yq-shim.sh not having src/cloud-api-adaptor in the path, or the workflow running in the wrong directory?

Anyway I can confirm that pulling this into my switch to main branch it allows me to install the yq v3 that the kata-agent in main needs, so thanks so much!

[mkulke]

So maybe there is an issue with the $(ROOT_PATH)/hack/yq-shim.sh not having src/cloud-api-adaptor in the path, or the workflow running in the wrong directory?

Anyway I can confirm that pulling this into my switch to main branch it allows me to install the yq v3 that the kata-agent in main needs, so thanks so much!

The shim wasn't present in the Dockerfile, I pushed a fix. The CI passes the docker build step now, let's see whether the podvm build also works.

I'm running a non-docker build on a fork at the moment that will build kata-agent (and hence intall yq v3) and then later CAA binaries that used to rely on v4, so we'll see whether the fallback works.

[beraldoleal]

Hi @mkulke I got it! :) :You was talking about the caa side, not kata side!

<joke>Apart from introducing another "shim"</joke>, lgtm.

[mkulke]

Hi @mkulke I got it! :) :You was talking about the caa side, not kata side!

Apart from introducing another "shim", lgtm.

I somehow thought that kata had an abstraction over yq queries, but I must have misremembered. For CAA it's easier to deal with the version ambiguity, as it's only a couple of queries.

[surajssd]

One nit, otherwise LGTM

[mkulke]

ok the worklfow in which the kata-build installs v3 and things still work passed

the libvirt tests also seem to execute now, so that's good. i'll merge after the libvirt tests hopefully pass

[surajssd]

BTW, one question, we should use the yq-shim only when dealing with stuff related to kata or everywhere in CAA now we should use yq-shim?

[stevenhorsman]

The amd-64 podvm image build failures look like a network flake:

#17 139.3 ==> qemu.ubuntu: Retrieving ISO
#17 139.3 ==> qemu.ubuntu: Trying https://cloud-images.ubuntu.com/releases/focal/release-20230107/ubuntu-20.04-server-cloudimg-amd64.img
#17 139.3 ==> qemu.ubuntu: Trying https://cloud-images.ubuntu.com/releases/focal/release-20230107/ubuntu-20.04-server-cloudimg-amd64.img?checksum=sha256%3A3895e38566e5c2c019f5c6f825ab7570ee34dac6b9142fab0c7e5a78084c4280
#17 1409.3 ==> qemu.ubuntu: Download failed read tcp 172.17.0.2:49390->185.125.190.37:443: read: connection reset by peer
#17 1409.3 ==> qemu.ubuntu: error downloading ISO: [read tcp 172.17.0.2:49390->185.125.190.37:443: read: connection reset by peer]
#17 1409.3 Build 'qemu.ubuntu' errored after 21 minutes 10 seconds: error downloading ISO: [read tcp 172.17.0.2:49390->185.125.190.37:443: read: connection reset by peer]

so I've re-run them

[mkulke]

BTW, one question, we should use the yq-shim only when dealing with stuff related to kata or everywhere in CAA now we should use yq-shim?

kata will mandate v3 for the time being, and it will even install it on you machine, overriding v4. so the yq-shim adds a compatibility layer that will make v4 queries (CAA) work with v3 and v4

this allows you to build kata + caa on the same machine. so you'd use the yq-shim when v3 is required (e.g. the azure build)