clusterctl init --infrastructure aws --list-images returns an error when env variables are not defined

[detiber]

What steps did you take and what happened:

• run clusterctl init --infrastructure aws --list-images without setting the AWS_B64ENCODED_CREDENTIALS env var
• Get: Error: failed to get provider components for the "aws" provider: failed to perform variable substitution: value for variables [AWS_B64ENCODED_CREDENTIALS] is not set. Please set the value using os environment variables or the clusterctl config file

What did you expect to happen:

• The command to run successfully without requiring the env variables to be set

Environment:

• Cluster-api version: master
• Minikube/KIND version: kind v0.6.1
• Kubernetes version: (use kubectl version): v1.16.7
• OS (e.g. from /etc/os-release): Fedora 31

/kind bug

[vincepri]

/assign @fabriziopandini
/priority important-soon
/milestone v0.3.x

[fabriziopandini]

@detiber @vincepri this is tricky because there is no guarantee that a YAML without env variables is a valid YAML.

e.g nothing prevents that the variable is the image repository or the image tag

In other words, from a clusterctl PoV only a fully compiled YAML can be input for further processing, like e.g. inspecting images.

[vincepri]

Has this been fixed?

[kferrone]

I'm getting the same issue too: #2990

[fabriziopandini]

@vincepri no this is not fixed.
clusterctl init requires all the variables to be in place, so it does when called with the --list-variables flag.

@wfernandes wdyt about making clusterctl init tolerate missing variable only when invoked with the --list-variables flag?

[fabriziopandini]

@kferrone this is slightly different from #2990; the former is fixed by v03.5, this one not

[vincepri]

/milestone v0.3.6

[wfernandes]

Yeah..this is interesting.

In other words, from a clusterctl PoV only a fully compiled YAML can be input for further processing, like e.g. inspecting images.

I agree with @fabriziopandini that ideally, we should be inspecting the images once a yaml is fully processed. Because if provider components can also be templated then the images themselves could be variables that require processing.

wdyt about making clusterctl init tolerate missing variable only when invoked with the --list-variables flag?

@fabriziopandini Did you mean --list-images flag for this specific issue?

[vincepri]

--list-images isn't really an option for init from what I can tell, does it makes sense to make it a separate command instead?

[wfernandes]

I believe --list-images is a flag for clusterctl init.
--list-variables is a flag for clusterctl config cluster.

  # Lists the container images required for initializing the management cluster.
  #
  # Note: This command is a dry-run; it won't perform any action other than printing to screen.
  clusterctl init --infrastructure aws --list-images

does it makes sense to make it a separate command instead

However, I've come across this exact statement before.

cluster-api/cmd/clusterctl/cmd/init.go

Lines 110 to 112 in 23a9ec3

	// TODO: Move this to a sub-command or similar, it shouldn't really be a flag.
	initCmd.Flags().BoolVar(&initOpts.listImages, "list-images", false,
	"Lists the container images required for initializing the management cluster (without actually installing the providers)")

IMO we should deprecate this flag entirely and not substitute it with anything.
If a user wants to know what images they are using when trying to
clusterctl init --infrastructure aws, they should be using
clusterctl config provider --infrastrucuture aws.
This will give them the information they need without having to specify the config variable AWS_B64ENCODED_CREDENTIALS.

[fabriziopandini]

IMO we should deprecate this flag entirely and not substitute it with anything.

The --list-images on clusterctl init is really helpful for the air-gapped scenario, because it gives you a list of all the required images in a single command.
If we drop this flag, the user will be required to call 4 times clusterctl config provider; additionally, the input of clusterctl config provider can't be easily piped to other commands, while the output of --list-images is designed for this.

Did you mean --list-images flag for this specific issue?

I mean that in case of clusterctl init called with --list-images, we can implicitly set skip-variables. In fact, we are already doing this for clusterctl config provider

[wfernandes]

@fabriziopandini Thanks for the context about the need for --list-images. TIL. 🙂

I'm fine with adding the skip variables bool in the call for InitImages. Did we want to move it to a separate sub-command or can we do that as part of a separate issue?

[fabriziopandini]

@wfernandes I'm in favor of keeping it simple and only changing the behavior of --list-images (without creating a new command).

But I defer this to @vincepri

[wfernandes]

/assign

[wfernandes]

/lifecycle active