k3s Binary is too big

[cjellick]

Recently the k3s binary grew pretty significantly from 60960768 bytes to 84463616 bytes. See:

SIZE	COMMIT	SUBJECT
84463616	a4b2953	add setup hook capabilities for rke2
60960768	5859f83	Merge pull request #2142 from brandond/fix_768

This is a problem because (according to @ibuildthecloud) the binary size correlates pretty closely to memory footprint.

This is the change that caused the size increase: #2146

Specifically, importing "github.com/rancher/k3s/pkg/daemons/config" in pkg/cli/cmds/server.go caused the k3s cli wrapper binary to pull in a ton of k8s dependencies.

When I say the k3s cli wrapper binary, I mean the one built on this line:
https://github.com/rancher/k3s/blob/master/scripts/package-cli#L59
Notice how that is building cmd/k3s/main.go - This program is a thin wrapper around other fatter binaries like k3s-server and k3s-agent. By wrapper I mean it execs out to those binaries, it does not compile in their code.

So, while those other binaries are already fat because they've pulled in all the k8s code, this wrapper binary hasn't. By importing "github.com/rancher/k3s/pkg/daemons/config", it transitively pulled in a bunch of k8s code and got really fat.

To fix this, we need to eliminate the import of github.com/rancher/k3s/pkg/daemons/config. I think the best way to do this is to define the function that is defined here:
https://github.com/rancher/k3s/blob/master/pkg/server/types.go#L15

StartupHooks     []func(context.Context, config.Control) 

Instead of having config.Control as a parameter, let's just change it to the two field off of that struct that are currently in use by the hooks we setup in rke2. These would be APIServerReady <-chan struct{}, KubeConfigAdmin string.

In addition to all this, let's also add a check in CI that will fail a PR i the binary size grows over 61000000 bytes

[liyimeng]

Great!

[rancher-max]

Validated binary pushed for recent commits and v1.19.1-rc1+k3s1 are all less than the acceptable size of 61000000 bytes

• CI should prevent any commits increasing binary size now as well