-
Notifications
You must be signed in to change notification settings - Fork 425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Any plans to support binary releases for controller-gen
?
#500
Comments
In metal3 we ended up creating a script to install controller-gen so we could create a temporary directory, run We would happily consume binary releases, too, if they existed. |
Sweet. Consider that script copied ;) Thanks @dhellmann! |
To give credit where due, I'm pretty sure that the approach in the script actually came from the commands kubebuilder put in the Makefile it generated for us. We moved it to a script as part of hacking it to ensure it always installed exactly the version we wanted, without overwriting a version the user may have already had in $GOBIN. |
We need to ensure that all ACK developers are using the same controller-gen version otherwise we get into situations where: 1) go.mod/go.sum change unnecessarily 2) The annotations for things like CRDs will be different for generated YAML manifests This patch makes v0.4.0 of the controller-tools repo and controller-gen binary our target version. It changes the `ensure_controller_gen` Bash function to ensure that controller-gen at that version is installed and refuses to proceed with building controllers if there is a different version. ``` jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ controller-gen --version Version: v0.3.1-0.20200716001835-4a903ddb7005 jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:52:56Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb FAIL: Existing version of controller-gen Version: v0.3.1-0.20200716001835-4a903ddb7005, required version is v0.4.0. FAIL: Please uninstall controller-gen and re-run this script, which will install the required version. make[1]: *** [Makefile:49: build-controller] Error 1 make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ sudo rm -f `which controller-gen` jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:53:07Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.4.0 **************************************************************************** WARNING: You may need to reload your Bash shell and path. If you see an error like this following: Error: couldn't find github.com/aws/aws-sdk-go in the go.mod require block simply reload your Bash shell with `exec bash` and then re-run whichever command you were running. **************************************************************************** Building Kubernetes API objects for dynamodb Error: couldn't find github.com/aws/aws-sdk-go in the go.mod require block make[1]: *** [Makefile:49: build-controller] Error 2 make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ exec bash jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:53:21Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb Building Kubernetes API objects for dynamodb Generating deepcopy code for dynamodb Generating custom resource definitions for dynamodb Building service controller for dynamodb Generating RBAC manifests for dynamodb Running gofmt against generated code for dynamodb make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' ``` Issue aws-controllers-k8s#349 Related: kubernetes-sigs/controller-tools#500
We need to ensure that all ACK developers are using the same controller-gen version otherwise we get into situations where: 1) go.mod/go.sum change unnecessarily 2) The annotations for things like CRDs will be different for generated YAML manifests This patch makes v0.4.0 of the controller-tools repo and controller-gen binary our target version. It changes the `ensure_controller_gen` Bash function to ensure that controller-gen at that version is installed and refuses to proceed with building controllers if there is a different version. ``` jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ controller-gen --version Version: v0.3.1-0.20200716001835-4a903ddb7005 jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:52:56Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb FAIL: Existing version of controller-gen Version: v0.3.1-0.20200716001835-4a903ddb7005, required version is v0.4.0. FAIL: Please uninstall controller-gen and re-run this script, which will install the required version. make[1]: *** [Makefile:49: build-controller] Error 1 make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ sudo rm -f `which controller-gen` jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:53:07Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb go: creating new go.mod: module tmp go: found sigs.k8s.io/controller-tools/cmd/controller-gen in sigs.k8s.io/controller-tools v0.4.0 **************************************************************************** WARNING: You may need to reload your Bash shell and path. If you see an error like this following: Error: couldn't find github.com/aws/aws-sdk-go in the go.mod require block simply reload your Bash shell with `exec bash` and then re-run whichever command you were running. **************************************************************************** Building Kubernetes API objects for dynamodb Error: couldn't find github.com/aws/aws-sdk-go in the go.mod require block make[1]: *** [Makefile:49: build-controller] Error 2 make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ exec bash jaypipes@thelio:~/go/src/github.com/aws/aws-controllers-k8s$ make build-controller SERVICE=dynamodb make[1]: Entering directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' go build -tags codegen -ldflags "-X main.version="v0.0.0" -X main.buildHash=598a3e29bb514d98660d04a088641922ccc020c9 -X main.buildDate=2020-10-05T16:53:21Z" -o bin/ack-generate cmd/ack-generate/main.go ./scripts/build-controller.sh dynamodb Building Kubernetes API objects for dynamodb Generating deepcopy code for dynamodb Generating custom resource definitions for dynamodb Building service controller for dynamodb Generating RBAC manifests for dynamodb Running gofmt against generated code for dynamodb make[1]: Leaving directory '/home/jaypipes/go/src/github.com/aws/aws-controllers-k8s' ``` Issue aws-controllers-k8s#349 Related: kubernetes-sigs/controller-tools#500
Bumping this. Perhaps tarring all 3 binaries by platform so the release artifact count doesn't blow up, ex. $ tar --list -f controller-tools_linux_amd64.tar.gz
controller-gen
helpgen
type-scaffold |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-contributor-experience at kubernetes/community. |
/remove-lifecycle stale |
Another reason for having this feature is the use of controller-tools behind a company firewall which rate limits/throttles downloads to Github and other sources. It sometimes can take an hour to get the controller tools loaded on a box for doing basic development. We started pre-packaging it into containers so we only have to download once; we're also looking at deploying a Go Proxy to cache them as well. Other methods involve creating a vendor directory on a local git repo and then installing the controller binary. These all seem like tedious workarounds to not having an offline installer binary. |
Vote +1, the binary releases can save more time than |
Issues go stale after 90d of inactivity. If this issue is safe to close now please do so with Send feedback to sig-contributor-experience at kubernetes/community. |
/remove-lifecycle stale |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close |
@k8s-triage-robot: Closing this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen |
@jaypipes: Reopened this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/remove-lifecycle rotten |
I've been attempting to install controller-gen (and conversion-gen from k8s code-generator) in a Docker container in order to standardize my organization's use of Unfortunately, no matter how much I try, I simply cannot get
Here's the Dockerfile I'm using:
I've spent hours trying to get this working and am kind of at the end of my rope. Wondering if @dhellmann or anyone else may have been able to solve this dilemma? Certainly having binary releases of controller-gen would (I think) make life a whole lot easier, no? |
In looking for a reason why that output is showing up when running |
Note that if I don't use a multi-stage Docker build and instead remove these lines from the Docker file:
I do get a working |
(Long-shot disclaimer) The fact that the command is being run in
Is it possible that directory is being picked up as the go executable by the |
@jaypipes did you already try |
No, I had no idea about that... I can give it a try. |
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten |
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues according to the following rules:
You can:
Please send feedback to sig-contributor-experience at kubernetes/community. /close not-planned |
@k8s-triage-robot: Closing this issue, marking it as "Not Planned". In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/reopen /remove-lifecycle rotten /assign |
@sbueringer: Reopened this issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Hi, just stopping by and sharing an antecdote. I'm new to contributing to kubernetes projects (currently interested in kubernetes-sigs/external-dns) Spinning up a local dev environment and I'm going through the getting started guide. https://kubernetes-sigs.github.io/external-dns/v0.14.2/contributing/getting-started/ Step 2 is
So I search the regular places. pamac, brew, github releases, no matches. So I search the issue trackers. Devs have been struggling with this issue since at least 2020. There's a documentation PR from 2021 which unfortunately never got merged. Then I find this issue where a suggested workaround https://github.com/metal3-io/baremetal-operator/blob/main/hack/install-controller-gen.sh 404s. I'm blocked and I'm frustrated. Granted, there is a solution in the PR at https://github.com/kubernetes-sigs/controller-tools/pull/537/files but I'll be honest. I don't understand golang's GOPATH/GO111MODULE yet. I ran those commands and I still don't have a controller-tools. I wish there were a controller-gen binary. |
Hi @insanity54, sorry that you are frustrated, I understand how you feel. Unfortunately, I don't think the project is ever going to release binaries for controller-gen. Here is what I end up doing for most of my projects (since many projects I work on require different versions of
and then I refer to the
and call the binary:
|
Once I get that for down my TODO list it will happen :) (can't promise an ETA though, just too much going on) |
Finally found the time to implement it. Starting with the next CT release we will publish controller-gen binaries. Now it's just up to folks consuming the binaries from release attachments |
@sbueringer you rock, mate! :) thank you so much! |
I did a quick check and everything looks fine, but if you (or someone else) wants to try them them out: https://github.com/kubernetes-sigs/controller-tools/releases/tag/v0.16.0-beta.0 |
First, want to say thanks for the great work on the controller-tools repo. It's generally awesome. :)
I work on a project (github.com/aws/aws-controllers-k8s) that makes heavy use of
controller-gen
to augment our own code generation. However, because there are no binary releases ofcontroller-gen
we've seen one issue crop up.Since the only way to use the controller-gen tool is to "install" it with
go get
, if a contributor to our project doesn't havecontroller-tools
locally and they rungo get
to install it, that invariably ends up modifying thego.mod/sum
file in our source repository. These are// indirect
entries in thego.mod
but still, it's kind of annoying to have to tell contributors to undo the changes to theirgo.mod/sum
files due to this.I was wondering if there are any plans to produce binary artifacts for the
controller-gen
tool? This would certainly make our lives a bit easier on the downstream consumer side of things.If not binary artifacts, perhaps publishing Docker images containing pre-built
controller-gen
binaries?The text was updated successfully, but these errors were encountered: