Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) #839

tstromberg · 2019-08-06T17:40:39Z

Hello!

We (minikube) would like to publish our .deb and .rpm packages to the official Kubernetes repositories as part of our release process. How do we get started?

Currently our release scripts publish to a Google Storage bucket:

https://github.com/kubernetes/minikube/blob/551b164017d2649a3857facb79ba80d00569fa18/hack/jenkins/release_build_and_upload.sh#L45

tstromberg · 2019-08-06T17:44:42Z

/cc @afbjorklund @justaugustus

afbjorklund · 2019-08-06T20:38:23Z

https://storage.googleapis.com/minikube/releases/v1.3.0/minikube_1.3.0.deb
https://storage.googleapis.com/minikube/releases/v1.3.0/docker-machine-driver-kvm2_1.3.0.deb

https://storage.googleapis.com/minikube/releases/v1.3.0/minikube-1.3.0.rpm
https://storage.googleapis.com/minikube/releases/v1.3.0/docker-machine-driver-kvm2-1.3.0.rpm

tstromberg · 2019-08-22T23:14:18Z

Clarifying title. I could use some help with getting the appropriate permissions. Alternatively, we could create our own repo, but it would be nice to preserve the maintainability benefits of having a single Kubernetes package repository.

tstromberg · 2019-09-20T18:55:38Z

/cc @justaugustus

Now that we are post-v1.16 release, does the release team have the cycles to help make this happen?

tstromberg · 2019-09-25T23:31:48Z

If #sig-release prefers, an alternative approach would be pulling files from our GCS or GitHub file stores. This would introduce a post-build race condition, but I'm happy with any solution so long as the deb files get published before we make public announcements.

medyagh · 2019-12-16T23:38:59Z

any update on this ?

afbjorklund · 2020-02-23T22:26:02Z

I guess there is no update on this, and it won't be possible to easily install minikube 1.8.0 ?

Only change since last time, was that we added the architecture to the package file names:

https://github.com/kubernetes/minikube/releases/download/v1.7.0/minikube_1.7.0-0_amd64.deb

https://github.com/kubernetes/minikube/releases/download/v1.7.0/minikube-1.7.0-0.x86_64.rpm

tstromberg · 2020-04-06T17:23:37Z

Is there anything I or others can do to help with the signing infrastructure? This issue has now been open for 244 days without a response on GitHub, though I did have a chance to briefly bring this up with @justaugustus at KubeCon in November.

Even if it takes a Googler to push a button to sign the key, that's fine with me to get this thing going.

justaugustus · 2020-04-06T17:51:23Z

From #sig-release Slack thread:

@tstromberg -- Sorry for letting this slip. The tl;dr:

SIG Release does not control the apt/yum repos that we publish packages to. Google does. So deb/rpm publishing that happens every cycle is done happens on the Google side.
The current package signing key is also Google-owned
Today, we don't have a process for enabling arbitrary Kubernetes projects to push to those repos
If we did, I would suggest using a signing key that doesn't imply SIG Release/Release Engineering is on the hook for issues in publication
We're planning to move Kubernetes to community-owned infra, including the package repos, but apt/yum repos are lower priority compared to other improvements we're trying to make

ref on the current work: #911
ref on future work: #913

Unfortunately, I don't have a good answer for you right now, given the other more pressing projects.

Potential solutions:

rolling your own
requesting a separate Google-owned repo for minikube

cc: @kubernetes/build-admins

afbjorklund · 2020-04-06T18:43:27Z

The main downside is that user would have to install two separate keys, but that seems to be the same situation for both solutions since if it was the same key used for new repo it could go into the main...

https://packages.cloud.google.com/apt/doc/apt-key.gpg
https://packages.cloud.google.com/yum/doc/yum-key.gpg

pub   2048R/A7317B0F 2015-04-03 [expired: 2018-04-02]
uid                  Google Cloud Packages Automatic Signing Key <[email protected]>

pub   2048R/BA07F4FB 2018-04-01 [expires: 2021-03-31]
uid                  Google Cloud Packages Automatic Signing Key <[email protected]>

So I guess minikube will have to invent some local process, that will be a part of the release build. And then find some hosting for both the (public) gpg keys as well as the rest of the apt/yum index files...

afbjorklund · 2020-04-06T19:06:03Z

The workaround is still the same as before, user will have to download and install unsigned packages:

https://github.com/kubernetes/minikube/releases/download/v1.9.2/minikube_1.9.2-0_amd64.deb

https://github.com/kubernetes/minikube/releases/download/v1.9.2/minikube-1.9.2-0.x86_64.rpm

Usually this leads to a warning from the web browser, and then another one from the pkg installer...

So in that sense it will be the same as the Mac and Win experience, outside their "walled gardens":

Apple Gatekeeper
Microsoft Smartscreen

They will also have to repeat the process for every minor version, since there is no update/upgrade.

afbjorklund · 2020-05-02T19:00:11Z

Now it is blocked in the Ubuntu Software as well, so users will need to use the Terminal...

https://en.wikipedia.org/wiki/Snapcraft (https://snapcraft.io/)

Still works in Fedora, but not sure for how long. It is possible that they will move to Flathub.

https://en.wikipedia.org/wiki/Flathub (https://flathub.org/)

It is getting harder and harder to distribute software, especially without being able to sign it.
We still have a workaround in the command-line and in the independent package managers*.

* Like Homebrew for Mac and Chocolatey for Win

Might have to find something similar for Linux as well...

medyagh · 2020-07-29T18:05:36Z

any update on this issue ?

afbjorklund · 2020-08-29T07:57:47Z

Can we close this ?

It's obviously not going anywhere, and minikube is not in of the upstream release or even a "supported" kubeadm distribution.

Potential solutions:

rolling your own

Until we have fixed the issues with all other package managers, I guess the users are back to curl and the GitHub releases ?

 curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
 sudo install minikube-linux-amd64 /usr/local/bin/minikube

tstromberg · 2020-09-22T19:41:42Z

I believe this is still something we should aim for, though ultimately, I agree that the goal is to ensure that the minikube .deb & .rpm packages are hosted somewhere that supports an automated update mechanism. If we have to create our own repo, it'll just take longer to achieve.

Shared infra for package hosting would be wonderful.

blueelvis · 2020-09-28T18:54:16Z

We also have the choice of removing the installations. Basically, removing all these installers and then just maintaining the script based installation like Anders has mentioned above. For Windows, we could switch exclusively to choco or PowerShell script.

afbjorklund · 2020-11-30T06:40:56Z

We also have the choice of removing the installations.

a.k.a. "give up"

Ultimately this comes down to how we want users to install and update minikube. Currently, the process is quite awful.
The benefit of the packages is that they are compressed and checksummed, without requiring too much from the user.

Currently using LinuxBrew is actually a nicer experience for doing the installation, than the system package manager...
This is because there are no minikube packages available from the system (Ubuntu) nor from the vendor (Kubernetes)

brew install minikube

We should probably just close these improvement issues...

And have the user use the command-line forever to install them, without the package signing and the automatic updates.
I used https://bintray.com for my own personal packages, but I'm not sure that we want to go there for the official releases.

For next release we want to start promoting the arm64 binaries and packages, in addition to the existing amd64 binaries.
This will only make the situation worse, since now that is one more step that the poor user will have to get involved in...

https://github.com/kubernetes/minikube/releases/download/v1.16.0/minikube_1.16.0-0_amd64.deb
https://github.com/kubernetes/minikube/releases/download/v1.16.0/minikube_1.16.0-0_arm64.deb

https://github.com/kubernetes/minikube/releases/download/v1.16.0/minikube-1.16.0-0.x86_64.rpm
https://github.com/kubernetes/minikube/releases/download/v1.16.0/minikube-1.16.0-0.aarch64.rpm

Using apt and yum would have fixed this too, automatically.

mgabeler-lee-6rs · 2020-11-30T22:31:58Z

We should probably just close these improvement issues...

It is not a good look to your users for a project of this scale and large scale backing to conclude it is unable to stand up a simple pair of apt & yum repositories. I understand that things are complicated, esp. when dealing with large enterprises & code signing systems there, but having a cloud hosting technology project give up on ... setting up cloud hosting is incredibly bad optics.

Currently using LinuxBrew is actually a nicer experience for doing the installation

Maybe if you already use LinuxBrew. If you don't, asking users to install a new package manager just to install your one package is still pretty awful, esp. when they may already be annoyed by having to have 3-4 other package managers already installed as part of their distro (apt/yum, snap, flatpak, ...)

I used https://bintray.com for my own personal packages, but I'm not sure that we want to go there for the official releases.

Lots of open source projects leverage that and similar services for this purpose. If getting things hooked up with the Google processes is going to be slow, using that as at least an intermediate solution seems completely reasonable to me as a user. My only question would be whether minikube fits within the 1TB/month download quota they have, but a quick skim through https://api.github.com/repos/kubernetes/minikube/releases suggests that this project would be well within those limits.

afbjorklund · 2020-12-07T21:24:16Z

Currently using LinuxBrew is actually a nicer experience for doing the installation

Maybe if you already use LinuxBrew

I meant that the experience for users using brew, is nicer than the experience for users of apt and yum.
This is mostly because they cannot just use "apt" or "yum", but have to download and use "dpkg" or "rpm".

If it had a reasonable amount of effort to add the repository and required keys, then it would compare...
Like https://kubernetes.io/docs/tasks/tools/install-kubectl/ when compared to brew install kubectl ?

Other than that, there is no way to compare - without getting minikube accepted into all the distributions.
But that has other implications, when it comes to update frequency and availability due to e.g. go versions

For the time being, promoting the binary as the default distribution method seems to be the "easiest".
Then we can dream of something different, like with https://apps.0install.net/kubernetes/minikube.xml

mgabeler-lee-6rs · 2020-12-07T21:33:49Z

I meant that the experience for users using brew, is nicer than the experience for users of apt and yum. This is mostly because they cannot just use "apt" or "yum", but have to download and use "dpkg" or "rpm".

This is, at best, an extremely misleading statement

apt uses dpkg under the hood. Nobody would ever have to "download" dpkg. You cannot remove dpkg from a Debian-based system
Similarly I don't think you can remove rpm from a RedHat-based system. yum fronts for rpm here AFAIK, much like apt fronts for dpkg
I don't know if you can use yum to directly install a downloaded .rpm file, but you can definitely use apt to install downloaded .deb files, benefitting from all of apt's dependency resolution logic. So, e.g., being a Debian user I typically update minikube via sudo apt install ~/Downloads/minikube-....deb

mgabeler-lee-6rs · 2020-12-07T21:37:31Z

If it had a reasonable amount of effort to add the repository and required keys, then it would compare...
Like https://kubernetes.io/docs/tasks/tools/install-kubectl/ when compared to brew install kubectl ?

Check out how Chrome, VS-Code, and many other "third party" apps do this. The package itself has a post-install script which handles adding the repo, and a crontab to ensure the repo config is kept up to date. The user only needs to download and install the package ones, and thereafter the OS-driven regular system updates keep it up to date. 🎉

afbjorklund · 2020-12-07T21:43:34Z

I meant that the experience for users using brew, is nicer than the experience for users of apt and yum. This is mostly because they cannot just use "apt" or "yum", but have to download and use "dpkg" or "rpm".

This is, at best, an extremely misleading statement

I meant that you have to 1) download the package file yourself and 2) run a command on the file.
Instead of just being able to install (and upgrade!), if we offered actual apt and yum repositories.

I think you are right that we could use apt install and yum localinstall instead of dpkg and rpm.
I mostly meant the two-step CLI process, unfortunately using "double-click" doesn't work anymore.

mgabeler-lee-6rs · 2020-12-07T21:48:51Z

I meant that you have to 1) download the package file yourself and 2) run a command on the file.

Most users will have a DE installed and they'll just be able to (double) click to open the downloaded package which will open it in a graphical package manager with a nice big "install" button, or so I would think. That's certainly how things work on Gnome-ish systems, I'm assuming KDE behaves similarly.

Instead of just being able to install (and upgrade!), if we offered actual apt and yum repositories.

Definitely agree this is preferred ... my earlier notes were more than the one-time install can configure the repo for a user automatically, so the complex steps you linked from the main kubernetes installation instructions are not necessarily something a user has to do themselves.

unfortunately using "double-click" doesn't work anymore.

Why? It works here. Granted I'm a terminal person so I don't use that capability, but it does work on my system. The .deb file opens with gnome-software which shows the package info (version, description, license) and a big friendly button to act on it ... in my case it's a big Remove button because I already have it installed, but if I didn't it'd be a big friendly Install button.

afbjorklund · 2020-12-07T21:50:25Z

The feature was disabled in Ubuntu 20.04, in order to promote the Ubuntu Software app store...

It used to work like you say. We have similar issues with the Mac and Win downloads not working.

mgabeler-lee-6rs · 2020-12-07T21:59:12Z

Aah, I wasn't aware of that change in Ubuntu. Seems rather user-hostile, which ... well that's part of why I use Debian :)

At any rate, if the package post-install script can setup the apt/yum repo, then the user would only need to do the CLI steps once, which would be a big improvement :)

afbjorklund · 2020-12-07T22:00:36Z

Aah, I wasn't aware of that change in Ubuntu. Seems rather user-hostile, which ... well that's part of why I use Debian :)

Getting the software accepted into Debian is a much longer journey, especially with the insane amount of dependencies.

For other distributions we get away with static linking and providing binaries, but that's just not the way that Debian rolls.

Not that I think that anyone actually tried (to add minikube) ?

mgabeler-lee-6rs · 2020-12-07T22:02:23Z

Getting the software accepted into Debian is a much longer journey, especially with the insane amount of dependencies.

Yes, was not suggesting trying to get this accepted into Debian, just that the type of user-hostile change you described from Ubuntu is why I don't use Ubuntu.

afbjorklund · 2020-12-07T22:07:21Z

... the type of user-hostile change you described from Ubuntu is why I don't use Ubuntu.

And of course the Snap wasn't updated either, because they were promoting Microk8s instead. 🙄

So in the end we ended up deleting it (put it out of its misery). See ~~kubernetes/minikube#7977~~

Which might be the way to "fix" the .deb and .rpm installations as well. Instead of fighting windmills.

The actual binary is the same either way, so we will just focus on producing those - "upstream".

fejta-bot · 2021-03-07T22:53:06Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale

afbjorklund · 2021-03-13T08:41:58Z

@tstromberg Can we close this ticket ? ("Wont Do")

We have called out for the web designers to make our workarounds look slightly less awful,
rather than hoping for support for any official packages. The old documentation was removed*.

https://minikube.sigs.k8s.io/docs/start/

https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb
https://storage.googleapis.com/minikube/releases/latest/minikube-latest.x86_64.rpm

https://storage.googleapis.com/minikube/releases/latest/minikube_latest_arm64.deb
https://storage.googleapis.com/minikube/releases/latest/minikube-latest.aarch64.rpm

BTW; Bintray is closing down services (May 1, 2021).

* The installation has been removed from the main docs:

~~https://v1-18.docs.kubernetes.io/docs/tasks/tools/install-minikube/~~
~~https://kubernetes.io/docs/setup/learning-environment/~~

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#installing-kubeadm-kubelet-and-kubectl

https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/
https://kubernetes.io/docs/tasks/tools/install-kubectl-windows/

tstromberg · 2021-04-08T14:00:11Z

While I still feel like minikube (as well as kubectl or other command-line utilities) should have an apt/yum repository to support automatic upgrades, it's clear that this request isn't going anywhere, so I'm going to close it for now.

I believe the right path going forward will be for minikube to establish its own repository.

dims · 2021-04-08T14:05:46Z

@tstromberg if you want to use any GCP assets for your own repository, do please ask in kubernetes/k8s.io and we'll figure out a way to support you.

justaugustus · 2022-03-21T23:12:18Z

Another request: https://kubernetes.slack.com/archives/CJH2GBF7Y/p1647903600618439

afbjorklund · 2022-03-22T07:17:26Z

@klaases we already gave up on this

This was referenced Aug 6, 2019

Make minikube (and kvm2) installable with "apt" kubernetes/minikube#3110

Open

Make minikube (and kvm2) installable with "yum" kubernetes/minikube#4716

Open

tstromberg changed the title ~~Add minikube packages (.deb, .rpm)~~ Help with adding minikube packages to Kubernetes repos (.deb, .rpm) Aug 22, 2019

tstromberg changed the title ~~Help with adding minikube packages to Kubernetes repos (.deb, .rpm)~~ Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) Aug 22, 2019

justaugustus added sig/release Categorizes an issue or PR as relevant to SIG Release. area/release-eng Issues or PRs related to the Release Engineering subproject labels Dec 9, 2019

This was referenced Apr 6, 2020

Sign minikube binaries for macOS kubernetes/minikube#5792

Open

fix windows packing installer alert: unknown publisher kubernetes/minikube#6735

Closed

afbjorklund mentioned this issue May 3, 2020

Update the Zero Install feed for Minikube kubernetes/minikube#7978

Closed

xmudrii mentioned this issue May 11, 2020

Review of the Release Engineering backlog kubernetes/sig-release#1069

Closed

7 tasks

afbjorklund mentioned this issue Sep 3, 2020

Add install minikube with homebrew on Linux kubernetes/minikube#9066

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Mar 7, 2021

tstromberg closed this as completed Apr 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) #839

Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) #839

tstromberg commented Aug 6, 2019

tstromberg commented Aug 6, 2019

afbjorklund commented Aug 6, 2019

tstromberg commented Aug 22, 2019

tstromberg commented Sep 20, 2019

tstromberg commented Sep 25, 2019

medyagh commented Dec 16, 2019

afbjorklund commented Feb 23, 2020

tstromberg commented Apr 6, 2020

justaugustus commented Apr 6, 2020

afbjorklund commented Apr 6, 2020

afbjorklund commented Apr 6, 2020

afbjorklund commented May 2, 2020

medyagh commented Jul 29, 2020

afbjorklund commented Aug 29, 2020

tstromberg commented Sep 22, 2020 •

edited

Loading

blueelvis commented Sep 28, 2020

afbjorklund commented Nov 30, 2020

mgabeler-lee-6rs commented Nov 30, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020 •

edited

Loading

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020 •

edited

Loading

fejta-bot commented Mar 7, 2021

afbjorklund commented Mar 13, 2021 •

edited

Loading

tstromberg commented Apr 8, 2021

dims commented Apr 8, 2021

justaugustus commented Mar 21, 2022

afbjorklund commented Mar 22, 2022

Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) #839

Permissions to add minikube packages to Kubernetes repos (.deb, .rpm) #839

Comments

tstromberg commented Aug 6, 2019

tstromberg commented Aug 6, 2019

afbjorklund commented Aug 6, 2019

tstromberg commented Aug 22, 2019

tstromberg commented Sep 20, 2019

tstromberg commented Sep 25, 2019

medyagh commented Dec 16, 2019

afbjorklund commented Feb 23, 2020

tstromberg commented Apr 6, 2020

justaugustus commented Apr 6, 2020

afbjorklund commented Apr 6, 2020

afbjorklund commented Apr 6, 2020

afbjorklund commented May 2, 2020

medyagh commented Jul 29, 2020

afbjorklund commented Aug 29, 2020

tstromberg commented Sep 22, 2020 • edited Loading

blueelvis commented Sep 28, 2020

afbjorklund commented Nov 30, 2020

mgabeler-lee-6rs commented Nov 30, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020 • edited Loading

mgabeler-lee-6rs commented Dec 7, 2020

afbjorklund commented Dec 7, 2020 • edited Loading

fejta-bot commented Mar 7, 2021

afbjorklund commented Mar 13, 2021 • edited Loading

tstromberg commented Apr 8, 2021

dims commented Apr 8, 2021

justaugustus commented Mar 21, 2022

afbjorklund commented Mar 22, 2022

tstromberg commented Sep 22, 2020 •

edited

Loading

afbjorklund commented Dec 7, 2020 •

edited

Loading

afbjorklund commented Dec 7, 2020 •

edited

Loading

afbjorklund commented Mar 13, 2021 •

edited

Loading