Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CVE-2022-0759 VULNERABILITY: Config defaults to VERIFY_NONE when kubeconfig doesn't specify custom CA - autoclosed #554

Closed
cben opened this issue Mar 23, 2022 · 10 comments
Labels
bug Mend: dependency security vulnerability Security vulnerability detected by WhiteSource

Comments

@cben
Copy link
Collaborator

cben commented Mar 23, 2022

this was assigned CVE-2022-0759

Dangerous bug present ever since Kubeclient::Config was created:

Whenever kubeconfig did not define custom CA (normal situation for production clusters with public domain and certificate!), Config was returning ssl_options[:verify_ssl] hard-coded to VERIFY_NONE :-(

Assuming you passed those ssl_options to Kubeclient::Client, this means that instead of checking server's certificate against your system CA store, it would accept ANY certificate, allowing easy man-in-the middle attacks.

This is especially dangerous with user/password or token credentials because MITM attacker could simply steal those credentials to the cluster and do anything you could do on the cluster.

Tightly related to #555, I'm fixing them together. This ticket is about the dangerous default, #555 is about inability to override it.

@cben cben added bug Mend: dependency security vulnerability Security vulnerability detected by WhiteSource labels Mar 23, 2022
@cben
Copy link
Collaborator Author

cben commented Mar 23, 2022

This was broken IN ALL RELEASES MADE BEFORE 2022. I'm releasing a 4.9.3 with fix today.
Anyone who needs backports to other versions, let me know

@cben
Copy link
Collaborator Author

cben commented Mar 23, 2022

@lnacshonredhat (or anyone else who knows): can you help describe the impact when using client-certificate auth? My partial understanding:

  • An attack on the app using kubeclient is still possible.

  • Attacker can still fool kubeclient by returning made-up responses, it doesn't need 2nd connection to real cluster (because the client doens't verify the servers's identity).

    • This way attacker may get from kubeclient content of any resources kubeclient writes, e.g. if kubeclient is used to create/update secrets those will leak ☹️.
  • However, the client cert's secret key is not sent, so attacker cannot steal the credentials for later use, only risks are during the active attack.

  • The server does verify client's identity! 😅
    What degree of man-in-the-middle attack is possible?

    • Obviously attacker pass through the data byte-for-byte both ways (kubeclient⭤MITM⭤real cluster), but that's what any internet router does, so attacker can't decrypt.
    • Can attacker relay the inner HTTP protocol unmodified AND decrypt it? Or does decryption require changing the certs in a way that will make the cluster reject the TLS connection?
    • Can can attacker execute any action on the cluster different from kubeclient requested?
    • Can attacker switch from exact relaying to modifying anything mid-way?

@cben
Copy link
Collaborator Author

cben commented Mar 23, 2022

✔️ 4.9.3 released on rubygems.

cben added a commit to cben/kubeclient that referenced this issue Mar 23, 2022
Tiny followup to ManageIQ#556.
Sorry for noise, had these locally but forgot to push before merging.
If I start backporting, CHANGELOG.md on master branch might not always be updated
with all backports (it SHOULD, but it will require separate merges to master).
So I prefer pointing to the vulnerability issue as the "source of truth".
Also, security impact will be better discussed on the issue.
cben added a commit that referenced this issue Mar 23, 2022
cben added a commit to cben/fluent-plugin-kubernetes-metrics that referenced this issue Mar 25, 2022
See ManageIQ/kubeclient#554, I fixed an embarrasing vulnerability in Kubeclient::Config — it could wrongly set `VERIFY_NONE`, allowing man-in-the-middle attacks and stealing cluster credentials 😳 
And I see this repo does use `Kubeclient::Config.read`.

kubeclient generally obeys SemVer, so upgrading 4.6.z to 4.9.z should be safe.  OTOH if you think upgrading is tricky, let us know on that kubeclient issue, we can backport the fix!

@harshit-splunk @rockb1017 I see several of fluent-plugin-* gems depend on '~> 4.6.0', and more than one are maintained by Splunk — I'm not going to send PRs to them all, please spread the word.
@cben
Copy link
Collaborator Author

cben commented Mar 25, 2022

Do we need more backports?
Found this way to graph what versions are getting pulled in practice over time, e.g. these graphs are over last two days: https://ui.honeycomb.io/ruby-together/datasets/rubygems.org/result/i7SSSySDaHJ
I see non-negligible downloads of 4.6.0 and 4.7.0 that do not upgrade automatically to 4.9.z. I also peeked at https://rubygems.org/gems/kubeclient/reverse_dependencies, and saw very few = dependencies, but quite a few dependencies that fix the minor version e.g. ~> 4.6.0.

So I'm thinking releasing 4.6.1 and 4.7.1 may be useful? (and then maybe 4.8.1 just to avoid situation where upgrading causes regression?)

I also see some really old versions like 1.1.4 still getting used. Somewhat silly, but the fix commit itself (without tests) is trivially backportable. How far should I go?

@cben cben changed the title VULNERABILITY: Config defaults to VERIFY_NONE when kubeconfig doesn't specify custom CA CVE-2022-0759 VULNERABILITY: Config defaults to VERIFY_NONE when kubeconfig doesn't specify custom CA Mar 27, 2022
cben added a commit to cben/eks_cli that referenced this issue Apr 12, 2022
Hi, I see your code uses `Kubeclient::Config.read("#{ENV['HOME']}/.kube/config")`.
4.9.3 fixed a severe issue in Config — ManageIQ/kubeclient#554

I did not test this but generally kubeclient obeys semver, AFAIK 4.9 remains backwards-compatible with 4.1.
cben added a commit to cben/beaker-gke that referenced this issue Apr 12, 2022
Hi, I see your code uses `Kubeclient::Config.read(ENV['KUBECONFIG'])`.
4.9.3 fixed a severe issue in Config, in some scenarios causing insecure VERIFY_NONE connections that may leak cluster credentials — ManageIQ/kubeclient#554

Your dependency range already allowed 4.9.3 but it's safer to disallow the older versions.
cben added a commit to cben/fog-kubevirt that referenced this issue May 11, 2022
4.9.3 fixed [CVE-2022-0759 in `Kubeclient::Config`](ManageIQ/kubeclient#554), which I see you do use, at least in `create_client_from_config`.
Current "~> 4.3" range already allows 4.9.x but safer to force it as minimum.
cben added a commit to cben/fog-kubevirt that referenced this issue May 11, 2022
4.9.3 fixed [CVE-2022-0759 in `Kubeclient::Config`](ManageIQ/kubeclient#554), which I see you do use, at least in `create_client_from_config`.
Current "~> 4.3" range already allows 4.9.x but safer to force it as minimum.
@mend-bolt-for-github
Copy link
Contributor

✔️ This issue was automatically closed by Mend because the vulnerable library in the specific branch(es) was either marked as ignored or it is no longer part of the Mend inventory.

@mend-bolt-for-github mend-bolt-for-github bot changed the title CVE-2022-0759 VULNERABILITY: Config defaults to VERIFY_NONE when kubeconfig doesn't specify custom CA CVE-2022-0759 VULNERABILITY: Config defaults to VERIFY_NONE when kubeconfig doesn't specify custom CA - autoclosed May 30, 2022
@cben
Copy link
Collaborator Author

cben commented May 30, 2022

@agrare is this "Mend" bot part of the standard setup in manageiq org?
IMHO a bot closing a vulnerability issue with a generic "either ... or ..." without specific details is more harmful than useful. In first case, where was it marked ignored, and by whom? In 2nd case, what is Mend's inventory, and why the library(?) being dropped from it justifies closing the library's github issue?


In this case, the issue has been fixed in a released version of kubeclient, so we can keep this closed.
4.9.2 to this day has much more downloads than 4.9.3 which is discouraging but I can't do anything about that... Probably lots of apps have a Gemfile.lock.

@agrare
Copy link
Member

agrare commented May 30, 2022

@cben I haven't seen the mend-bolt-for-github before, @Fryguy looks like this has something to do with whitesource scans?

@Fryguy
Copy link
Member

Fryguy commented Jun 6, 2022

Yes, mend is the new name for whitesource, so this is the whitesource bot - I'm as surprised as you are that they closed an issue they didn't own. I presume because it said CVE-2022-0759 in the title, but that's not a very good reason. I'm going to open a support ticket with them.

In this case, the issue has been fixed in a released version of kubeclient, so we can keep this closed.

We can reopen if you want...it's up to you

@zeisss
Copy link

zeisss commented Aug 30, 2022

Can we add this issue / release to the CHANGELOG.md?

Edit: Nevermind, I see that this is an older issue. I was trying to find our what changed in 4.10.0 ...

@cben
Copy link
Collaborator Author

cben commented Aug 31, 2022

Thes one i believe is covered in https://github.com/ManageIQ/kubeclient/blob/master/CHANGELOG.md#493-2021-03-23

4.10 - ah good catch, it's only described on v4.y branch:
https://github.com/ManageIQ/kubeclient/blob/v4.y/CHANGELOG.md#4100--2022-08-29
Let me fix that.... FIXED ✔️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Mend: dependency security vulnerability Security vulnerability detected by WhiteSource
Projects
None yet
Development

No branches or pull requests

4 participants