Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Native Security Intro #14495

Merged
merged 6 commits into from
May 30, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions content/en/docs/concepts/security/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: "Security"
weight: 81
---
153 changes: 153 additions & 0 deletions content/en/docs/concepts/security/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
---
reviewers:
- zparnold
title: Overview of Cloud Native Security
content_template: templates/concept
weight: 1
---

{{< toc >}}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @zparnold. I think the content is a great start. Do you have an outline of topics to be covered for k8s security? How does this introduction page fit with the outline?
I have a few nits too 😃,

  • Do you need the toc shortcode?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I don't think so @kbhawkey because our goal is to organize and enhance the content by providing a landing page (here) for it...so it is a concept, but I don't know how much more there's going to be beyond that

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • OK. I am just suggesting that a project listing the proposed content issues would be useful, otherwise, it is hard to tell how this page fits with the overall plan. Security is a big topic.
  • I thought most of the concept files removed the toc shortcode and just used the headings to generated the toc.

{{% capture overview %}}
Kubernetes Security (and security in general) is an immense topic that has many
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My ideal: sketch out a framework for security that one day grows into a broader scope, eg “cloud-native information security”

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Is Kubernetes Security a concept in itself?
  • The comment in parens does not add much
  • immense is not a great descriptor

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kbhawkey what would you suggest as a better descriptor?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Kubernetes security vs Kubernetes Security
  • We know security is a big, important, complicated topic. What makes security challenging with cloud native? Since this is a single page, I think it would be helpful to declare/define the categories most important to k8s security -- you begin to do that with the 4C diagram. What does k8s offer today, by way of security? What else is needed?

There have been a number of blog posts about security,
https://www.cncf.io/blog/2019/01/14/9-kubernetes-security-best-practices-everyone-must-follow/

highly interrelated parts. In today's era where open source software is
integrated into many of the systems that help web applications run,
there are some overarching concepts that can help guide your intuition about how you can
think about security holistically. This guide will define a mental model for
for some general concepts surrounding Cloud Native Security. The mental model is completely arbitrary
and you should only use it if it helps you think about where to secure your software
stack.
{{% /capture %}}

{{% capture body %}}

## The 4C's of Cloud Native Security
Let's start with a diagram that may help you understand how you can think about security in layers.
{{< note >}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check your use of Let's and we

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is The 4Cs of Cloud Native Security an accepted concept by the cloud native security community? I am challenging the use of we.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a member of the CNCF sig-security, while I never claim to speak on behalf of them, I believe that it is accepted.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, though, do you want to make a note that the CNCF SIG-security endorses or supports ...
It sounded vague, we think about security in layers

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, so let me think about it

This layered approach augments the [defense in depth](https://en.wikipedia.org/wiki/Defense_in_depth_(computing))
approach to security, which is widely regarded as a best practice for securing
software systems. The 4C's are Cloud, Clusters, Containers, and Code.
{{< /note >}}

{{< figure src="/images/docs/4c.png" title="The 4C's of Cloud Native Security" >}}


As you can see from the above figure,
each one of the 4C's depend on the security of the squares in which they fit. It
is nearly impossibly to safeguard against poor security standards in Cloud, Containers, and Code
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • do you need increasingly smaller areas of focus? The components are nested with levels of security increasing as you move to the outermost square?
  • remove double of

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, got it, thanks

by only addressing security at the code level. However, when these areas are dealt
with appropriately, then adding security to your code augments an already strong
base. These areas of concern will now be described in more detail below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reword: Each of the areas ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

## Cloud

In many ways, the Cloud (or co-located servers, or the corporate datacenter) is the
[trusted computing base](https://en.wikipedia.org/wiki/Trusted_computing_base)
Copy link
Contributor

@sftim sftim May 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a separate issue (and, later, documentation page), maybe it's possible to document the Kubernetes trusted computing base for a working cluster.

A page about Kubernetes' TCB could complement this one about principles.

Suggested TCB elements
  • API server
  • Security-critical controllers running in the control plane
  • Cluster DNS
  • Container runtime(s)
    • Assuming the runtime relies on Linux security properties, the kernel and its TCB
  • Node system userland tools (including kubelet and kube-proxy), because:
    • these able to run outside a Pod
    • these tools need to protect the nodes from rogue Pods. A node that allowed host-based rlogin from anywhere should have no place in a secure cluster design

How does that sound?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it, I'm gonna pin this comment for the next iteration and file it as an issue against this page.

of a Kubernetes cluster. If these components themselves are vulnerable (or
configured in a vulnerable way) then there's no real way to guarantee the security
of any components built on top of this base. Each cloud provider has extensive
security recommendations they make to their customers on how to run workloads securely
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

numerous vs detailed or extensive?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like extensive. Thanks!

in their environment. It is out of the scope of this guide to give recommendations
on cloud security since every cloud provider and workload is different. Here are some
links to some of the popular cloud providers' documentation
for security as well as give general guidance for securing the infrastructure that
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this area be reworded: ... popular cloud providers' [own] security documentation ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded

makes up a Kubernetes cluster.

### Cloud Provider Security Table



IaaS Provider | Link |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could these go in a whatsnext block?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what that is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-------------------- | ------------ |
Alibaba Cloud | https://www.alibabacloud.com/trust-center |
Amazon Web Services | https://aws.amazon.com/security/ |
Google Cloud Platform | https://cloud.google.com/security/ |
IBM Cloud | https://www.ibm.com/cloud/security |
Microsoft Azure | https://docs.microsoft.com/en-us/azure/security/azure-security |
VMWare VSphere | https://www.vmware.com/security/hardening-guides.html |


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are running on your own hardware or a different cloud provider you will need to
consult your documentation for security best practices.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we capture some overarching principles, ones that we think any cluster operator should have regard to?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, below :)


### General Infrastructure Guidance Table

Area of Concern for Kubernetes Infrastructure | Recommendation |
--------------------------------------------- | ------------ |
Network access to API Server (Masters) | Ideally all access to the Kubernetes Masters is not allowed publicly on the internet and is controlled by network access control lists restricted to the set of IP addresses needed to administer the cluster.|
Network access to Nodes (Worker Servers) | Nodes should be configured to _only_ accept connections (via network access control lists) from the masters on the specified ports, and accept connections for services in Kubernetes of type NodePort and LoadBalancer. If possible, this nodes should not exposed on the public internet entirely.
Kubernetes access to Cloud Provider API | Each cloud provider will need to grant a different set of permissions to the Kubernetes Masters and Nodes, so this recommendation will be more generic. It is best to provide the cluster with cloud provider access that follows the [principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege) for the resources it needs to administer. An example for Kops in AWS can be found here: https://github.com/kubernetes/kops/blob/master/docs/iam_roles.md#iam-roles
Access to etcd | Access to etcd (the datastore of Kubernetes) should be limited to the masters only. Depending on your configuration you should also attempt to use etcd over TLS. More info can be found here: https://github.com/etcd-io/etcd/tree/master/Documentation#security
etcd Encryption | Wherever possible it's a good practice to encrypt all drives at rest, but since etcd holds the state of the entire cluster (including Secrets) its disk should especially be encrypted at rest.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • etcd vs any datastore?
  • be consistent in using markdown style links in the table

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

etcd is the datastore of kubernetes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call on styling


## Cluster

This section will provide links for securing
workloads in Kubernetes. There are two areas of concern for securing
Kubernetes:

* Securing the components that are configurable which make up the cluster
* Securing the components which run in the cluster

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

italics needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to highlight the difference, deleting


### Components _of_ the Cluster
For more information on securing the components of the cluster, [here is a link](/docs/tasks/administer-cluster/securing-a-cluster) to the associated documention.

### Components _in_ the Cluster (your application)
Depending on the attack surface of your application, you may want to focus on specific
aspects of security. For example, if you are running a service (Service A) that is critical
in a chain of other resources and a separate workload (Service B) which is
vulnerable to a resource exhaustion attack, by not putting resource limits on
Service B you run the risk of also compromising Service A. Below is a table of
links of things to consider when securing workloads running in Kubernetes.

Area of Concern for Workload Security | Recommendation |
------------------------------ | ------------ |
RBAC Authorization (Access to the Kubernetes API) | https://kubernetes.io/docs/reference/access-authn-authz/rbac/
Authentication | https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both of these links are referenced indirectly on line 92 above (assuming the securing-a-cluster link is what we plan to link to).

As a reader, I wonder why RBAC is mentioned in both places? Maybe it's worth calling out explicitly that RBAC plays important roles in securing the cluster (e.g., the API itself, authorization between components with service accounts) and securing things within the deployment (e.g., pods, namespaces). At first glance, a reader might interpret RBAC and authorization as a tool exclusively for only accomplishing one of those goals.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is definitely a goal. This page however is the jumping off point. We will be re-organizing a good chunk of the kubernetes documentation over the lifecycle of our working group.

Application secrets management (and encrypting them in etcd at rest) | https://kubernetes.io/docs/concepts/configuration/secret/ <br> https://kubernetes.io/docs/tasks/administer-cluster/encrypt-data/
Pod Security Policies | https://kubernetes.io/docs/concepts/policy/pod-security-policy/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Again, etcd is specifically called out
  • Clean up the <br?
  • Do you need to write out the links, as they are internal to the k8s docs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

etcd is the brains of Kubernetes. You must use it in order to use Kubernetes. The <br> tag is there to force hugo's BlackFriday markdown engine to separate the links so they are more readable to an end user

Quality of Service (and Cluster resource management) | https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/
Network Policies | https://kubernetes.io/docs/concepts/services-networking/network-policies/
TLS For Kubernetes Ingress | https://kubernetes.io/docs/concepts/services-networking/ingress/#tls



## Container

In order to run software in Kubernetes, it must be in a container. Because of this,
there are certain security considerations that must be taken into account in order
to benefit from the workload security primitives of Kubernetes. Container security
is also outside the scope of this guide, but here is a table of general
recommendations and links for further exploration of this topic.

Area of Concern for Containers | Recommendation |
------------------------------ | ------------ |
Container Vulnerability Scanning and OS Dependency Security | As part of an image build step or on a regular basis you should scan your containers for known vulnerabilities with a tool such as [CoreOS's Clair](https://github.com/coreos/clair/)
Image Signing and Enforcement | Two other CNCF Projects (TUF and Notary) are useful tools for signing container images and maintaining a system of trust for the content of your containers. If you use Docker, it is built in to the Docker Engine as [Docker Content Trust](https://docs.docker.com/engine/security/trust/content_trust/). On the enforcement piece, [IBM's Portieris](https://github.com/IBM/portieris) project is a tool that runs as a Kubernetes Dynamic Admission Controller to ensure that images are properly signed via Notary before being admitted to the Cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about suggesting specific tools; documenting generalized approaches seem easier to maintain

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are, but this is the community accepted tool, so I don't have any other pointers outward except possibly OPA

Disallow privileged users | When constructing containers, consult your documentation for how to create users inside of the containers that have the least level of operating system privilege necessary in order to carry out the goal of the container.

## Code

Finally moving down into the application code level, this is one of the primary attack
surfaces over which you have the most control. This is also outside of the scope
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clean up: but we recommend a few of the following [what]?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cleaned up

of Kubernetes but here are a few recommendations:

### General Code Security Guidance Table

Area of Concern for Code | Recommendation |
--------------------------------------------- | ------------ |
Access over TLS only | If your code needs to communicate via TCP, ideally it would be performing a TLS handshake with the client ahead of time. With the exception of a few cases, the default behavior should be to encrypt everything in transit. Going one step further, even "behind the firewall" in our VPC's it's still a good idea to encrypt network traffic between services. This can be done through a process known as mutual or [mTLS](https://en.wikipedia.org/wiki/Mutual_authentication) which performs a two sided verification of communication between two certificate holding services. There are numerous tools that can be used to accomplish this in Kubernetes such as [Linkerd](https://linkerd.io/) and [Istio](https://istio.io/). |
Limiting port ranges of communication | This recommendation may be a bit self-explanatory, but wherever possible you should only expose the ports on your service that are absolutely essential for communication or metric gathering. |
3rd Party Dependency Security | Since our applications tend to have dependencies outside of our own codebases, it is a good practice to ensure that a regular scan of the code's dependencies are still secure with no CVE's currently filed against them. Each language has a tool for performing this check automatically. |
Static Code Analysis | Most languages provide a way for a snippet of code to be analyzed for any potentially unsafe coding practices. Whenever possible you should perform checks using automated tooling that can scan codebases for common security errors. Some of the tools can be found here: https://www.owasp.org/index.php/Source_Code_Analysis_Tools |
Dynamic probing attacks | There are a few automated tools that are able to be run against your service to try some of the well known attacks that commonly befall services. These include SQL injection, CSRF, and XSS. One of the most popular dynamic analysis tools is the OWASP Zed Attack proxy https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project |


## Robust automation

Most of the above mentioned suggestions can actually be automated in your code
delivery pipeline as part of a series of checks in security. To learn about a
more "Continuous Hacking" approach to software delivery, [this article](https://thenewstack.io/beyond-ci-cd-how-continuous-hacking-of-docker-containers-and-pipeline-driven-security-keeps-ygrene-secure/) provides more detail.

{{% /capture %}}
Binary file added static/images/docs/4c.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.