Merge pull request #125 from JustinCappos/securityaudit

First cut at security audit guidelines

assessments/README.md

@@ -0,0 +1,61 @@
+# Security Assessments
+
+## Goals
+The [security assessment process](guide) is designed to accelerate the adoption
+of cloud native technologies, based on the following goals and assumptions:
+
+### 1) Reduce risk across the ecosystem
+
+The primary goal is to reduce the risk from malicious attacks and accidental breaches of privacy. This process supports that goal in two ways:
+
+   * Clear and consistent process for communication increases detection &
+   reduces time to resolve known or suspected vulnerability issues
+   * A collaborative evaluation process increases domain expertise
+   within each participating project.
+
+### 2) Accelerate adoption of cloud native technologies
+
+Security reviews are a necessary, yet time consuming process, where each
+company, organization and project must perform its own reviews to ensure
+that it meets its unique commitments to its own users and stakeholders.
+In open source, simply finding security-related information can be a very
+time consuming part of the the process. The process is designed to enable improved discovery of security information & streamlined security reviews in multiple ways:
+
+   * Consistent documentation reduces review time
+   * Established baseline of security-relevant information reduces Q&A
+   * Clear rubric for security profile enables organizations to align their
+   risk profile with project’s risk profile and effectively allocate resources
+   (for review and needed project contribution)
+   * Structured metadata allows for navigation, grouping and cross-linking
+
+We expect that this process will raise awareness of how specific open source
+projects affect the security of a cloud native system; however, separate
+activities may be needed to achieve that purpose using materials generated by
+the assessements.
+
+## Outcome
+
+Each project assessment will:
+1. ensure a clear description of the project's design goals with respect to
+security
+2. uncover design flaws and document known limitations
+3. document next steps toward increasing security of the project itself and/or
+increasing the applications of the project toward increasing security of the
+cloud native ecosystem
+
+Due to the nature and timeframe for the analysis, *this review is not meant
+to subsume the need for a professional security audit of the code*.  Audits
+of implementation vulnerabilities and similar issues at that level are not
+intended to be covered by this assessment.  The purpose of this effort is to
+uncover design flaws and to obtain a clear articulation of what the project's
+design goals and security properties are intended to be.
+
+## Process
+
+The project assessment is intended to be a collaborative process for
+the benefit of the project and the community, where the primary content is
+generated by the [project lead](project-lead.md) and revised based on feedback
+from [security reviewers](security-reviewer.md) and other members of the
+working group (WG).
+
+See [security assessment guide](guide) for more details.

assessments/guide/README.md

@@ -0,0 +1,46 @@
+To provide the CNCF’s TOC with effective
+information about the security of different projects,
+this document outlines the procedure by which a project should be evaluated.
+
+## Roles
+
+* [project lead](project-lead.md)
+* [security reviewers](security-reviewer.md)
+
+## Steps
+
+1. Create tracking issue
+   * Issue may be a request from TOC liason or project itself
+2. Project provides self-assessment
+   * [project lead](project-lead.md) responds to the issue with draft document (see [outline](outline.md))
+   * issue assigned to lead [security reviewer](security-reviewer.md) who
+   will recruit a second reviewer, if one not already assigned, and facilitate
+   the process
+3. Inital review
+   * Upon request by the project, security reviewer may do an inital review to
+   verify completeness, ask for clarifications and provide quick feedback
+   * Project posts their document to the group mailing list, allowing at least
+   one week for review before presenting to the WG
+4. Presentation
+   * Project lead presents to WG
+   * This will be recorded as part of standard WG process
+5. Final assessment
+   * Reviewers provide final feedback with recommendations
+   * Either project lead or reviewers may request further WG discussion
+   * Project lead prepares a PR to /assessments/project-docs/project-name/
+
+## Additional Process Notes
+
+Iteration is expected; however, we expect quick turnaround (at most a week).
+In rare cases unrelated issues can unexpectedly interrupt the process and
+it may be appropriate to address specific concerns rather than continuing with
+the assessment. We encourage open communication between project lead and s
+ecurity reviewers:
+
+* At any time, the project lead may request additional time to respond
+  to feedback from security reviewers
+* Project lead or lead security reviewer may pause the process where a delay
+  of over a week cannot be accomodated by the review team. Simply close
+  the github issue with a note to SIG co-chairs.
+
+

assessments/guide/outline.md

@@ -0,0 +1,114 @@
+# Outline
+
+First of all, the burden is primarily on the proposing project to
+demonstrate it is secure in a manner that is understandable to the broader
+community.  The [reviewers](security-reviewer.md) will help to assess and probe the design.
+
+The proposing project must provide a written document that describes the
+project and its security.  The document must contain the following
+information, at a minimum. Where security considerations do not fit into
+the outline below, if possible, add a sub-section when possible such that the
+additional content conforms to the general flow of the review.
+
+
+## Metadata
+A table at the top for quick reference information, later used for indexing.
+
+|   |  |
+| -- | -- |
+| Software | A link to the software’s repository.
+| Security Provider | Yes or No. Is the primary function of the project to support the security of an integrating system?
+|||
+
+## Overview
+
+One or two sentences describing the project -- something memorable and accurate
+that distinguishes your project to quickly orient readers who may be
+reviewing multiple assessments.
+
+* Background. Provide information for reviewers who may not be familiar with
+your project's domain or problem area.
+* Goal. The intended goal of the projects including the security guarantees
+the project is meant to provide (e.g., Flibble only allows parties with an
+authorization key to change data it stores)
+* Non-goals.  Non-goals that a reasonable reader of the project’s literature
+could believe may be in scope (e.g., Flibble does not intend to stop a
+party with a key from storing an arbitrarily large amount of data, possibly
+incurring financial cost or overwhelming the servers)
+
+## Intended use
+
+* Target users and use cases. Provide a mapping from [standard personas](../../usecases.md) to the
+nomenclature used in your project docs (which you should then use consistently
+for the remaider of this document). Describe the scenarios in which the project is expected to be used. This must be specific enough to provide context for analysis. For example:
+
+    Flibble can be used in any cloud environment.  Three diverse examples are
+    as follows:
+    1. when a Flibble server is used by legacy servers as a
+    database for salted password hashes.
+    2. a Flibble cloudlet may be run
+    on virtualized fog hardware near smartphone users.
+    3. a Flibble distributed service may serve as a backend for a Notary image registry.)
+* Operation.  A description of the operational aspects of the system, such as
+how keys are likely to be managed and stored.
+
+## Project design
+* Design.  A description of the system design that discusses how it works.
+This is especially critical for any mechanisms that relate to the security
+of the system.
+
+## Security analysis
+* Attacker motivations.  A discussion about the likely goals of an attacker.
+This likely relates closely to the impact of different attacks in the
+scenarios.  (e.g., In the password hash case, the attacker wants to expose
+those hashes on the Flibble server.  However, a Flibble cloudlet attacker
+may find it more interesting to bring down the service.)
+* Expected attacker capabilities.  A description of likely capabilities that
+the attacker has in these scenarios should be described.  Both assumptions
+about the strength and limitations of attackers should be described.
+(e.g., We assume that an attacker may be able to exploit implementation
+errors in some set of the servers to take control of them.  However, we
+assume the attacker cannot break AES or SHA256.)
+* Attack risks and effects.  A rough estimation of the risk posed by different attacks, and potential negative consequences.
+(e.g., The master Flibble server only communicates with Flibble servers
+using a minimalistic API that is formally verified and written in Rust.)
+* Security degradation.  A discussion about the resulting security when
+various attacks are launched.  Note, that no system is secure in all
+scenarios, hence it is expected that this will include areas where attacks
+compromise all meaningful security.  (e.g., If an attacker is able to
+compromise the “master” Flibble server, they may read, write, or delete any
+content stored on any system)
+
+## Secure development practices
+* Development pipeline.  A description of the testing and review processes
+that the software undergoes as it is developed and built.
+* Communication Channels. Reference where you document how to reach your
+team or describe in corresponding section.
+  * Internal. How do team members communicate with each other?
+  * Inbound. How do users or prospective users communicate with the team?
+  * Outbound. How do you communicate with your users? (e.g. flibble-announce@ mailing list)
+* Vulnerability Response Process. Who is responsible for responding to a report. What is the reporting process? How would you respond?
+* Ecosystem. How does your software fit into the cloud native ecosystem?  (e.g. Flibber is integrated with both Flocker and Noodles which covers virtualization for 80% of cloud users. So, our small number of "users" actually represents very
+wide usage across the ecosytem since every virtual instance uses Flibber
+encryption by default.)
+
+## Roadmap
+
+* Project next steps. Link to your general roadmap, if available, then list
+prioritized next steps that may have an impact on the risk profile of your
+project, including anything that was identified as part of this review.
+* CNCF requests. In the initial draft, please include whatever you believe
+the CNCF could assist with that would increase security of the ecosystem.
+
+## Appendix
+
+* Known Issues over time. List or sumarize statistics of past vulerabilities
+with links. If none have been reported, provide data, if any, about your
+track record in catching issues in code review or automated testing.
+* CII Best Practices. A brief discussion of where the project is at with
+respect to CII best practices and what it would need to achieve the badge.
+* Case Studies. Provide context for reviewers by detailing 2-3 scenarios of
+real-world use cases.
+* Related Projects / Vendors. Reflect on times prospective users have asked
+about the diffeerences between your project and projectX. Reviewers will have
+the same questions.

assessments/guide/project-lead.md

@@ -0,0 +1,13 @@
+# Project Lead
+
+In the context of the security assessment, the "project lead" should be
+someone on the security team for the project. For new or smaller projects
+without an established security team, this could be a project maintainer
+or they may delegate to a regular contributor who has an interest in security.
+
+## Expected time / effort
+
+The level of effort for the team providing the information is expected to
+be around 80 hours of work.  Note, that projects that have already
+performed a security analysis internally are expected to have much lower
+requirements.

assessments/guide/security-reviewer.md

@@ -0,0 +1,31 @@
+# Security Reviewers
+
+Reviewers will try to understand the
+system and probe its security.  Specifically design level issues are meant
+to be addressed as well as high level problems with the project’s setup and
+operation. This is meant to provide an independent analysis and estimation
+of the above information.  The goal is to ask questions to understand if
+there are hidden assumptions, underestimated risk, or design issues that
+harm the security of the project.  It may be useful to reach out to
+community members to understand the answers to some questions, especially
+involving deployment scenarios and the impact of attacks.
+
+## Qualifications
+
+Unless approved by the SIG chairs, at least one of the reviewers will
+have previously performed a SAFE WG audit.  (Exemptions are expected to be
+granted bootstrap the process but only in extreme cases later on.)  In
+general, a reviewer should have performed security audits for diverse
+organizations.  The ideal reviewer should also have been the recipient
+of security audits for a software project they manage.  Note that it is
+encouraged to have participation (shadowing) from participants that are not
+yet qualified to help them gain the necessary skills to be a reviewer
+in the future.
+
+## Time and effort
+
+The level of effort for the reviewers is expected to be 10 hours.
+Despite the fact that there may be some back and forth to get clarification
+on a few points, it is expected analysis can usually be concluded in a few
+weeks.  This will primarily involve carefully reading the written
+document and analyzing the security assertions and assumptions.