Merge pull request #156 from owaspsamm/update-benchmark-page

Resolves #155

config/_default/config.toml

@@ -157,9 +157,9 @@ paginate = 10
     weight = 42
 
 [[menu.main]]
-    name = "Benchmarking"
+    name = "Benchmark"
     parent = "guidance"
-    url  = "/benchmarking/"
+    url  = "/benchmark/"
     weight = 43
 
 [[menu.main]]

content/en/assessment.md

@@ -34,4 +34,4 @@ SAMMwise is part of the official SAMM toolset and allows you to self-host an onl
 
 Based on your assessment, you can now set targets for improvement and use the Toolbox to create and track your own SAMM roadmap.
 
-We encourage you to share your SAMM assessment data with our [Benchmark](/benchmarking/) project. Once we have enough data available, you can start to compare yourself with your peers!
+We encourage you to share your SAMM assessment data with our [Benchmark](/benchmark/) project. Once we have enough data available, you can start to compare yourself with your peers!

content/en/benchmark.md

@@ -0,0 +1,84 @@
++++
+title = "Benchmark"
+description = "Benchmark"
+keywords = ["benchmark","what is","questions"]
++++
+
+## Goals
+
+### How important is it for your organization to compare against peers?
+This question got a 7 or higher reply from 69% of SAMM users during our 2022 questionnaire. Helping our users answer the critical questions “How am I doing?” and “What might be working for other similar organizations?” has been part of our roadmap for quite some time. Now, we’ve made this sub-project a priority.
+It is our goal to build a database for companies to measure the maturity of their security development practices against the industry based on variables such as verticals and company size. In turn, the information we collect can help the SAMM model evolve based on actual information from the field.
+As an added benefit, this project can facilitate research on secure development practices worldwide as universities and researchers analyze and interpret the information the community donates, providing valuable insights.
+<br/>
+<br/>
+
+## Roles
+
+This initiative can only succeed with the help of the community of practitioners that surround the OWASP SAMM project. 
+Practitioners help organizations with SAMM assessments and serve as their benchmark data owner. They demonstrate thorough knowledge of the SAMM model and assess organizations according to the guidelines you can find in our [Getting started](/getting-started) page, and our {{< external-link "https://owaspsamm.thinkific.com/courses/samm" "Fundamentals Course">}}. Also, check out the [Determining Scope](/blog/2023/05/24/determining-scope-when-implementing-samm/) blog post. Practitioners might be internal to the organization that is submitting data, or [external consultants](/practitioners).
+
+
+{{< responsive-image  "/img/pages/benchmark_roles.svg" "benchmark roles">}}
+Visit the [SAMM Core Team](/team) page.
+
+## Contributions
+The OWASP SAMM core team accepts only benchmark contributions through verified practitioners. They act as intermediaries between the OWASP SAMM Core Team and their clients of various industries. 
+A particular organization is only represented by an ID in the benchmark project database. The practitioner who submitted the data for the organization can use this ID to trace the submission back to the organization.
+From the perspective of the SAMM core team, “the submitter” in this approach is always a practitioner, not a company, even though it could be an internal practitioner. 
+We accept data from SAMM v1.5, SAMM v2.x, and beyond. There is support for partial comparisons between SAMM v1.5 and SAMM v2.x, but as the model has undergone breaking changes for v2.0 it will not be a full comparison between the versions.
+
+### Submission process
+1. Any practitioner can contact the SAMM core team and subject themselves to be vetted. This process is a sanity check to ensure a practitioner exists, is knowledgeable on SAMM and carries out assessments according to the best practices.
+2. The SAMM core team vets the practitioner (within a 30-day window), and upon request lists the practitioner on the owaspsamm.org website
+3. The practitioner requests permission from their client to share assessment data with the benchmarking initiative
+4. The practitioner sends an email with the required metadata and SAMM assessment results to [email protected] 
+5. The data becomes part of the benchmark
+
+### Metadata
+The following metadata are (*) required or optional when submitting benchmark datasets:
+* \* Contributor Name (submitter name) 
+* \* Contributor Contact Email
+* \* Date assessment conducted (MM/YYYY) 
+* \* Type of Assessment (Self or 3rd Party) 
+* \* Scope of Assessment (Org, BU, Product, Team, … )
+* \* Answers to the SAMM Assessment Questions 
+* Geographic Region (Africa, Asia-Pacific, Europe, Latin America, North America, Global)
+* Primary Industry (Multiple, Government, Critical Infrastructure, Defense, Aerospace, Automotive, Manufacturing, Healthcare, Finance, Fintech, IoT, Cloud, ISV, Retail, Other -please expand)
+* Approximate number of developers in org (1-100, 101-1000, 1001-10000, 10000+)
+* Approximate number of developers in scope of assessment (1-100, 101-1000, 1001-10000, 10000+)
+* Approximate number of primary appsec (1-5, 6-10, 11-20, 20+)
+* Approximate number of secondary appsec (0-20, 21-50, 51-100, 100+)
+* Primary SDL Methodology (Waterfall, Agile, DevOps, Other )
+The more information provided, the better the comparative analysis will be.
+<br/>
+<br/>
+
+## Tapping into the benchmark data
+The end-goal of the benchmark initiative is for companies to use the data to measure themselves against their peers in the industry. Our main hurdle at this point is to gather a large enough dataset to guarantee accurate comparisons and maintain full anonymity for the contributing parties. 
+As an added benefit, we’ll use the data to prioritize the publication of guidance for streams and activities.
+
+
+### Data visibility
+We’ll start off with limited access to the raw data, and as the number of data submissions increases we’ll be able to have all the (aggregated) data publicly available. We defined the following stages of data visibility so we can give back to the community as soon as possible
+Stages of data visibility
+1. Report (starting 2023)  
+    During this stage, the core team will create an annual report on the state of software security.  
+    <br/>
+2. Academic Papers (starting 2024)  
+    At this point, we’ll share the information with academic researchers who can analyze the data and share their insights with the community.  
+    <br/>
+3. Practitioners that contributed (Q4 2024)  
+    Vetted practitioners that contributed datasets will be able to compare scores or even get a copy of the database. To reach the latter, we need to have a large enough dataset to ensure anonymity of all contributors.  
+    <br/>
+4. Public availability (2025)  
+    The complete benchmark database is published for all to download or integrate into their tooling via API calls to the underlying infrastructure. 
+
+### Data Retention
+There is no predefined retention period for the submitted data. Data will age out over the years and we’ll probably leave it out of reports, for instance. All data will remain part of the dataset, though, unless removed upon specific request. 
+The core team can remove data from the dataset upon request from the original submitter (practitioner), or if the team detects fraudulent activity.
+<br/>
+<br/> 
+
+## Updates
+We will be updating this page and the process as the project progresses. If you have any questions, please send an email to [email protected]

content/en/blog/determining-scope-when-implementing-samm.md

@@ -88,7 +88,7 @@ When determining your scope, set realistic expectations and targets. This means
 
 Starting with a single team is an effective strategy when introducing SAMM in your organization. By focusing on one willing team, you can build a solid foundation for future growth. You can build momentum and create a positive environment where relevant stakeholders feel motivated to continue improving. With the foundation in place, you can expand your efforts to other teams or areas, building on the success of the initial team to create a more robust and secure SDLC.
 
-Once you have a full SAMM cycle, consider donating your datasets to the benchmarking initiative. What is a full SAMM cycle, you ask? Follow these scoping guidelines, perform your assessments, carry out the roadmap plan, and have all your teams aligned. Then, you’ll be ready to contribute to a stronger community and better SAMM.
+Once you have a full SAMM cycle, consider donating your datasets to the benchmark initiative. What is a full SAMM cycle, you ask? Follow these scoping guidelines, perform your assessments, carry out the roadmap plan, and have all your teams aligned. Then, you’ll be ready to contribute to a stronger community and better SAMM.
 
 
 

content/en/blog/how-iso-and-samm-complement-each-other.md

@@ -38,7 +38,7 @@ In turn, an ISO certified ISMS can facilitate implementing an SDLC with SAMM. Fo
 
 Implementing SAMM, you measure progress and build roadmaps but it will never be the simple stamp of approval you get from an audit. While SAMM offers tools to keep track of your progress along maturity levels, there is no such thing as an “approved SAMM implementation” quality label. SAMM is not meant to be a certification.
 
-That being said, the SAMM team is working on a benchmarking initiative that will allow you to compare your organization’s security posture to your industry peers’. 
+That being said, the SAMM team is working on a benchmark initiative that will allow you to compare your organization’s security posture to your industry peers’. 
 
 
 ### Conclusion

content/en/blog/owasp-samm-roadmap.md

@@ -55,7 +55,7 @@ Within the next few months
 * Mappings
 * OWASP references
 * PDF generation
-* Data model for assessments and benchmarking
+* Data model for assessments and benchmark
 * Migration to new GitHub organization https://github.com/owaspsamm/
 * Roadmap templates
 * Improved quickstart guide

content/en/blog/samm2-release.md

@@ -16,7 +16,7 @@ The new SAMM v2 consists of the following components:
 * The [SAMM Model](/model/) overview and introduction, explaining the maturity model in detail
 * A [Quick-start Guide](/guidance/quick-start-guide/) with different steps to improve your secure software practice
 * An updated [SAMM Toolbox](/assessment/) to perform SAMM assessments and create SAMM roadmaps
-* A new [SAMM Benchmark initiative](/benchmarking/) to compare your maturity and progress with other similar organizations and teams
+* A new [SAMM Benchmark initiative](/benchmark/) to compare your maturity and progress with other similar organizations and teams
 
 ### What's changed with SAMM v2?
 

content/en/contributing.md

@@ -13,9 +13,9 @@ Link icons {{< icon "fas fa-link">}} indicate links to pages on our website wher
 
 * **<a href="#Testimonials">Testimonials</a>**  
 
-* **[Benchmarking](/benchmarking)</a>**  {{< icon "fas fa-link">}}  
+* **[Benchmark](/benchmark)</a>**  {{< icon "fas fa-link">}}  
     How am I doing? What works for similar organizations? Donate SAMM data sets!  
-    Visit our benchmarking page.
+    Visit our benchmark page.
 
 * **<a href="#GitHub">GitHub issues</a>**  
     Read more about how we use GitHub for our content and how you can contribute.

content/en/faq.md

@@ -50,7 +50,7 @@ Glad you asked, here is a list of suggestions:
 
 * Share your experience with a SAMM testimonial? We are building a list of testimonials for the website.
 * Encourage your peers to use SAMM and share their experiences as well.
-* Donate SAMM data sets to our [Benchmark initiative](https://owaspsamm.org/benchmarking/). Get in touch with [Brian](mailto:[email protected]).
+* Donate SAMM data sets to our [Benchmark initiative](https://owaspsamm.org/benchmark/). Get in touch with [Brian](mailto:[email protected]).
 * Check out the list of open issues in any of our {{< external-link "https://github.com/owaspsamm" "GitHub repositories">}} and let us know if you can help with any of these
 * Help us translate SAMM to other languages on {{< external-link "https://crowdin.com/project/owasp-samm" "CrowdIn">}}.
 * Sponsor SAMM. Get more information [here](https://owaspsamm.org/sponsors/).

content/en/guidance/quick-start-guide.md

@@ -95,7 +95,7 @@ steps:
         resources:
             resource1:
                 heading: SAMM roadmap chart
-                description: Worksheet (part of the SAMM Benchmarking as a comparative source)
+                description: Worksheet (part of the SAMM Benchmark as a comparative source)
             resource3:
                 heading: 
                 description: Leverage the Roadmap worksheet in the SAMM Toolbox to help calculate maturity score improvements based on future answers

content/en/release-notes-v2.md

@@ -11,7 +11,7 @@ The new SAMM version 2 consists of the following components:
 * the [SAMM Model](/model/) overview and introduction, explaining the maturity model in detail
 * a [Quick-start Guide](/guidance/quick-start-guide/) with different steps to improve your secure software practice
 * an updated [SAMM Toolbox](/assessment/) to perform SAMM assessments and create SAMM roadmaps
-* a new [SAMM Benchmark initiative](/benchmarking/) to compare your maturity and progress with other similar organizations and teams
+* a new [SAMM Benchmark initiative](/benchmark/) to compare your maturity and progress with other similar organizations and teams
 
 
 The original model [OpenSAMM 1.0](https://www.opensamm.org/) was written by Pravir Chandra and dates back to 2009. Over the last 10 years, it has proven a widely distributed and effective model for improving secure software practices in different types of organizations. With SAMM v2, further improvements have been made to deal with some of its perceived limitations.

content/en/stream-guidance.md

@@ -1,7 +1,7 @@
 +++
 title = "Stream Guidance"
 description = "Stream Guidance"
-keywords = ["benchmarking","what is","questions"]
+keywords = ["benchmark","what is","questions"]
 +++
 
 ## Guidance per Stream in the model

content/en/user-day/samm-benchmark.md

@@ -7,7 +7,7 @@ image: /img/team/brian.jpg
 affiliation: 
 role: 
 abstract: |
-    Join us as we traverse the landscape of OWASP SAMM Benchmarking. Whether you're a seasoned SAMM veteran or a newcomer to the software assurance world, this presentation provides insights into the new SAMM Benchmark collection and visualization processes. Well walkthrough how to contribute and what you can expect for information once we reach a critical mass of data.
+    Join us as we traverse the landscape of OWASP SAMM Benchmark. Whether you're a seasoned SAMM veteran or a newcomer to the software assurance world, this presentation provides insights into the new SAMM Benchmark collection and visualization processes. Well walkthrough how to contribute and what you can expect for information once we reach a critical mass of data.
 
 bio: |
    Brian has 22 years of experience in various roles in IT with the majority of that in application development and security. His day job is serving as an Assistant Professor teaching a full load of Computer Science and Cybersecurity classes at Union University. He helped build the FedEx AppSec team, worked on the Trustworthy Computing team at Microsoft, consulted on software security for years, and served as a project lead and active contributor for SAMM v1.1-2.0+ and OWASP Top 10 2017, 2021, 2024, and the OWASP DAVID project. Brian is a contributor to the RABET-V Pilot Program for assessing non-voting election technology. He holds several Cybersecurity and IT certifications and is working on his Doctor of Computer Science in Cybersecurity and Information Assurance.

data/features/benchmarking.yaml → data/features/benchmark.yaml

@@ -1,5 +1,5 @@
 weight: 3
-name: "Benchmarking"
-url: "/benchmarking"
+name: "Benchmark"
+url: "/benchmark"
 icon: "fas fa-map-marker"
 description: "Where is your organization on the map? Contribute. Be a part of SAMM."