FedRAMP updated the Plan of Actions and Milestones (POA&M) template to include two new columns. The additional columns were added at the behest of agency partners to help them track Cybersecurity and Infrastructure Security Agency (CISA) Binding Operational Directive (BOD) 22-01 findings, and the associated Common Vulnerabilities and Exposures (CVEs).
-:ET \ No newline at end of file diff --git a/.jekyll-cache/Jekyll/Cache/Jekyll--Converters--Markdown/2a/1c643656e3a6e1bec4b78fed528c33e6f9acdc7c520b7749e1fd6e3893f3fe b/.jekyll-cache/Jekyll/Cache/Jekyll--Converters--Markdown/2a/1c643656e3a6e1bec4b78fed528c33e6f9acdc7c520b7749e1fd6e3893f3fe deleted file mode 100644 index c861463c3..000000000 --- a/.jekyll-cache/Jekyll/Cache/Jekyll--Converters--Markdown/2a/1c643656e3a6e1bec4b78fed528c33e6f9acdc7c520b7749e1fd6e3893f3fe +++ /dev/null @@ -1,2 +0,0 @@ -I"IFedRAMP security control baselines specify control parameter requirements and organizational parameters specific to the provider’s control implementation. Since certain controls may be required to govern Agency user interaction, control organizational parameters may need to be included in the task order and specified. The FedRAMP PMO suggests that agencies review the FedRAMP security control baseline, and that agencies do not contractually specify parameters for controls in the FedRAMP baseline, except from the perspective of a consumer’s implementation of a control.
-:ET \ No newline at end of file diff --git a/_data/navbar.yml b/_data/navbar.yml index 47e646657..75fd42f18 100755 --- a/_data/navbar.yml +++ b/_data/navbar.yml @@ -107,6 +107,13 @@ assigned: show_in_menu: true show_in_footer: false + - + text: ET Framework + href: pages/et-framework.md + permalink: /et-framework/ + show_in_menu: true + show_in_footer: false + - text: Blog href: pages/blog.md diff --git a/_layouts/artificial-intelligence.html b/_layouts/artificial-intelligence.html new file mode 100644 index 000000000..417b12cf0 --- /dev/null +++ b/_layouts/artificial-intelligence.html @@ -0,0 +1,209 @@ +--- +layout: full-width +--- + + +FedRAMP is prioritizing Generative AI, specifically when used for chat interfaces and code generation and debugging tools that use Large Language Models (LLMs), and prompt-based image generation. This applies both to use cases where a human is directly interfacing with the service, and when the service is accessed using associated application programming interfaces (APIs).
+ +To ensure that FedRAMP is appropriately focused on CSOs that offer these services, FedRAMP will request that CSPs include service-related documentation in the form of a model card that provides agencies easily understandable information about the models provided by a vendor, as well as potential uses and limitations. The template model card that FedRAMP requests is included in this GitHub repository for Artificial Intelligence. The FedRAMP PMO will perform a qualitative review of this CSP provided model card(s) to ensure necessary fields are completed, responses are relevant to the field, and clearly presented.
+Artificial intelligence (AI) (as defined in 15 U.S.C. 9401(3)): A machine-based system that can, for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments. Artificial intelligence systems use machine- and human-based inputs to perceive real and virtual environments; abstract such perceptions into models through analysis in an automated manner; and use model inference to formulate options for information or action.
+ +AI model: A component of an information system that implements AI technology and uses computational, statistical, or machine-learning techniques to produce outputs from a given set of inputs.
+ +AI model card: Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type [15]) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information.
+ +Foundation model: An AI model that is trained on broad data, generally uses self-supervision, contains at least tens of billions of parameters, and is applicable across a wide range of contexts.
+ +Generative AI: The class of AI models that emulate the structure and characteristics of input data in order to generate derived synthetic content. This can include images, videos, audio, text, and other digital content.
+Primary Purpose: The core features of the CSP’s offering are to allow end-users or developers to interact with generative AI capabilities. CSPs with generative AI capabilities embedded within a broader product may not receive prioritization.
+ +Use of Algorithms: The offering utilizes a foundation model as an underlying technology for its generative AI capabilities. These models will generally have been released after September 2022. Use of a particular algorithm or technical approach is not as important as the performance of the generative AI to produce accurate, valid, and reliable outputs.
+ +Features align to capabilities: The CSO meets one or more of the following capabilities: 1) large language model-based chat interfaces, 2) code-generation and debugging tools, and 3) prompt-based image generators. Specifically, as stated in Executive Order 14110, “…generative AI offerings that have the primary purpose of providing large language model-based chat interfaces, code-generation and debugging tools, and associated application programming interfaces, as well as prompt-based image generators…”.
+ +CSO Prioritization Target: The number of CSOs that will be targeted for prioritized review process for each emerging technology capability.
+Description: Products that deliver conversational chat interfaces to provide text prompts to the user and then use information consumed by the large language model (LLM) to produce aggregate text-based content on behalf of the user.
+ +Technical Characteristics: The offering should be capable of discerning meaning from open-ended user inputs and provide an appropriate response. The interface may provide an externally accessible API facilitating integration and use by other systems.
+ +Capability Prioritization Target: Three cloud service offerings
+ +Description: A tool used by software developers to help them with creating and debugging software.
+ +Technical Characteristics: The offering should be capable of auto-detecting relevant programming language based off of submitted code snippets or direct prompts. The offering should be able to generate complete methods, subroutines or functions based on input prompts. The offering can refine responses if requested by the user and can debug code inputs. The offering may be able to provide links to additional documentation or directly explain output with appropriate attribution or citation. The offering may be able to translate code into a newer version or into another programming language. The offering may be able to generate code off of describing a potential problem or use-case and generate suggested code. The interface may provide an externally accessible API facilitating integration and use by other systems.
+ +Capability Prioritization Target: Three cloud service offerings
+ +Description: A product that takes text or photographic input and generates new images or videos based on those inputs.
+ +Technical Characteristics: The offering should generate new visual content by recapitulating pixels based on patterns in the training data. The offering may be able to modify an input image based on accompanying text. The offering may be able to generate the output’s subject in multiple graphical or artistic styles. The interface may provide an externally accessible API facilitating integration and use by other systems.
+ +Capability Prioritization Target: Three cloud service offerings
+ +Description: A general purpose API-based solution available to developers and system integrators to incorporate chat, code generation, or image generation capabilities into their new or existing products and services.
+ +Technical Characteristics: The offering should meet the requirements for chat interfaces, code generation, or prompt-based image generation above. The solution must provide a general purpose model(s). APIs must allow external integration and use by both government and commercial systems. The offering must include well-formed documentation with examples for system integrations. The leveraged APIs must support either REST (representational state transfer) or RPC (Remote Procedure Call). The API-based solution may allow for further refinement, training, or fine tuning by the consuming system.
+ +Note: Offerings that only do not make the API externally available, and only leverage or utilize the API from within their offering are not considered for this prioritization category.
+ +Capability Prioritization Target: Three cloud service offerings
+FedRAMP is adopting the Hugging Face model card template as a standard way to provide agencies with easily consumable information about the models provided by a vendor, potential uses, limitations, as well as relevant benchmarks.
+ +The provided model card template can be located here in GitHub. FedRAMP PMO will be doing a qualitative review of the provided model card and will use this assessment in contributing to the prioritization decision. The PMO will verify completeness of data provided to the Mandatory fields. Optional Fields will not be considered as part of the review.
+ +The following areas and fields are required to be completed:
+These are a sampling of reasonable, public or open source model cards that includes information that reflects expected content in a model card. FedRAMP has elected to use the Hugging Face model card as its base. The examples provided below may not follow the Hugging Face model card standard and are provided as illustrative examples. Offerers are expected to use the FedRAMP model card template and must provide a response to all non-optional fields.
+Below are some sample model card writing guides or utilities that may be useful to CSPs in developing their model card. This is not an endorsement of any included tools or guides. CSPs are encouraged to follow the FedRAMP model card template located in GitHub.
+ +https://huggingface.co/docs/hub/en/model-card-annotated
+ +This model card writing tool can be used by a CSP to generate a basic model card and then use an editor which supports markdown, such as visual studio code or https://hackmd.io/ to modify the generated model card.
+ +https://huggingface.co/spaces/huggingface/Model_Cards_Writing_Tool
+ +In response to the President’s Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (AI), FedRAMP is establishing a framework for prioritizing emerging technologies (ETs) for FedRAMP authorization. The term “emerging technology” means those technologies listed in the Office of Science and Technology Policy Critical and Emerging Technologies. This framework will enable routine and consistent prioritization of the most critical cloud-relevant ETs needed for use by federal agencies. This prioritization will control how FedRAMP prioritizes its own work and review processes, and will not address how sponsoring agencies manage their own internal priorities.
+ +This document describes the operational framework to prioritize certain cloud service providers (CSPs) during the FedRAMP authorization process whose cloud service offerings (CSOs) provide certain ET capabilities; how FedRAMP will govern and evaluate this new process; detail the first prioritized technology, generative AI. Importantly, the prioritization process will be integrated on top of existing FedRAMP Authorization paths and will not create new pathways for authorization.
+ +Not all ETs will be prioritized. The initial ET prioritization list includes generative AI capabilities: chat interfaces, code-generation and debugging tools, and associated application programming interfaces (APIs), as well as prompt-based image generators, as required by Executive Order 14110. FedRAMP will update and maintain an evolving list of prioritized ETs at least annually with input from agencies and industry followed by approval from the Board. Technologies will be removed from prioritization either by decision of the FedRAMP Board, or when the target number of CSOs with the desired capabilities are available within the marketplace.
+ +FedRAMP will determine initial CSO prioritization targets, generally three per ET capability, and qualification criteria when publishing or updating the ET List. CSOs meeting the qualifying criteria move to the front of the authorization review process. After the ET capability prioritization target is reached, additional offerings of the same ET capability will not be prioritized. Initially, the FedRAMP ET prioritization framework is designed to facilitate rapid authorization of commercial AI capabilities for agency use. Prioritized CSOs that fail to make consistent progress on authorization will be returned to their former place in the queue opening up an opportunity for additional CSOs. Because this is a new paradigm for the program, FedRAMP will continually evaluate the process and make revisions as needed. Additionally, the board may approve another round of prioritization for some or all the ET capabilities based on demand.
+This guidance document describes the prioritization framework for ETs for FedRAMP. The intent is to ensure ET CSOs are prioritized during the FedRAMP authorization review and available for agency reuse.
+ +Agencies will need to prioritize CSO sponsorship and authorization prior to FedRAMP’s prioritized authorization. Additionally, cloud service providers (CSPs) need to prioritize their remediation of any findings identified by the 3PAO, agency, and PMO in order to accelerate the complete authorization process.
+ +Existing CSPs who wish to incorporate ET CSOs into their offerings via the significant change process must work with their authorizing official for prioritized review and approval.
+ +This framework will initially be applied to artificial intelligence (AI), specifically the three generative AI capabilities discussed in Executive Order 14110: chat interfaces, code generators and debugging tools, image generators, and associated application program interfaces.
+ +FedRAMP will update this document, and update the list of prioritized emerging technology as necessary to meet the goals of the executive order and agency mission needs. While this document has an initial focus on the generative AI technologies outlined in the executive order, FedRAMP anticipates the prioritized ET list to continually evolve and will publish updates to our website. Appendix B - Artificial Intelligence details the initial prioritized Generative AI capabilities, prioritization targets, and related criteria.
+ +General information including resources, blogs, templates, and documentation for the ET prioritization process can be found on FedRAMP’s website under the resources section labeled prioritization.
+ +The FedRAMP authorization process will validate an ET CSO against existing FedRAMP security controls. Agencies must consider their specific use, including data and privacy risks, and follow appropriate authorization and governance processes when deploying any ETs. In the case of generative AI, agencies should consult closely with their Chief AI Officer.
+ +The ET Prioritization Framework provides a process that is integrated across all of the program’s available authorization paths. There are two primary parts to the ET Prioritization Framework (seen in Figure 1):
+ +Figure 1: ET Prioritization Framework
+ + +The Governance Process defines how up to three capabilities will be prioritized for “skip the line” access to FedRAMP at any given time, and the amount of CSOs with a given capability that will be prioritized.
+ +The CSP Evaluation Process outlines how new cloud service providers will have their CSOs qualified to access an accelerated review.
+ +Existing cloud service providers must work with their authorizing official and will follow the significant change request(SCR) process to include new ET CSOs in their authorization.
+ +The goal of the governance process is to identify appropriate ETs and respective capabilities for prioritization and accelerated authorization. Agencies and Industry will be able to submit ideas for additional capabilities to be authorized for future revisions. An electronic submission form will be prepared and made available as the time approaches for new capabilities to be identified.
+ +To maintain the benefit of prioritization, up to three ET capabilities may be identified at any given time. These may be in a singular ET category, such as generative AI, or may span multiple ET categories. These will be updated annually at minimum – regardless of whether authorizations have been completed – and can also be updated as needed, either at the request of the FedRAMP Board or based on stakeholder demand.
+ +Responsible: Chief Information Officer (CIO) Council and Federal Councils including (Federal Chief Information Security Officer (CISO) Council, Chief AI Officer, Chief Data Officer or others)
+ +Accountable: FedRAMP PMO
+ +Consult: FSCAC, NIST, TAG, 3PAOs, Agencies, CSPs
+ +Informed: FedRAMP Board
+ +What: The FedRAMP PMO will coordinate nominations for new emerging technologies from Agencies and Industry partners including CSPs and 3PAOs. FedRAMP PMO will consult with NIST and the TAG to help define definitions and technical characteristics for the nominated ET. Nominations will be validated with and confirmed by the CIO Council, in coordination with other councils or bodies (such as Federal CISO Council, Chief AI Officer Council or Chief Data Officer and others as appropriate). Nominations will include emerging technologies and capabilities that address agencies mission needs.
+ +Output: Recommendation from federal CIOs or CISOs to the FedRAMP PMO regarding their ET needs.
+ +Estimated Duration: Variable depending on inputs from CIO and federal councils, Agencies, and Industry.
+ +Responsible / Accountable: FedRAMP PMO
+ +Consult: Office of Management and Budget (OMB)
+ +Informed: CIO and CISO Federal Councils, FSCAC
+ +What: The FedRAMP PMO proposes an updated list of up to three ET capabilities for prioritization for the Board’s approval. This will include an analysis of the PMO’s ability to process authorizations for these capabilities, including:
+ +Insights from the analysis will be discussed with OMB. They will be compiled into a briefing that provides an estimate of FedRAMPs ability to authorize and communicate a given area of ET services, considering maturity across PMO staff, Agencies, 3PAOs, CSPs, and the broader technology and cybersecurity community. The briefing will outline potential activities necessary to support the prioritized authorization of a given ET if approved.
+ +Input: Nominations from the Federal CIO and CISO Councils, Agencies, and Industry to the FedRAMP PMO, as described above.
+ +Output: Proposed ET List, FedRAMP ET Impact Assessment
+ +Service Level Agreement Target: 1 Month
+Service Level Agreement Threshold: 2 Months
+ +Responsible / Accountable: FedRAMP Board
+ +Consult: FedRAMP PMO, OMB
+ +Informed: CIO and CISO Federal Councils
+ +What: The FedRAMP PMO will brief the FedRAMP Board on its ET prioritization recommendation. The FedRAMP PMO will send all materials in advance of the Board meeting when the ET is on the agenda. The FedRAMP Board will approve a final list of ET capabilities.
+ +EO 14110 specifies the high-level capabilities for the initial list of ETs. This initial list of capabilities is considered adopted once this framework is finalized. Future capabilities will receive review and approval by the FedRAMP Board.
+ +Input: Proposed ET List, FedRAMP ET Impact Assessment
+ +Outputs: Approved ET capability list
+ +Estimated Duration: Variable depending on Board discussion and feedback to the PMO.
+ +Responsible / Accountable: FedRAMP PMO
+ +Consult: Agency Partners and Liaisons, CSPs
+ +Informed: None
+ +What: The FedRAMP PMO will update its process documentation, website, and all necessary systems to align changes to the approved ET list. Updated criteria and resources will be developed and published on the Resources Prioritization page on the FedRAMP website and communicated to stakeholders. The FedRAMP blog and email notification will be used to communicate prioritization application windows.
+ +Input: Approved ET List
+ +Outputs: ET Criteria for Prioritization, ET capability targets, ET CSO Request Form , ET Demand Form
+ +Service Level Agreement Target: 2 weeks
+Service Level Agreement Threshold: 4 weeks
+ +Responsible / Accountable: FedRAMP PMO
+ +Consult: FedRAMP Board, CIO Council and Federal CISO Council, Agency Partners and Liaisons
+ +Informed: None
+ +What: When the PMO authorizes the defined target of CSOs in a given ET capability, the ET will be automatically removed from the prioritization list. The FedRAMP PMO will repeat step 4.2.4 (Update ET Capability List and Criteria) to reflect the removal. These updates will not impact any CSOs who have already been prioritized for another ET capability. The FedRAMP PMO will notify the FedRAMP Board and other relevant stakeholders that the target CSO count has been achieved and may begin step 4.2.1 (Nominate ETs for Prioritization)
+ +Input: Metrics Report on # of CSPs Prioritized for ET Capabilities
+ +Output: Updated ET List
+ +Service Level Agreement Target: 2 week
+Service Level Agreement Threshold: 4 weeks
+The Evaluation Process determines if a CSO meets the ET definition and qualifying criteria. Once identified and verified, then the CSP will be placed in an accelerated position in the FedRAMP authorization queue. These steps take place during the current Preparation phase of the FedRAMP authorization process.
+ +The evaluation process kicks off after a CSP has:
+CSPs can submit their ET CSO Request Form at any point in this process, though as described above, a CSP will not be evaluated for prioritization until completing their 3PAO security assessment. As part of the intake process, the FedRAMP PMO will review the ET criteria for prioritization with the CSP. Initially, there will be two open prioritization application windows during each fiscal year. ET CSO Request Forms submitted outside these active windows will be held until the next application window opens and then evaluated with other CSPs submitting in the application window.
+ +Responsible / Accountable: CSP
+ +Consult: Agency Partner and Liaison
+ +Informed: None
+ +What: CSPs in consultation with their agency partner will submit the ET CSO Request Form and ET Demand Form which includes demand justification and attestation to the ET prioritization qualifying criteria for FedRAMP PMO review. Detailed instructions on how to fill out the forms can be found in the Emerging Technologies Prioritization Criteria and Guidance document.
+ +Input: Justification of how CSP meets ET criteria, and a decision to complete the ET prioritization process with FedRAMP PMO
+ +Outputs: Completed ET CSO Request and Demand Forms submitted to the FedRAMP PMO for consideration.
+ +Estimated Duration: Variable depending on type of ET, complexity of CSO and qualifying criteria, availability of demand data.
+ +Responsible/Accountable: FedRAMP PMO
+ +Consult: Agency Partner and Liaison, CSP
+ +Informed: None
+ +What: The FedRAMP PMO determines if the CSO meets the ET prioritization qualifying criteria in consultation with the agency partner. If the criteria are met, the FedRAMP PMO will review the ET demand forms and calculate the total demand score. The PMO will select qualifying CSOs based on their demand score up to the ET category target.
+ +If the criteria are not met, or if other offerings have the demand score, the CSO will still be placed in queue to be reviewed following the standard process.
+ +Input: ET CSO Request Form, ET Demand Forms
+ +Outputs: Approve or deny that the CSO meets the ET criteria, ET CSO Prioritization Decision Justification, Process CSO accordingly
+ +Service Level Agreement Target: 2 weeks
+Service Level Agreement Threshold: 4 weeks
+ +Responsible/Accountable: FedRAMP PMO
+ +Consult: FedRAMP Board
+ +Informed: CIO Federal and CISO council
+ +What: The FedRAMP PMO will monitor the prioritized CSOs throughout the lifecycle of the ET prioritization process. FedRAMP PMO Security Review Team regularly tracks and communicates the progress of prioritized CSOs towards authorization.
+ +Notification will be sent to the FedRAMP Board and the CIO and Federal Councils when the ET capability target is reached. Subsequently, the FedRAMP Board may initiate a request to do an off-cycle round of the ET governance process.
+ +Input: Authorization Packages
+ +Outputs: Metrics Dashboard and Reports, Report on Prioritized and Non-Prioritized CSOs
+ +Service Level Agreement Target: 2 weeks
+Service Level Agreement Threshold: 4 weeks
+ + +All documents and forms below are authored by the FedRAMP PMO, and will be published on the FedRAMP.gov website.
+ +The 1-2 page list of ETs will be updated annually at a minimum and on-demand based on stakeholder feedback. This list is further refined by the FedRAMP PMO after inputs from the CIO Council, the Federal CISO Council, and other industry inputs as described by the process in Section 4.2 on the governance process. The updated list must be approved by the FedRAMP Board.
+An electronic form available on the FedRAMP website. The form serves to identify whether the CSP or CSO meets the ET criteria for prioritization, and collects information about the CSO. Additionally, this form can be found on fedramp.gov here.
+An electronic form available on the FedRAMP website. The form serves to identify the CSOs demand. This form can be found on fedramp.gov here.
+A detailed instruction manual on how to fill out the ET CSO Request Form and the ET Demand Form which is located electronically in this document, Emerging Technologies Prioritization Criteria and Guidance.
+To assist in clarifying roles and responsibilities, this RACI Matrix is provided as an alternate view of what each member in the ecosystem is performing.
+ +R = Responsible A = Accountable C = Consulted I = Informed
+| + | FedRAMP PMO | +FedRAMP Board | +CIO & Federal Councils | +OMB | +FSCAC | +NIST | +Agency Partners & Liaisons | +CSPs | +3PAOs | +Tag | +
|---|---|---|---|---|---|---|---|---|---|---|
| Governance Process | +||||||||||
| Nominate ETs for Prioritization | +A | +I | +R | ++ | C | +C | +C | +C | +C | +C | +
| Propose ET List | +R A | ++ | I | +C | +I | ++ | + | + | C | ++ |
| FedRAMP Board Decision on the ET List | +C | +R A | +I | +C | ++ | + | + | + | + | + |
| Update ET Capability List and Criteria | +R A | ++ | + | + | + | + | C | +C | ++ | + |
| Shifting Prioritization of CSOs with ET | +R A | +C | +C | ++ | + | + | C | ++ | + | + |
| Evaluation Process | +||||||||||
| Submit the ET CSO Request Form | ++ | + | + | + | + | + | C | +R A | ++ | + |
| Qualification Determination and Queue Placement | +R A | ++ | + | + | + | + | C | +C | ++ | + |
| Monitor ET Approvals | +R A | +C | +I | ++ | + | + | + | + | + | + |