From fdbe975c485565fd7d4716c7674b2f365a7dac91 Mon Sep 17 00:00:00 2001 From: jmc <33655003+jmcook1186@users.noreply.github.com> Date: Wed, 11 Oct 2023 16:18:57 +0100 Subject: [PATCH 1/2] update ccf docs and add test impl --- docs/implementations/ccf.md | 105 +++++++++++++++++++++++++++++------- examples/impls/ccf-test.yml | 19 +++++++ 2 files changed, 105 insertions(+), 19 deletions(-) create mode 100644 examples/impls/ccf-test.yml diff --git a/docs/implementations/ccf.md b/docs/implementations/ccf.md index 920401136..b867cf0d0 100644 --- a/docs/implementations/ccf.md +++ b/docs/implementations/ccf.md @@ -1,27 +1,32 @@ # Cloud Carbon Footprint -Cloud Carbon Footprint is an open source tool that provides visibility and tooling to measure, monitor and reduce your cloud carbon emissions. We use best practice methodologies to convert cloud utilization into estimated energy usage and carbon emissions, producing metrics and carbon savings estimates that can be shared with employees, investors, and other stakeholders. +"Cloud Carbon Footprint is an open source tool that provides visibility and tooling to measure, monitor and reduce your cloud carbon emissions. We use best practice methodologies to convert cloud utilization into estimated energy usage and carbon emissions, producing metrics and carbon savings estimates that can be shared with employees, investors, and other stakeholders." - [CCF](https://www.cloudcarbonfootprint.org/) -## Implementation +## IEF Implementation -IEF implements this plugin to the IEF Specification based on the computational methodology used by the Cloud Carbon Footprint. +IEF reimplements the Cloud Carbon Footprint methodology fro scratch conforming to the IEF specification. This means the CCF models can be run inside IEF without any external API calls and can be invoked as part of a model pipeline defined in an `impl`. -Cloud Carbon Footprint includes calculations for three cloud providers namely AWS, Azure and GCP. +Cloud Carbon Footprint includes calculations for three cloud providers: AWS, Azure and GCP. -By default, Linear interpolation is used to estimate the energy. Interpolation is performed using the CPU Design Cycle, for example in Intel Chips, **CoffeeLake** and for example in AMD Chips, **EPYC 2nd Gen**. +The general methodology is as follows: -Additionally, **TEADS model curve for AWS** available in the CCF dataset can be used for spline curve interpolation for instances in AWS infrastructure. +`Total CO2e = operational emissions + embodied Emissions` -Resulting values are generally approximation and should be revalidated across different models as there can be significant difference between values. +Where: -New instances across all cloud providers might not be recognized by CCF. Earliest possible instances recognized are released before 2021 December. +`Operational emissions = (Cloud provider service usage) x (Cloud energy conversion factors [kWh]) x (Cloud provider Power Usage Effectiveness (PUE)) x (grid emissions factors [metric tons CO2e])` -## Usage +And: + +`Embodied Emissions = estimated metric tons CO2e emissions from the manufacturing of datacenter servers, for compute usage` -Configure method has to be called on the instantiated object before any other calls are done. +You can read a detailed explanation ofn the calculations in the [CCF docs](https://www.cloudcarbonfootprint.org/docs/methodology/) and see the code for our implementation in [this repository](../../src/lib/ccf/). -Calculate method expects an array of observations. Each observation should feature `duration`,`cpu`,`datetime` in the formats specified below. +## Usage +In IEF, the model is called from an `impl`. An `impl` is a `.yaml` file that contains configuration metadata and usage observations. This is interpreted by the command line tool, `rimpl`. There, the model's `configure` method is called first. The model config should define a `provider` and `instance-type`. Each observation is expected to contain `duration`,`cpu-util` and `timestamp` fields. + +You can see example Typescript invocations for each provider below: ### AWS @@ -31,13 +36,13 @@ import {CloudCarbonFootprint} from 'ief'; const ccf = new CloudCarbonFootprint(); ccf.configure({ provider: 'aws', - instance_type: 'c6i.large' + instance_type: 'm5n.large' }) const results = ccf.calculate([ { duration: 3600, // duration institute - cpu: 0.1, // CPU usage as a value between 0 and 1 in floating point number - datetime: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp + cpu-util: 10, // CPU usage as a percentage + timestamp: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp } ]) ``` @@ -55,8 +60,8 @@ ccf.configure({ const results = ccf.calculate([ { duration: 3600, // duration institute - cpu: 0.1, // CPU usage as a value between 0 and 1 in floating point number - datetime: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp + cpu-util: 10, // CPU usage as a percentage + timestamp: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp } ]) ``` @@ -68,16 +73,78 @@ import {CloudCarbonFootprint} from 'ief'; const ccf = new CloudCarbonFootprint(); ccf.configure({ - provider: 'aws', + provider: 'gcp', instance_type: 'n2-standard-2' }) const results = ccf.calculate([ { duration: 3600, // duration institute - cpu: 0.1, // CPU usage as a value between 0 and 1 in floating point number - datetime: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp + cpu-util: 10, // CPU usage as a percentage + timestamp: '2021-01-01T00:00:00Z', // ISO8601 / RFC3339 timestamp } ]) ``` +## Example Impl + +The following is an example of how CCF can be invoked using an `impl`. + +```yaml +name: ccf-demo +description: example impl invoking CCF model +initialize: + models: + - name: ccf + kind: builtin +graph: + children: + child: + pipeline: + - ccf + config: + ccf: + provider: aws + instance_type: m5n.large + observations: + - timestamp: 2023-07-06T00:00 # [KEYWORD] [NO-SUBFIELDS] time when measurement occurred + duration: 1 + cpu-util: 10 +``` + +This impl is run using `rimpl` using the following command, run from the project root: + +```sh +npx ts-node scripts/rimpl.ts --impl ./examples/impls/ccf-test.yml --ompl ./examples/ompls/ccf-test.yml +``` +This yields a result that looks like the following (saved to `/ompls/ccf-test.yml`): + +```yaml +name: ccf-demo +description: example impl invoking CCF model +initialize: + models: + - name: ccf + kind: builtin +graph: + children: + front-end: + pipeline: + - ccf + config: + ccf: + provider: aws + instance_type: m5n.large + observations: + - datetime: 2023-07-06T00:00 + duration: 1 + cpu: 10 + impacts: + - timestamp: 2023-07-06T00:00 + duration: 1 + cpu-util: 10 + provider: aws + instance_type: m5n.large + energy: 0.000018845835066981333 + embodied_emissions: 0.02553890791476408 +``` \ No newline at end of file diff --git a/examples/impls/ccf-test.yml b/examples/impls/ccf-test.yml new file mode 100644 index 000000000..a7b978272 --- /dev/null +++ b/examples/impls/ccf-test.yml @@ -0,0 +1,19 @@ +name: ccf-demo +description: example impl invoking CCF model +initialize: + models: + - name: ccf + kind: builtin +graph: + children: + child: + pipeline: + - ccf + config: + ccf: + provider: aws + instance_type: m5n.large + observations: + - timestamp: 2023-07-06T00:00 # [KEYWORD] [NO-SUBFIELDS] time when measurement occurred + duration: 1 + cpu-util: 10 From 2296bbaaa0b382d785f5d99999c5a7296ac3f8db Mon Sep 17 00:00:00 2001 From: Joseph Cook <33655003+jmcook1186@users.noreply.github.com> Date: Thu, 12 Oct 2023 09:28:11 +0100 Subject: [PATCH 2/2] Update docs/implementations/ccf.md Signed-off-by: Joseph Cook <33655003+jmcook1186@users.noreply.github.com> --- docs/implementations/ccf.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/implementations/ccf.md b/docs/implementations/ccf.md index b867cf0d0..8c2bfc042 100644 --- a/docs/implementations/ccf.md +++ b/docs/implementations/ccf.md @@ -136,7 +136,7 @@ graph: provider: aws instance_type: m5n.large observations: - - datetime: 2023-07-06T00:00 + - timestamp: 2023-07-06T00:00 duration: 1 cpu: 10 impacts: