Figure out examples of reference architecture for supporting the Real Time Carbon Footprint Standard

[mrchrisadams]

Hey @rossf7 - I have an idea how you might implement reporting based on this Real Time Carbon Footprint Standard, but I figured it would make sense to start a convo in the open about what a reference architecture might look like.

Here's the proposal:
https://github.com/orgs/Green-Software-Foundation/discussions/34

And here's the original writeup
https://medium.com/@adrianco/proposal-for-a-realtime-carbon-footprint-standard-60b71c269948

And
https://github.com/adrianco/slides/blob/master/Cloud%20DevSusOps%20London.pdf
The deck is really worth a look, and the green web fdn might be in a particularly interesting place to surface some of this info, given we ask providers for information about how they are accounting for their emissions and they often tell us about how which RECs they're purchasing and so on.

The PR FAQ is worth reading too
Green-Software-Foundation/software-standards-wg#67

I'll add some notes here

How I think we could do this with code now:

You can use the Green Web Exporter for Grafana to provide CO2 figure for every hour - it’s got integrations with existing providers of data, and we’re friendly with them:
https://github.com/thegreenwebfoundation/grid-intensity-go/

We know about Scaphandre, which gives the very precise, per process figures at very high temporal resolution for power, along with the ability to tag various processes.
https://github.com/hubblo-org/scaphandre/blob/main/docs_src/explanations/how-scaph-computes-per-process-power-consumption.md

I think Kepler essentially does the same thing, providing high resolution figures, but at the container level instead, and via ebpf. I'm less familiar with the internals, and how tightly it is tied to k8s and containers:
https://github.com/sustainable-computing-io/kepler

Any chance we can try sketching this out on a call?

[mrchrisadams]

For future reference:

This paper is the first I've seen that tried to address the assumptions made by lots of energy tracking tools:

https://hongyu.nl/papers/2023_hotcarbon_energat.pdf

Check it - the diagram in the paper specifically refers to disaggregated compute and storage - I haven't seen any tooling explicitly refer to how this assumption impacts readings:

Speaking of which, the numbers from this new tool seem significantly higher than Scaph and other tools. I wonder how Kepler compares?

Here's the paper:
https://hongyu.nl/papers/2023_hotcarbon_energat.pdf

and here's the video
https://mediaspace.ucsd.edu/media/HotCarbon%E2%80%9923%3A%20EnergAt%3A%20Fine-Grained%20Energy%20Attribution%20for%20Multi-Tenancy%20(He%20et%20al.)/1_ac1bpnvt/307441832