Test for efficiency: Requests Per Second / Watt

[naturallymitchell]

This test would help those looking to deploy hardware and software on a budget.
Searching Google for "requests per second per watt" yields many results like this Register processor review. AFAIR, an Intel chief said that the most important measure of a processor wasn't performance, but performance per watt. Perhaps a software-defined power meter like powerapi could help.

[matbrgz]

I think this question is very valid, here in my university I already hear discussions about it. But nothing very profitable. To implement the PowerAPI next to the Benchmarks of the Frameworks it is necessary a discussion about it, so I understood PowerAPI would run alongside the framework, being necessary an implementation for each tested framework which today are more than 300 (if I am not mistaken), it runs in Scala then it is also necessary to reshape all docker file. I think it would be very valuable to take this discussion forward, I'm sure TechEmpower community?

[naturallymitchell]

It looks like s-tui CPU monitoring utility supports collecting power consumption data from any Intel CPU after Sandy Bridge, eg the Skylake CPUs in TechEmpower's servers, and can export it to csv.

[bhauer]

It's fun to see this idea come up here. We've tossed similar ideas around internally but haven't put the time into putting the necessary math together.

We even (mostly jokingly) toyed with working the math all of the way through to CO2 emissions from electricity generation in a given data center market, then mapping that to anything we can find from the IPCC. It would be fun to know your application server choice means 0.00000001 C per year (or whatever it works out to).

[bhauer]

"Your Software’s Carbon Footprint" is a blog entry/article discussing the notion I raised in my previous comment.

[GaelGirodon]

What about trying to use Scaphandre during benchmarks to measure power consumption? This tool looks promising, modern and optimized (written in Rust). This could be a good addition to existing metrics (RPS, latency, memory, CPU, ...) and could make these benchmarks even more relevant as more and more people are looking for energy and cost efficiency more than raw performance.

[franz1981]

Great idea! Carbon could be interesting and relevant too!