Add timeseries graphs

[corradio]

No description provided.

[martindaniel4]

@corradio - would love to help on that one.

Do you have a spec ? What's your idea here ?

[nikkozzblu]

We should certainly be using c3.js here which come on top of d3
I did it for another project and can brind some code samples if needed.
As an interface, because we have so many variables, maybe a variable picking mecanism from current legend can be better. For instance if you click current FR wind, it open the last 3days historial, if you click it again, it disappears, if you click another variable, it add the curve to the current plot.
Wind and solar would probably be activated by picking a point on the map
It will also need datetime start/end pickers

[corradio]

The idea here is that, everytime the feeder finishes to run (every 5min), it should updates the server cache (memcached) with timeseries data (power, co2 and prices) for all countries for the latest X hours (6 hours?).
We then have a fast way of serving that data to the frontend.

Frontend-wise, I'd suggest making the top-left panel fill the whole screen height, turning it into a left panel (height: 100%, width: fixed). Under the import/exports part, we can then add a graph showing the last X hours of production. Clicking on the "show emissions" button would show emissions instead.

The last open question in my mind is how/where to show the timeseries of co2.

c3.js indeed does sound interesting. However, we use d3 v4 so I don't know if they are compatible. We have to dig deeper.

Let me know your thoughts.

[nikkozzblu]

C3 running in D3v3 is indeed quite unfortunate; we can still plot with d3 directly but we loose a lot of nice tooltip, data loading ability from C3....
On another hand is there any specific needs on other parts that require running v4 rather than v3 ?

[corradio]

Mmh d3 is used a bit everywhere in the webapp, so it will be hard to downgrade. It won't be hard to do the graph by ourselves anyway.
I'm working on www.co2signal.com right now, but I'll attack this next week probably. @nikkozzblu @martindaniel4 if you guys want to start let me know.

[martindaniel4]

@corradio - I am not sure to understand the front-end side. Are you guys thinking of something like this ?
@nikkozzblu - What did you have in mind for the solar and wind timelines ? One tooltip where you click on the map ?

[corradio]

Yeah something like that. And when you hover on each of the lines it shows you the history of that line in the graph placeholder.

[corradio]

Fiddle by @nikkozzblu : https://jsfiddle.net/sz6yrp7s/3/
I'll try and see if I can port c3 to d3v4

[AndyinHawick]

I would be interested in seeing a country-by-country comparison of price versus carbon content. Each country (which has both data) would be represented by a dot (perhaps sized according to total power). X-axis carbon intensity; Y-axis day-ahead price.

Clicking through the data suggests that there may be patterns here but it is difficult to visualise without actually plotting on a graph.

[corradio]

Hi @AndyinHawick ,

This sounds like a notebook to be added in the datascience folder! Are you up for the task?

Olivier

[AndyinHawick]

HI Oliver Unfortunately, this is not a language that I am familiar with but I'm happy for anyone else to run with the idea. Andy

…

On 27 Jan 2017, at 21:36, Olivier Corradi ***@***.***> wrote: Hi @AndyinHawick <https://github.com/AndyinHawick> , This sounds like a notebook to be added in the datascience folder! Are you up for the task? Olivier — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYNhKJlGdtJ22AWdoPad1IjjuXlfDBSzks5rWmN1gaJpZM4LpB3H>.

[corradio]

On top of my mind there's several curves we could show over the last X hours:

• CO2 intensity
• Production + import/export (stacked per production unit and per import/export)

A simple toggle would do I think. Is there anyone up for trying this? I will be ready with the backend quite soon. @nikkozzblu and I looked at c3 but we either need to port it to d3v4 or code our own graph (which I think we should do instead).

We should probably write a small module that can plot a stacked (area) timeseries, with a small tooltip and vertical bar.

[nikkozzblu]

Maybe 2 graphs ? One co2 + productions stacks One prices + import/export stacked ? Nico Juguet

…

-------- Mensaje original -------- Asunto: Re: [corradio/electricitymap] Add timeseries graphs (#305) De: Olivier Corradi Para: corradio/electricitymap CC: nikkozzblu ,Mention On top of my mind there's several curves we could show over the last X hours: * CO2 * Prices * Production (stacked per production unit) * Import/Export (stacked per country) That's a lot of curves. How can we deal with this? - You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#305 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/APk4-yig5y6W_euXAcjq8KcKQ8WTc64Sks5rXzD-gaJpZM4LpB3H>.

[corradio]

My proposal:

• one graph with co2 intensity (g/kWh) with a gradient background representing the co2 scale
• one graph with production+import/export stacked (kW)
  We should leave prices out for now as they are not that interesting compared to other metrics.

[martindaniel4]

That's a lot of information for the left panel. Some countries (DE, FR etc..) have a lot of imports / exports, leaving less real estate for those graphs. Also on smaller screen (most of desktop screen I think), the left panel is already taking full window height. Here are few ideas to display all those info: - Display graph in tooltip when you mouseover on a country - Have a toggle / button to expand the import / export panel. What do you think? 2017-01-31 8:50 GMT-08:00 Olivier Corradi <[email protected]>:

…

My proposal: - one graph with co2 intensity (g/kWh) with a gradient background representing the co2 scale - one graph with production+import/export stacked (kW) We should leave prices out for now as they are not that interesting compared to other metrics. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAqa3ozUJ25Rw39sdoEWtRo13yjYXwhiks5rX2ZZgaJpZM4LpB3H> .

-- Martin DANIEL @martindaniel4 + 1 415 910 6340

[AndyinHawick]

My interest remains in the correlation (or not) of carbon intensity and price (settlement price ideally but day-ahead would be a reasonable proxy or even day-ahead from the previous day). Looking at the various data points, there seems to be a reasonable amount of alignment between low price and low carbon intensity. If the data show that in these real-life situations, the low-carbon options are the cheaper sources (even averaged over each country), that would give confidence to the industry that it is also good business to maximise low-carbon generators. My suggestion would be an X-Y plot of CO₂ against price with a dot for each country (perhaps sized according to power) and perhaps with previous data points behind in decreasing colour intensity, giving the effect of older data fading into the background. Andy

…

On 31 Jan 2017, at 1837, Martin Daniel ***@***.***> wrote: That's a lot of information for the left panel. Some countries (DE, FR etc..) have a lot of imports / exports, leaving less real estate for those graphs. Also on smaller screen (most of desktop screen I think), the left panel is already taking full window height. Here are few ideas to display all those info: - Display graph in tooltip when you mouseover on a country - Have a toggle / button to expand the import / export panel. What do you think? 2017-01-31 8:50 GMT-08:00 Olivier Corradi ***@***.***>: > My proposal: > > - one graph with co2 intensity (g/kWh) with a gradient background > representing the co2 scale > - one graph with production+import/export stacked (kW) > We should leave prices out for now as they are not that interesting > compared to other metrics. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#305 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AAqa3ozUJ25Rw39sdoEWtRo13yjYXwhiks5rX2ZZgaJpZM4LpB3H> > . > -- Martin DANIEL @martindaniel4 + 1 415 910 6340 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AYNhKFnvxTORmjawTUf6cty88bbt91Rdks5rX39kgaJpZM4LpB3H>.

[corradio]

@AndyinHawick, I'm opening an issue specifically for this. We won't put it in the realtime electricity map, because the analysis you're looking for needs to be done on more than a couple of hours (probably weeks/months). We might however put a time series of price, but not a scatter plot. Issue: #336

@martindaniel4 here's some ideas:

Graph in tooltip:

• Works well for isolated time series (so good for CO2 intensity for example)
• Hard to compare with others, so not so good for each production mode
• Impossible to hover in order to get more detailed quantifications of past values
• Would be nice to click on a point in the past and show the map at that time

What about making the left panel scrollable? Yes we can add a toggle, but I have a hard time seing it work. I'll try to prepare a branch with some ideas. Feel free to do the same!

[corradio]

I'm starting on this today.

[martindaniel4]

@corradio do you have a way to pull data locally?

Looks like the feeder is only fetching data for current datetime

[martindaniel4]

Ok obviously it looks like you thought about everything already and you have a toggle to fetch data from remote endpoint ;) - https://github.com/corradio/electricitymap/blob/master/web/app/main.js#L33

[corradio]

Yeah keep in mind that the feeder can't grab data historically, so it needs to run on your laptop for a day to get a day of history :(

…

On Mon, Feb 6, 2017 at 12:57 AM, Martin Daniel ***@***.***> wrote: Ok obviously it looks like you thought about everything already and you have a toggle to fetch data from remote endpoint ;) - https://github.com/corradio/electricitymap/blob/master/web/app/main.js#L33 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#305 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABlEKOY_PenDML2HEochVpBpViJ49DJEks5rZmHfgaJpZM4LpB3H> .