Gadfly > Plotly converter

[jackparmer]

Hello! This is Jack from Plotly.

We would love to help extend Gadfly's plotting capabilities. Plotly.js is the graphing library that powers Plotly's Python client and is now 100% open-source under the MIT license:
https://plot.ly/javascript/open-source-announcement/

With Python we've taken the approach of having both a Plotly native API:
https://plot.ly/python/

And a matplotlib figure converter:
https://plot.ly/matplotlib/

All Plotly charts are based on a JSON schema that makes chart and data interchange between Python, Julia, R, MATLAB and other supported languages easy. It also makes it straightforward to write clients for new languages:
http://help.plot.ly/json-chart-schema/

We'd love to see something similar with Julia and Gadfly. Plotly's Julia client lives here:
https://github.com/plotly/plotly.jl

Though it is greatly out-of-date compared to the Python client and no Gadly > Plotly figure converter currently exists.

If this is of interest to the community, Plotly could help sponsor the work. Is anyone excited about helping out with this?

There are a few potential areas for contribution in my mind:

1. Develop a Gadly > Plotly figure converter akin to https://plot.ly/matplotlib/ (GitHub: https://github.com/plotly/plotly.py) or https://plot.ly/ggplot2/ (GitHub: https://github.com/ropensci/plotly)
2. Mature the native Plotly Julia client to the level of Python one (https://plot.ly/python/ GitHub: https://github.com/plotly/plotly.py)
3. Flesh out documentation for the Gadfly > Plotly converter akin to https://plot.ly/python/ to get folks excited about Julia, Gadfly, and Plotly

Would love to hear any thoughts or feedback. My email is jack[at]plot[dot]ly if you want to chat offline.

Thanks for reading!

[tbreloff]

Hi @jackparmer thanks for the post. It seems as though you're not aware of some of the current efforts to access Plotly from Julia.

@tbreloff Plots.jl

lots of support... missing subplots, embedding in IJulia notebooks, and a few other things, but much of it is already supported. See: JuliaPlots/Plots.jl#77 and also you can test it out yourself:

Pkg.add("Plots")
Pkg.checkout("Plots")
Pkg.clone("https://github.com/tbreloff/ExamplePlots.jl")
using ExamplePlots
ExamplePlots.test_examples(:plotly)

@joshday PlotlyLocal

https://github.com/joshday/PlotlyLocal.jl

@EricForgy PlotlyJS

https://github.com/EricForgy/PlotlyJS.jl

@spencerlyon2 Plotlyjs.jl

https://github.com/spencerlyon2/Plotlyjs.jl

We're probably going to attempt to consolidate efforts a little... please let us know how you'd like to help!

[EricForgy]

For more details about my initial efforts with Julia and Plotly, you can refer to this julia-users post:

https://groups.google.com/d/msg/julia-users/ix5fxt68rwM/Fry-zRVICQAJ

And just for fun, I can't resist posting this video of me streaming plotly charts to 4 different browser controlled from the Julia REPL :D

https://www.youtube.com/watch?v=mWDyyfVNqP0

As a startup founder scrambling to put a product together and father of 3, my efforts need to be a little selfish, i.e. if helping with this effort helps me (which I think it could), then I'm happy to do what I can.

[jackparmer]

Nice! Looks great! Thanks the detailed update, this was fun to read through.

For subplots, I recommend taking the extra time to write a helper routine. From our experience with the Python and R clients, we found that subplots / facets with the declarative syntax were too cumbersome. No one made them until we supported a helper routine.

This may quickly become more mature than Plotly's current Julia client in Github... At that point we may switch or merge them and can help write docs!

[rohitvarkey]

@spencerlyon2 is also working on getting a Plotly Julia client up which has Blink and IJulia integration.

https://github.com/spencerlyon2/Plotlyjs.jl

Develop a Gadly > Plotly figure converter akin to https://plot.ly/matplotlib/ (GitHub: https://github.com/plotly/plotly.py) or https://plot.ly/ggplot2/ (GitHub: https://github.com/ropensci/plotly)

I think the way to go would be to write a Plotly backend for Gadfly Compose once one of the Julia to Plotly packages have enough features to support Gadfly. So then just calling draw with the Plotly backend would generate a Plotly image.

[sglyon]

Thanks for brining me in to the conversation @rohitvarkey -- I'm happy to pitch in here

I have been doing some work over there at my version of plotlyjs (btw I think we should only have one Julia package for exposing plotly.js, but that's a conversation for outside of the Gadfly issue tracker) and have been pleased with the progress. It is currently possible to generate any chart plotly.js can render. I moved over all the scatter/line examples as a proof of concept

Sorry if this is a naive question, but what would the goal/purpose to of the Gadfly > Plotly converter be other than having a different rendering engine? Would doing this somehow extend Gadfly's capabilities to be the union of current Gadfly and Plotly.js features?

[tbreloff]

Sorry @spencerlyon2 for not including you in the list. I actually didn't realize you made so much progress!

[jackparmer]

@spencerlyon2 @tbreloff Nice! Such a delight for us to look through all this. Thanks for summarizing!

what would the goal/purpose to of the Gadfly > Plotly converter be other than having a different rendering engine?

I might be misunderstanding Gadfly, but there wouldn't be another purpose besides another rendering engine. Julia folks seem to know and like the Gadfly syntax and the aesthetic is really nice, so providing the option to publish these charts to D3.js / Plotly with the Gadfly syntax seems like a cool feature for folks who don't want to learn another charting syntax. Plotly.js has most of the graphics capability of MATLAB or matplotlib, so there's potentially a lot of graphics Julia users can pick up for free that Gadfly might not be otherwise have available right now (like interactive 3D charts and maps).

[sglyon]

@jackparmer thanks for clarifying. This seems very cool. Another thought I had was that we could use gadfly's syntax to generate most of the plot and then move to plotly where we have more options for tweaking details and customizing.

Sounds like a good contribution.

[Godisemo]

I'd love to see a way of using gadfly syntax and getting plotly plots.

[tbreloff]

FYI there's some additional discussion happening here: JuliaPlots/Plots.jl#81

@jackparmer In particular, I could really use some help/advice in generating PNGs automatically, as well as the right way to update plot components after-the-fact.

Thanks for a great package btw, and for releasing the javascript/json version.

[sglyon]

@tbreloff I update after the fact via a js call.

See here: https://github.com/spencerlyon2/Plotlyjs.jl/blob/master/src/display.jl#L88-L98

[jackparmer]

Hey @tbreloff ! There is some current work and discussion on easier (offline) PNG programmatic access here by @etpinard

plotly/plotly.js#83

Let us know if that looks like it will help!

[jackparmer]

@Godisemo @dcjones I might be misunderstanding Gadfly, but I was thinking having "PLOTLY()" in addition to "D3()" in the draw() command might be a cool way to add plotly rendering:

http://gadflyjl.org/

So many people seem to know and love the Gadfly syntax, it seems like it would be cool if this was all that was necessary to bring Plotly chart rendering into the loop.

Does this seem like a potential longer-term approach?

[Godisemo]

@jackparmer That makes sense from a usability perspective. I guess this would have to be defined in Gadfly instead of compose like all other backends though.

[tbreloff]

Yes it doesn't quite make sense to me how you add a Plotly backend to Gadfly. The multiple backends are through Compose, which requires low level drawing primitives/commands. Adding a backend to Compose implies you don't know about the high level plot structure, only the low level shape/color compositions. Adding as a backend to Gadfly would short-circuit 90% of the package, maintaining only what I consider to be the weakest part of Gadfly... The interface. Plus I think it would be a substantial undertaking to actually get it working well.

Summary: Plotly shouldn't be a part of Gadfly, it should be an alternative to Gadfly.

On Dec 14, 2015, at 8:51 AM, Godisemo [email protected] wrote:

@jackparmer That makes sense from a usability perspective. I guess this would have to be defined in Gadfly instead of compose like all other backends though.

—
Reply to this email directly or view it on GitHub.

[Godisemo]

When I started planning on how to improve the interface of Gadfly a couple of months ago I used Plotly as a reference. It is no doubt the best plot interface I have ever encountered. Now when Plotly have been open sourced I see no reason not to try to incorporate it into Gadfly. For me, the interface alone is enough to stop using Gadfly if Plotly was a separate package with decent plotting syntax.

I disagree with you on this point @tbreloff that it should be an alternative. It could possibly be a separate package in the form of an add-on. Hopefully we can find a solution that works well from a usability, as well as from a maintenance, point of view.

[tbreloff]

@Godisemo These seem to be contradictory?

I see no reason not to try to incorporate it into Gadfly

For me, the interface alone is enough to stop using Gadfly if Plotly was a separate package with decent plotting syntax

And can you expand how you envision this might work:

It could possibly be a separate package in the form of an add-on.

[Godisemo]

It was not meant to be read as a contradiction, but rather an optional approach.
I have not had time to think about details of how this would be done yet.

[robsmith11]

Is this a realistic approach to getting a Julia "grammar of graphics" to Plotly workflow working?

ggplotly is the only reason I still keep R around. I haven't found a suitable replacement for it with Julia yet.