Violins and Boxes in subplots oddly offset

[nicolaskruchten]

Split off of #3390 ...

the image below (see https://plot.ly/~nicolaskruchten/260) shows that each subplot "leaves room" for traces that are present only in other subplots, leading to undesirable horizontal offsets:

[nicolaskruchten]

Upon further reflection, I realize why this is: there is nothing saying that the traces in the vertically-offset subplots are actually the same as the other ones and hence should be aligned. It only looks that way because they're coloured the same. I haven't tried it but I assume the same thing will happend with grouped bars.

I think this calls for a new cross-trace argument allowing for grouping of groups or something. I've already used legendgroup above to link the traces' visibilities together, and I think we need a new mechanism for linking offset/grouped categorical traces across subplots (this doesn't affect numerical axes, or overlaid/stacked situations).

[nicolaskruchten]

Interestingly, this does not happen with grouped bars:

[nicolaskruchten]

(although the order of the bars in the lowest subplot is odd) (this was a difference in the figure JSON, please ignore this comment, and the image above has been corrected)

[nicolaskruchten]

After further thought, given that we already do what seems like the right thing for bar, I think we should align box and violin with this same behaviour.

[nicolaskruchten]

After even further thought, I don't think that bar is doing the "right thing" by default here either in terms of vertically-offset subplots with shared X axis.

My proposal would be to introduce a new attribute for bar, box and violin traces called alignmentgroup (I wanted to call it groupgroup initially :P) that would serve to let the layout system know that these elements need to be vertically aligned (in v orientation).

[nicolaskruchten]

To be a bit clearer, here are two plots: same except that one is bar and one is violin.

Right now there's no way to have the bars offset horizontally among the subplots if e.g. traces 0 and 2 (blue and green) are un-related. Similarly, there's no way to align the violins such that trace 0 is vertically aligned with trace 2 if the are related.

I propose to resolve this with the alignmentgroup attribute. If it's null for all traces, just do the current thing. If it's set, then lay out such that things in the same group are always aligned, and things not in the same group are never aligned. If any trace has an alignmentgroup then we could assume that the nulls mean "all different" as a reasonable default.

Note that I would expect that when #1549 is implemented, that alignment would take into account matched-but-not-colinear axes, such that "room is left" for unaligned traces in non-colinear subplots.

[alexcjohnson]

related: #78

[nicolaskruchten]

Hmmm. #78 confuses me a bit, but I think my proposal is a way forward for some of those cases?

[etpinard]

I might be nice to combine @nicolaskruchten 's alignmentgroup proposal with an attempt to augment trace-layout attributes (e.g. barmode and boxmode) to something similar to scatter's stackgroup like we discussed when specing out stacked area.

It would also be nice to find a solution that would allow us to plot stacked-grouped bar charts (cc #1835).

From its description, looks like stackgroup doesn't work across subplots, so these new group attributes would have to be a little different. From #78, we might even have to add special logic for overlaying axes 😓

This might be worth discussing during tomorrow's meeting.

[nicolaskruchten]

Wow that seems like a big chunk of work! Let’s discuss :)

…

On Wed, Jan 16, 2019 at 18:23 Étienne Tétreault-Pinard < ***@***.***> wrote: I might be nice to combine @nicolaskruchten <https://github.com/nicolaskruchten> 's alignmentgroup proposal with an attempt to augment trace-layout attributes (e.g. barmode and boxmode) to something similar to scatter's stackgroup like we discussed when specing out stacked area <#1217>. It would also be nice to find a solution that would allow us to plot stacked-grouped bar charts (cc #1835 <#1835>). From its description <https://github.com/plotly/plotly.js/blob/master/src/traces/scatter/attributes.js#L83>, looks like stackgroup doesn't work across subplots, so these new group attributes would have to be a little different. From #78 <#78>, we might even have to add special logic for overlaying axes 😓 This might be worth discussing during tomorrow's meeting. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3402 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMbA7XT5tUj7F8OLWe2BfSPjAJR_4q3ks5vD7SCgaJpZM4ZxuRz> .

[nicolaskruchten]

Just so I understand, @etpinard you're thinking that we could model cross-subplot alignment as a kind of stacking and rolling it into stackgroup?

[alexcjohnson]

stackgroup doesn't make sense for box and violin, and alignmentgroup doesn't make sense for scatter - we could probably get stackgroup to work for bar but it seems to me alignmentgroup is the more precise meaning there anyway, so I think we should use that for bar, box, and violin and not try to shoehorn scatter into the same attribute, and as I'll describe below I think bar can make use of BOTH alignmentgroup and stackgroup.

alignmentgroup

So how exactly would that work? I'm going to use our internal nomenclature of position axis = x for vertical orientation, y for horizontal. If I'm understanding your proposal right @nicolaskruchten, any traces with the same alignmentgroup, regardless of trace type or what subplot they're on, get the same position shift, and that position shift should be DIFFERENT from any traces with a different alignmentgroup.

One place I'm not totally clear on the correct behavior is when we're talking about not just a different subplot, but a different position axis. Certainly if the two axes have different types it wouldn't make sense to match alignments between them, but even for the same type I don't think it makes sense, at least not always - let's say you have two date axes, one aggregating data monthly and another daily. Or two numeric axes with grouped histograms on totally different variables, with different bin sizes. I suppose we could try to translate alignment between the two as consistent fractions of the position difference between successive items but that feels a bit magical, I think I'd prefer distinct position axes to handle alignment independently of each other. That said, when we add matching axes #1549 I think there's a strong case that these should count as the same axis for alignment purposes.

alignmentsupergroup or something

One other case to consider: what if you want independent alignment in different subplots with the same position axis? Say you have two stacked subplots sharing an x axis, with one bar trace on the top, that you want to have fill the entire space, and several traces on the bottom that you want grouped. That particular chart I can see being both realistic and intuitive. If we extend this idea a little, say you have 2 traces on top and 5 on the bottom, again you want to fill the whole space on each subplot rather than making space for all 7 on each subplot. Now it doesn't seem particularly easy to read a chart like that, but it's going to come up and it's what we do today with data like that. The only way I see to support something like that is a second attribute, something like alignment(batch|collection|supergroup) that you'd set to the same value within a subplot and a different value for other subplots. That would also be needed if we make alignmentgroup apply across trace types (which I think we should do) but you put say one set of boxes on the same subplot as two sets of bars and again you want each trace type to fill the space.

You could also use this attribute to make consistent alignment across subplots that we would otherwise treat independently - as long as the position axis has the same type.

Stacking

Then for bars only, how does this relate to stacking? With no more attributes it seems like with barmode: 'stack|group|relative' we would stack any traces on the same subplot in the same alignmentgroup, but with barmode: 'overlay' we stack nothing. Then later we could add a stackgroup attribute to bar traces, so that any traces on the same subplot with the same stackgroup - REGARDLESS of their alignmentgroup - would be stacked. That would let you do per-trace overlay (different stackgroup and same alignmentgroup) or offset stacking (waterfall-ish - same stackgroup and different alignmentgroup - then barmode: 'stack' with negative bars would actually be useful and mostly not confusing!)

Implementation

I believe the current default behavior can be preserved as we add these options just by appropriately choosing default values for the new attributes as we add them, then ONLY using those attributes during rendering, and ignoring the layout-wide (bar|box|violin)mode. We'll still need additional per-trace gap attributes, to be applied across the appropriate group based on the first value we find... but we can leave that for later.

[etpinard]

To (attempt to) sum up @alexcjohnson's #3402 (comment)

would be declared as

[{
  x: ['A', 'B', 'C', 'D'],
  y: [1, 2, 3, 4],
  alignmentgroup: 0
}, {
   x: ['A', 'B', 'C', 'D'],
   y: [2, 3, 1, 5],
   alignmentgroup: 1
}, {
  x: ['A', 'B', 'C', 'D'],
  y: [1, 2, 3, 4],
  yaxis: 'y2',
  alignmentgroup: 0
}, {
   x: ['A', 'B', 'C', 'D'],
   y: [2, 3, 1, 5],
   yaxis: 'y2',
   alignmentgroup: 1
}]

and

as

[{
  x: ['A', 'B', 'C', 'D'],
  y: [1, 2, 3, 4],
  alignmentgroup: 0
}, {
   x: ['A', 'B', 'C', 'D'],
   y: [2, 3, 1, 5],
   alignmentgroup: 1
}, {
  x: ['A', 'B', 'C', 'D'],
  y: [1, 2, 3, 4],
  yaxis: 'y2',
  alignmentgroup: 2
}, {
   x: ['A', 'B', 'C', 'D'],
   y: [2, 3, 1, 5],
   yaxis: 'y2',
   alignmentgroup: 3
}], {
  // N.B. need to set this
  // as the violinmode dflt is 'overlay'
  violinmode: 'group'
}

But something like

[{
  y: [1, 2, 1],
  yaxis: 'y2'
}, {
  y: [2, 1, 2]
}, {
  y: [1, 3, 0]
}]

which currently looks like:

couldn't be declared using just alignmentgroup and would need something like:

[{
  y: [1, 2, 1],
  alignmentsupergroup: 'top'
  yaxis: 'y2'
}, {
  y: [2, 1, 2],
  alignmentsupergroup: 'bottom',
  alignmentgroup: 0
}, {
  y: [1, 3, 0],
  alignmentsupergroup: 'bottom',
  alignmentgroup: 1
}]

[nicolaskruchten]

let's say you have two date axes, one aggregating data monthly and another daily. Or two numeric axes with grouped histograms on totally different variables, with different bin sizes.

I don't think alignmentgroup makes sense for histogram, and we can't group histograms anyway, so I'm not at all sure what the concern is here :)

[nicolaskruchten]

re the alignmentsupergroup thing, I propose we continue to support the current behaviour (of bar!) if all alignmentgroups on the same position axis are null.

[nicolaskruchten]

we can't group histograms

Actually of course we can, sorry, brainfart. I still don't think alignmentgroup makes sense for histograms, and I think it should only apply across the same position axis (colinear as we have now, or non-colinear with a future matched-axis feature)

[nicolaskruchten]

Looks like grouped histograms already do the right thing here IMO:

Although I think it would be nice if they could share autobin logic in the case of a shared position axis (colinear or not):

[nicolaskruchten]

(just to clarify my comment on alignmentsupergroup: I'm trying to say I don't think we need to implement it in the short run, as the current behaviour could be grandfathered in)

[nicolaskruchten]

Hmm, upon further thought, looking at that last chart, given that I was in group mode, I think I see the case for alignmentgroup in histograms: I might have expected to see these "leaving room" for each other, or have two grouped histograms that represent A&B and B&C.

Secondly, I wonder if the "one trace above, two below" case isn't modelable by having them all have the same alignmentgroup? Otherwise what does it mean for two traces on the same subplot to have the same alignmentgroup ?

[alexcjohnson]

just to clarify my comment on alignmentsupergroup: I'm trying to say I don't think we need to implement it in the short run, as the current behaviour could be grandfathered in

I feel like it would be just as easy - possibly even easier - to add the full functionality in one go, with the default values set to match current behavior. Then we don't need to carry around a special algorithm for the grandfathered case, and as a bonus we're done with this feature. Implementation can proceed by first adding a bunch of test cases of the edge cases we've identified in the current behavior (ensuring of course that we don't consider them bugs!) then adding the new behavior and ensuring the new (and existing) test cases all still pass.

Secondly, I wonder if the "one trace above, two below" case isn't modelable by having them all have the same alignmentgroup? Otherwise what does it mean for two traces on the same subplot to have the same alignmentgroup ?

If two traces (on the same position axis) have the same alignmentgroup then their positions are offset identically wrt the data values. I think it's that simple. So if those two traces are on the same subplot, they either stack or overlay depending on barmode or its per-trace successor, and if they're on different subplots then they're aligned.

Still not so happy about the names... what about alignmentsupergroup -> alignmentgroup and alignmentgroup -> alignmentkey?

[nicolaskruchten]

If two traces (on the same position axis) have the same alignmentgroup then their positions are offset identically wrt the data values. I think it's that simple. So if those two traces are on the same subplot, they either stack or overlay depending on barmode or its per-trace successor, and if they're on different subplots then they're aligned.

So for bar in single-subplot context with barmode set to group, what does it mean for two traces to have the same alignmentgroup?

[nicolaskruchten]

OK so just to sum up the verbal conversation we just had (assuming orientation is vertical):

• offsetgroup controls whether bars occupy the same positional range as each other or not
• stackgroup controls whether bars are stacked on top of one another
• alignmentgroup controls whether bars compute their positional range dependently or independently
• barmode would be recast as providing different default values to these, and therefore becomes trace-overrideable
• the foregoing will apply to bar and histogram and we need to think a bit more about coordinated binning logic
• box and violin get the same treatment except for stackgroup which doesn't apply

So we need to crisp up the definitions of offsetgroup and alignmentgroup a bit but I think we all have a similar mental model at this point.

[nicolaskruchten]

Note: we'll also need a way of doing relative stacking here like we currently do with boxmode

[nicolaskruchten]

Here are the photos we discussed over Slack. Note that the bottom-right quadrant was eventually considered not worth supporting (stacking on top of bars in another subplot)

if we consider 2 bar traces with 1 datum x=1/y=1
and 4 variables: yaxis, offsetgroup, stackgroup and alignment group
and we look at all the permutations of equal vs unequal we get:

here below the orange boxes have the SAME alignmentgroup/yaxis and DIFFERENT offsetgroup/stackgroup (so they always render like the top-left box of the original diagram)
so the zones refer to the relationship between the black box and the right-most orange box:

[etpinard]

Some progress:

[nicolaskruchten]

hehe, so awesome to see whiteboard-to-test-case!

[etpinard]

@nicolaskruchten what are the minimal requirements for this ticket?

I'm thinking for v1.45.0 to:

• ✔️ add alignmentgroup and offsetgroup to bar, histogram, box and violin traces. Note that alignmentgroup and offsetgroup only have an effect if layout.(bar|box|violin)mode: 'group'. This will be enough to resolve the discrepancies shown in Violins and Boxes in subplots oddly offset #3402 (comment)
• ✔️ group across "matching" position axes cc Implementing matching axes #3506
• ⛔ leave stackgroup for later
• ⛔ leave per-group (bar|box|violin)mode, (bar|box|violin)gap, (bar|box|violin)groupgap for later

[nicolaskruchten]

I’ll have to think about what it would mean to leave out stacking... for PX we don’t need bars to stack spanning each other and we don’t need mixes of groups and stacks but we do need aligned stacks. I’ll try to demo what I mean.

[nicolaskruchten]

After some thought, I think your proposal works for me for 1.45.0, and we don't need stackgroup in the very short run. PX works fine with stacking behaviour as it stands as far as I can tell.