Sidebar filtering of data

[jameshadfield]

This PR shows some very quick work (sub 1 hour) to implement a typing-based filtering system into auspice.

Background

I think it's well accepted that the current system of listing filters in the footer can be improved. @ivan-aksamentov has implemented a faceted-table approach visible at clades.nextstrain.org and there have been a few discussions about ways to incorporate this into auspice. We also have a "strain select" search box in the sidebar, with the limitation that only one strain can be selected at a time.

My use case (and I suspect many others') is that I know what I want to filter by, and find it frustrating to scroll down, visually search a reasonably large space, and click the appropriate element. I also often want to filter by categories which the dataset maintainer has not exposed as "filters".

This solution

This implements a typing-based filtering system into the main sidebar of auspice. If you want to filter to North America for instance, you could quickly type "amer" and be presented with

Currently the implementation only uses the same data as the footer-displayed filters, as this was quick to do.

Proposal

The filtering rules would be the same as currently (take the union of entries within each category, and then take the intersection of those unions). If people agree this is a good direction, I'd also implement it for

• the sample names, thus eliminating the "Search Strains" box and allowing us to select multiple samples (useful in of itself)
• all of the colorings (not just those selected as filters). For continuous traits, I'd plan on using the groupings shown in the legend.

This does not necessarily replace plans for a filtering UI similar to clades.nextstrain.org, but instead provides a solution which allows users who know what they want to get there extremely fast.

cc @rneher @trvrb

[ivan-aksamentov]

@jameshadfield Cool! Let me know how I can help.

I think @trvrb wanted something alternative to the top sliding panel, and to be honest I don't entirely like it either. What are the options in terms of user experience you are considering?

If we go with the same filtering sub-panels (filtering groups) as in Nextclade, but put them into sidebar, it may work. But the sidebar will probably overflow by height and thus will have to be scrollable. Which makes it into scrollables-inside-scrollable kind of thing - always mildly infuriating to use.

Switching sidebar between normal and filtering mode, as Trevor suggested, may free up some vertical space for filters, but on most screens (1080p and smaller) it will probably still overflow.

Maybe we could make filter groups collapsible?

Another problem I see immediately, even in your prototype is that some entries are way too long to fit in sidebar width. This will have to be addressed somehow too. My solution in Nextclade is to ellipsis-truncate and add a tooltip with full string, but that's not perfect at all.

Additionally, instead of adding stuff to the right sidebar, we might consider adding an additional collapsible sidebar on the left, to avoid switching.

[emmahodcroft]

I've only played with this a little, but I like it! It's very straightforward to use and space-saving. I like it in addition to the 'blue box of text' as you could both look to see the options available (sometimes this is still useful) - and the counts! (I do actually sometimes find those very useful!) But can quickly get to the filtering settings you want, if you already know them.

[rneher]

I like it too. would definitely never use the paragraphs again... One worry I have is that it is not super obvious what filters act as unions and what as intersections. I guess this could be made more obvious by grouping/coloring the filter vignettes at the top.

[trvrb]

This is really cool @jameshadfield! I like where this is headed.

Generally, it seems there are two UX desires to be designing for:

1. The ability to quickly and easily filter the data to a desired slice
2. The ability to see categories and counts across categories

I agree with Emma in the usefulness of (2) for quickly scanning through to find countries with lots of samples. I also liked including authors as a filter in the footer as it surfaces list of names like an acknowledgements list.

The current PR provides these functions in Auspice with (1) being a typeahead search box and (2) continuing to have paragraph-style lists at the bottom of the page.

While clades.nextstrain.org accomplishes both (1) and (2) in the filters UI itself (can scroll through each individually to look at what's in them, including counts).

I had previously commented on moving the panels in clades.nextstrain.org into the sidebar in order to:

• Better fill out screen real-estate. The row at the top pretty strongly compresses the tree. Also, I don't know how to do this in mobile well, whereas the sidebar toggle is mobile compatible.
• Make it easier to deal with variable numbers of filters. If current clades.nextstrain.org UI had 2 filters it would look weird having blank real estate to the right. If it had 10 there would need to be horizontal scrolling which I'm not a huge fan of.

Stacking panels vertically in the sidebar was my suggestion to address both these issues, however, I see the problem with:

If we go with the same filtering sub-panels (filtering groups) as in Nextclade, but put them into sidebar, it may work. But the sidebar will probably overflow by height and thus will have to be scrollable. Which makes it into scrollables-inside-scrollable kind of thing - always mildly infuriating to use.

and capturing vertical scroll. Collapsible panels is probably the right way to address this, but now it's fair number of clicks to toggle to filters list in side bar, expand the relevant panel, start typing to limit options, click to filter. Compared to the same outcome being accomplished with typing into the immediately available generic filter box and clicking on result.

This probably requires more thought, but at this point my immediate preference for Auspice filter UI would be very similar to what's created in this PR with (1) and (2) split between sidebar and main page footer.

My biggest issue with current implementation is that it's surprising to me to click or type into the filter box in the sidebar and then have nothing happen to the sidebar UI element. Instead we get blue pillboxes under the page title after "Showing...". My small suggestion here would be to move these cancelable boxes to directly under the "Filter Data" box. They would stack up vertically and could be dismissed 1 by 1. I like how the clades.nextstrain.org pillboxes have "Country", "Region", etc... in their name.

Get rid of the blue pillboxes following "Showing...". (Not 100% sure here, but this is my suspicion.) Still keep "Showing 543 of 543 genomes sampled between Feb 2013 and Jul 2019" as this provides important broader context. But don't include anything after this. Currently zooming the tree will result in "Showing 29 of 543 genomes..." without an explanation of why the counts are reduced. So it would still fit with the rest of Auspice UI.

In this scenario, I'd suggest to swap Filter Data and Color By in the sidebar ordering. Filter Data and Date Range are doing exactly the same thing just with different affordances and it makes sense to me to have them next to each other. The ordering would then be Date Range, Filter Data, Color By. I like having the visual distinction between the dropdown for Dataset select and the handlebar for Data Range.

Also, I'd change name from Filter Data to Filter By for consistency. "Data" seems to vague to me here.

In this scenario, I'd keep the paragraph filters as they work now in the footer.

the sample names, thus eliminating the "Search Strains" box and allowing us to select multiple samples (useful in of itself)

Fully agree.

all of the colorings (not just those selected as filters). For continuous traits, I'd plan on using the groupings shown in the legend.

I'm not sure how this would work in some cases, for example: https://nextstrain.org/flu/seasonal/h3n2/ha/2y?c=weighted_distance_to_future_by_cTiter_x-ne_star. You start typing "Future population" and then get 10 options with different bounds to click on? Ie it's better being able to type a value, rather than just a key for numeric values.

This does not necessarily replace plans for a filtering UI similar to clades.nextstrain.org, but instead provides a solution which allows users who know what they want to get there extremely fast.

Agreed. This quick filter UI could exist in sidebar and be readily available, while still having a more full featured UI ala clades.nextstrain.org available if more specificity is desired. This could help for this exact "Distance to future" example above.

[jameshadfield]

Thanks for all the comments @ivan-aksamentov @emmahodcroft @rneher @trvrb

My main response would be that I believe there are two separate use cases for filtering data which will require different UI. The first is tackled by this PR -- I know I want to filter to a country and I want to do it really quickly. The second is for users who want to explore the available filters, see counts etc. The latter necessitates a (visually) larger UI with more clicking / scrolling. Nextclade and the current Auspice footer are examples of such a UI. There have been good points made about such a UI (avoiding scrolling-within-scrolling, avoiding too much clicking) however I am purposefully avoiding offering a solution here! I will say that the "button" to expose this UI could be next to the search-box from this PR, e.g. a "show me all available filters" button.

---

My biggest issue with current implementation is that it's surprising to me to click or type into the filter box in the sidebar and then have nothing happen to the sidebar UI element.

My small suggestion here would be to move these cancelable boxes to directly under the "Filter Data" box.

I have is that it is not super obvious what filters act as unions and what as intersections. I guess this could be made more obvious by grouping/coloring the filter vignettes at the top.

I'll explore how this looks to show the pillboxes below the search box in the sidebar, and completely agree that some visual cues as to how the individual selections interact will be beneficial. This will also free up space in the header which could be nice.

---

Another problem I see immediately, even in your prototype is that some entries are way too long to fit in sidebar width. This will have to be addressed somehow too.

Good point! This happens in the strain-search box and they "pop-out" which is functional but doesn't look very nice.

---

You start typing "Future population" and then get 10 options with different bounds to click on? Ie it's better being able to type a value, rather than just a key for numeric values.

Yes, this is less than ideal. I think exposing them will be better than not exposing them, but agree there are future improvements over this beyond the scope I'm considering here -- you could imagine using typed queries, or when you've written 14 you see "Distance to future population: 12-16" as an option.

[emmahodcroft]

Just a comment on the pillboxes appearing in the sidebar - I'm a little hesitant to make the sidebar even longer. It's already kind long and I feel like when working on my laptop I spend a lot of time scrolling up and down, I'm a little unsure of exacerbating that when you've got a lot of filters applied. I also find it pretty handy that when viewing the trees you can always remove them right there, very conveniently.
I wonder if perhaps in the sidebar just something like 'X filters applied' could appear as you apply more filters, but the pillboxes themselves stay at the top?

what filters act as unions and what as intersections. I guess this could be made more obvious by grouping/coloring the filter vignettes at the top.

I also really like this idea! It would be great to better expose when filters work together or not

[trvrb]

I wonder if perhaps in the sidebar just something like 'X filters applied' could appear as you apply more filters, but the pillboxes themselves stay at the top?

Good point @emmahodcroft. I'd be okay with this split scenario. It just felt weird typing into and selecting in search box in sidebar and then nothing happening to the UI element that was interacted with.

Though generally, I don't know if pillboxes would be better in sidebar or header without prototyping.

[ivan-aksamentov]

selecting in search box in sidebar and then nothing happening to the UI element that was interacted with

We might implement a dropdown that contains the list with checkboxes. This will provide visual feedback on select and, optionally, will also allow to choose multiple items at once. The filtering itself could be applied on checkbox click or on blur (clicking away).

It's a bit trickier to implement, but I think it should be possible with react-select (the lib that is currently used).

We could probably also make custom dropdown wider than the input box or sidebar. Should watch out for mobile experience in this case.

[jameshadfield]

Thanks @ivan-aksamentov @trvrb @emmahodcroft @rneher for helpful comments both here and elsewhere! Excited to get this out there, and will create issues for remaining tasks :)